#Tubular World - robots.txt User-agent: Slurp Crawl-delay: 360 User-agent: ISSB Website Loader 1.1.35 Disallow: / User-agent: Pockey-GetHTML/4.11.6 Disallow: / User-agent: STEROID Download Disallow: / User-agent: Exabot/2.0 Disallow: / User-agent: SBIder Disallow: / User-agent: ConveraCrawler/0.8 Disallow: / User-agent: ConveraCrawler/0.9d Disallow: / User-agent: NutchCVS Disallow: / User-agent: BlitzBOT @ tricus.net Disallow: / User-agent: ia_archiver Disallow: / User-agent: Xaldon WebSpider Disallow: / User-agent: EmailCollector Disallow: / User-agent: Roverbot Disallow: / User-agent: WebCopier Disallow: / User-agent: WebCopier v4.3 Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Download Ninja Disallow: / User-agent: k2spider Disallow: / User-agent: wget Disallow: / User-agent: WebReaper Disallow: / User-agent: SiteSnagger Disallow: / User-agent: TeleportPro Disallow: / User-agent: Teleport Disallow: / User-agent: HTTrack Disallow: / User-agent: Fetch Disallow: / User-agent: Xenu Disallow: / User-agent: larbin Disallow: / User-agent: * Disallow: /admin Disallow: /bundled-libs Disallow: /cgi-bin Disallow: /css Disallow: /deployment Disallow: /docs Disallow: /download Disallow: /fehler Disallow: /htmlarea Disallow: /impressum Disallow: /include Disallow: /lang Disallow: /misc Disallow: /pics Disallow: /plugins Disallow: /scripts Disallow: /sql Disallow: /suchen Disallow: /templates Disallow: /templates_c Disallow: /uploads Disallow: /vorlage