User-agent: * Disallow: /captcha/ Disallow: /complaint/ Disallow: /vRest/ User-agent: bl.uk_lddc_bot Disallow: / User-Agent: 360Spider Disallow: / User-Agent: 80legs Disallow: / User-agent: Aboundexbot Disallow: / User-agent: AddThis Disallow: / User-agent: AddThis.com Disallow: / User-agent: archive.org_bot Disallow: / User-agent: Affectv Robot Disallow: / user-agent: AhrefsBot disallow: / User-agent: b2w/0.1 Disallow: / User-agent: bitlybot Disallow: / User-agent: BizzInformation Disallow: / User-agent: BLEXBot Disallow: / User-agent: butterfly Disallow: / User-agent: careerbot Disallow: / User-agent: ChangeDetection Disallow: / User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: coccoc Disallow: / User-agent: CompSpyBot Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Crescent Disallow: / User-Agent: discoverybot Disallow: / User-agent: dotbot Disallow: / User-Agent: dotnetdotcom Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Exabot Disallow: / User-agent: ExtractorPro Disallow: / User-agent: Ezooms Disallow: / User-agent: FAST Enterprise Crawler Disallow: / User-agent: Genieo Disallow: / User-agent: grapeshot Disallow: / User-agent: grub Disallow: / User-agent: grub-client Disallow: / User-agent: ia_archiver Disallow: / User-agent: ichiro Disallow: / User-agent: IntegromeDB Disallow: / User-agent: Jetbot/1.0 Disallow: / User-agent: larbin Disallow: / User-agent: linguatools Disallow: / User-Agent: lipperhey Disallow: / User-agent: LNSpiderguy Disallow: / User-agent: looksmart Disallow: / User-agent: magpie-crawler Disallow: / User-agent: MIIxpc Disallow: / User-agent: MJ12bot Disallow: / User-agent: moget/2.1 Disallow: / User-agent: Muscat Ferret Disallow: / User-agent: NetMechanic Disallow: / User-agent: Phantom.js bot Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: ProWebWalker Disallow: / User-agent: proximic Disallow: / User-agent: Python-urllib Disallow: / User-Agent: purebot Disallow: / User-agent: rogerbot Disallow: / User-agent: sistrix Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: SEOMoz Disallow: / User-Agent: sitebot Disallow: / User-agent: SiteSnagger Disallow: / User-agent: ShopWiki Disallow: / User-agent: SMTBot Disallow: / User-Agent: Snoopy Disallow: / User-agent: Sogou Disallow: / User-agent: sogou spider Disallow: / User-agent: Sogou web spider Disallow: / User-Agent: SolomonoBot Disallow: / User-agent: Sosospider Disallow: / User-agent: spbot Disallow: / User-agent: SpiderJack Disallow: / User-agent: Speedy Disallow: / User-agent: stalker Disallow: / User-Agent: SurveyBot Disallow: / User-agent: Teleport Disallow: / User-agent: Telesoft Disallow: / User-agent: thesubot Disallow: / User-agent: thumbshots-de-bot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: UnisterBot Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: WebBandit Disallow: / User-agent: WebCopier Disallow: / User-agent: WebSauger Disallow: / User-agent: WebStripper Disallow: / User-agent: Website Quester Disallow: / User-agent: WebZip Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WBSearchBot Disallow: / User-agent: yacybot Disallow: / User-Agent: YamanaLab-Robot Disallow: / # cinsky bot User-agent: YoudaoBot Disallow: / User-Agent: ZmEu Disallow: / User-Agent: SemrushBot Disallow: / User-Agent: SpiderLing Disallow: / User-Agent: webcrawler Disallow: / User-agent: Arquivo-web-crawler Disallow: / User-agent: MJ12bot Disallow: / User-agent: IDG/IT Disallow: / User-agent: Cliqzbot Disallow: / User-agent: ADmantX Disallow: / User-agent: MixrankBot Disallow: / User-agent: Findxbot Disallow: / User-agent: YetiBOT Disallow: / User-agent: crawler4j Disallow: / User-agent: Scrapy Disallow: / User-agent: metajobbot Disallow: / User-agent: MegaIndex Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: Qwantify Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: LCC Disallow: / User-agent: BDCbot Disallow: / User-agent: DomainCrawler Disallow: / User-agent: DomainCrawler/3.0 Disallow: / User-agent: netEstate NE Crawler Disallow: / User-agent: DomainAppender Disallow: / User-agent: wiederfreibot Disallow: / User-agent: wiederfreibot/1.0 Disallow: / User-agent: CCBot Disallow: / User-agent: LLC Disallow: / User-agent: ltx71 Disallow: / User-agent: Riddler Disallow: / User-agent: DeuSu Disallow: / User-agent: linkdexbot Disallow: / User-agent: ips-agent Disallow: / User-agent: GarlikCrawler Disallow: / User-agent: wotbox Disallow: / User-agent: HybridBot Disallow: / User-agent: GetIntent Crawler Disallow: / User-agent: ias_crawler Disallow: / User-agent: nekstbot Disallow: / User-agent: linkfluence Disallow: / User-agent: AhrefsBot Disallow: / User-agent: oBot Disallow: / User-agent: netseer Disallow: / User-agent: adbeat_bot Disallow: / User-agent: LightspeedSystemsCrawler Disallow: / User-agent: sentibot Disallow: / User-agent: BUbiNG Disallow: / User-agent: i4ds-bot Disallow: / User-agent: Eyeotabot/1.0 Disallow: / User-agent: LumtelBot/1.0 Disallow: / User-agent: TTD-Content Disallow: /