# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file Crawl-Delay: 10 Sitemap: http://www.polyteknisk.dk/sitemap.xml User-agent: pricecloud Disallow: / User-agent: Baiduspider Disallow: / User-agent: baiduspider Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: Baiduspider-video Disallow: / User-agent: Baiduspider-image Disallow: / User-agent: YandexBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: sistrix Disallow: / User-agent: EC2LinkFinder Disallow: / User-agent: netEstate NE Crawler Disallow: / User-agent: Mail.Ru Disallow: / User-agent: spbot Disallow: / User-agent: EasouSpider Disallow: / User-agent: SeznamBot Disallow: / User-agent: Sosospider Disallow: / User-agent: moget User-agent: ichiro Disallow: / User-agent: NaverBot Disallow: / User-agent: Yeti Disallow: / User-agent: sogou spider Disallow: / User-agent: YoudaoBot Disallow: / User-agent: * Disallow: /Leo_pharma Disallow: /DTU Disallow: /haldor_topsoe Disallow: /novo/ Disallow: /dong/ Disallow: /home/dong/ Disallow: /test/ Disallow: /images/ Disallow: /system/ Disallow: /danskebank/ Disallow: /ferring/ Disallow: /pricewaterhousecoopers/ Disallow: /current_boxes/ Disallow: /content/rate_item/ Disallow: /home/nautilus/abonnent/ Disallow: /home/scion/