# /robots.txt for http://www.bcs.rochester.edu/ # comments to webmaster@bcs.rochester.edu # February 21, 2002 # Last modified: 4/29/2002 #Disallow all robots from restricted files and directories User-agent: * Disallow: /ARCHIVES/ Disallow: /cgi-bin/ Disallow: /private/ Disallow: /Documentation/ Disallow: /info/ Disallow: /JRun/ Disallow: /logs/ Disallow: /Plug-Ins/ Disallow: /Plug-Ins (disabled)/ Disallow: /Tools & Examples/ # Disallow Inktomisearch which was still going into disallowed areas on broca # 4/5/2000 User-agent: Inktomi Disallow: /ARCHIVES/ Disallow: /cgi-bin/ Disallow: /private/ Disallow: /info/ Disallow: /Documentation/ Disallow: /JRun/ Disallow: /logs/ Disallow: /Plug-Ins/ Disallow: /Plug-Ins (disabled)/ Disallow: /Tools & Examples/