# Robots.txt file for www.webdubois.org # # Bans web crawlers and spiders from # indexing selected subdirectories # # Disallow: /DirectoryName/ User-agent: * Disallow: /_Academics/ Disallow: /_RW_Projects/ Disallow: /__Previous/ Disallow: /Previous/ Disallow: /_Teaching/ Disallow: /Dept/ Disallow: /htsdata/ Disallow: /images/ Disallow: /manual/ Disallow: /Merchant2/ Disallow: /misc/ Disallow: /vmanual/ Disallow: /ztest/ Disallow: /cgi-bin/ Disallow: /cgi-local/ Disallow: /temp/ Disallow: /testing/ Disallow: /stats/ Disallow: /11/ Disallow: /stpos/ Allow: /wdb-1900exp.html Allow: /wdb-AtlUniv.html Allow: /dbEvolOfRaceProb.html Allow: /dbANParis.html