# Robots.txt file from http://www.searchengineworld.com # # Bans from cgi-bin and images # # Disallow /cgi-bin/ User-agent: * Disallow: /cgi-bin/ Disallow: /images/ Disallow: /favorites/ Disallow: /building/ Disallow: /assets/ Disallow: /php/ Disallow: /MT/ Disallow: /webstat/ Disallow: /webstat_old/ Disallow: access_log Disallow: /PB/ Disallow: /PB/Cob/ Disallow: /PB/LACMTA 370/ Disallow: /PB/LACMTA - New Flyer/ Disallow: /cats/ Disallow: /cats/cats move-in/