User-agent: * Dissallow: TheRemodelers.flv User-agent: * Dissallow: TheRemodelers.swf block Google's image crawler completely User-agent: Googlebot-Image Disallow: / -------------------------------------------------------------------------------- ; block all spiders and bots from those 2 directories User-agent: * Disallow: /cgi-bin/ Disallow: /images/ -------------------------------------------------------------------------------- ; allow Googlebot to access everything except /cgi-bin ; and all other bots can access nothing ; finally allow ia_archive (alexa.com) to access everything! User-agent: * Disallow: / User-agent: Googlebot Disallow: /cgi-bin/ User-agent: ia_archiver Allow: /