#use as template to structure the robots.txt file for each client #The following allows all robots to visit all files because the wildcard "*" specifies all robots. User-agent: * Disallow: /cgi-bin/ Disallow: /awstats/ Disallow: /Forms/ Disallow: /style.css Disallow: /contactform.css Disallow: /*.js$ Disallow: /*.swf$ Disallow: /*.gif$ Disallow: /*.jpg$