# A robots.txt file on a website will function as a request that specified # robots ignore specified files or directories in their search. User-agent: * Disallow: /js Disallow: /styles Disallow: /captcha # Sitemap location # Sitemap: http://domain.com/sitemap.xml