# Crawlers Setup User-agent: * Allow: #GOOGLE User-agent: Googlebot Allow: User-agent: Googlebot-Image Allow: User-agent: Mediapartners-Google Allow: User-agent: Adsbot-Google Allow: # Allowable Index Allow: /*?p= Allow: /media/ Allow: /js/ Allow: /skin/ # Directories Disallow: /teste Disallow: /404/ Disallow: /app/ Disallow: /cgi-bin/ Disallow: /downloader/ Disallow: /includes/ #Disallow: /js/ Disallow: /lib/ Disallow: /magento/ # Disallow: /media/ Disallow: /pkginfo/ Disallow: /report/ #Disallow: /skin/ Disallow: /stats/ Disallow: /var/