# Disallow all crawlers access to certain pages. User-agent: * Disallow: /css/ Disallow: /js/ Disallow: /mdb-database/ Disallow: /mailings/ Disallow: /public/ Disallow: /aspnet_client/ Disallow: /_private/ Disallow: /_vti_log/ Disallow: /app_data/ Disallow: /cgi-bin/