# robots.txt for https://synlighet.no/ sitemap: https://synlighet.no/sitemaps-1-sitemap.xml sitemap: https://synlighet.se/sitemaps-1-sitemap.xml sitemap: https://synlighet.com/sitemaps-1-sitemap.xml # live - don't allow web crawlers to index cpresources/ or vendor/ User-agent: * Crawl-delay: 30 Disallow: /cpresources/ Disallow: /vendor/ Disallow: /.env User-agent: AhrefsSiteAudit Crawl-delay: 30 User-agent: AhrefsBot Crawl-delay: 30