User-agent: * # Disallow everything by default except specific paths, but typical approach is to block specific non-public paths: Disallow: /auth/ Disallow: /dashboard/ Disallow: /api/ Disallow: /login Disallow: /search Disallow: /oauth2/ # Explicitly allow crawling of specific areas if needed, but since we want to rely on the sitemap: Sitemap: https://www.indexrender.io/sitemap.xml