# AI crawler policy: public pages are open to search and AI crawlers (GPTBot, OAI-SearchBot, # Claude-SearchBot, PerplexityBot and others). Only private paths below are disallowed. User-agent: * Allow: / Disallow: /portal Disallow: /auth Disallow: /api Disallow: /checkout Disallow: /access Disallow: /verify-code Sitemap: https://www.codebreakthrough.com/sitemap.xml # LLMs: https://www.codebreakthrough.com/llms.txt