# StatOrigin
# Data is published under CC BY 4.0. Attribution required, commercial use
# permitted. AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended
# and the rest) are explicitly welcome under the rules below — being cited
# is the point. There is deliberately no per-bot group: a separate group
# would replace these rules rather than add to them.

User-agent: *
Allow: /

# Private surfaces
Disallow: /admin
Disallow: /admin/
Disallow: /api/admin
Disallow: /api/admin/
Disallow: /search
Disallow: /mcp
Disallow: /api/v1/internal/
Disallow: /api/corrections

# Crawl budget: nothing here is rate-limited, but please honour the sitemap
# lastmod values rather than recrawling everything on every visit.

Sitemap: https://statorigin.org/sitemap.xml