User-agent: *
Allow: /
Disallow: /auth
Disallow: /app
Disallow: /dashboard
Disallow: /settings
Disallow: /account
Disallow: /billing
Disallow: /onboarding
Disallow: /runs/
Disallow: /compare

Sitemap: https://www.crawlly.ai/sitemap.xml