Audit AI readiness and llms.txt

Audit domain AI-readiness via robots.txt and llms.txt

Quick audit of whether a domain properly advertises itself to LLM crawlers via robots.txt, llms.txt, and llms-full.txt with structured pass/fail scoring

• · Domain (e.g. anthropic.com or docs.example.com)

• · Pass/Warn/Fail findings per check
• · Actionable fixes
• · Formatted report

• · curl with bash_tool
• · robots.txt
• · llms.txt
• · llms-full.txt

Domain must be reachable via curl; well-known file paths must respond with HTTP 200 or 404 (not timeout)

• · Domain unreachable (curl 000)
• · Files return 403 or 5xx
• · Malformed XML in llms.txt

• · Uses curl directly (no guessing)
• · Captures HTTP status codes
• · Detects build-time vs real lastmod stamps