Audit technical SEO crawlability
seo-technicalskillsetup L2★128
AgriciDaniel/codex-seo ↗What it does
Audit technical SEO crawlability and indexability
Best for
Conducting comprehensive technical SEO audits for crawlability, indexability, and security issues across large sites.
Inputs
- · Domain URL
- · robots.txt
- · XML sitemap
- · Page list
Outputs
- · Crawl depth report
- · Index status analysis
- · Security audit
- · Mobile optimization issues
Requires
- · Web crawler access
- · DOM parser
Preconditions
- · Site accessible via HTTP(S)
- · robots.txt readable
Failure modes
- · JavaScript-heavy sites fail without browser rendering
- · Dynamic sitemap URLs missed
- · Auth-protected pages inaccessible
- · Redirect chains not fully traced
Trust signals
- · Explicit AI crawler taxonomy (GPTBot, ClaudeBot, Bytespider)
- · robots.txt implementation rules with examples
- · Step-by-step audit categories
- · HSTS preload list inclusion guidance