Crawler access
See indexability, robots directives, and AI bot access derived from robots.txt.
Site Auditor
Audit AI search readiness across your entire site.
Review the technical signals that affect how AI crawlers and search engines access, understand, and discover your content. In our citation study, 92.5% of AI-cited pages exposed their main content without JavaScript rendering.
Built for teams managing serious AI search programs
What it audits
The foundation of AI search visibility
One crawl shows where access, structure, and technical signals need attention.
Page understanding
Review canonicals, schema, titles, descriptions, headings, and duplicate content.
Site discovery
Review status codes, redirects, linking pages, crawl depth, and internal link structure.
Page quality
Inspect word count, page weight, headings, metadata, and connected data when available.
Clear evidence
Know what to fix first.
Filter the page inventory by issue, indexability, crawl depth, or template. Group repeated findings so one template or system change can address problems across multiple pages.
- 01Crawl from a website, sitemap, or URL list.
- 02Review the affected pages and supporting evidence.
- 03Fix the underlying issue and run another crawl.
AI readiness report
AI crawler rulesAllowed in robots.txt
Indexability conflictsReview
Canonical issuesReview
Deep pagesReview
Structured dataDetected
Site Auditor questions
What is AI search readiness?
AI search readiness is the extent to which search engines and AI crawlers can access, understand, and navigate a website. Site Auditor surfaces the technical signals that affect that foundation, including crawler access, indexability, canonicals, schema, internal links, crawl depth, duplicate content, and page structure.
What does Site Auditor check?
Site Auditor builds a page inventory and reports HTTP status, redirects, indexability, robots directives, canonical URLs, metadata, headings, word count, crawl depth, linking pages, schema, page weight, duplicate content, and AI crawler access derived from robots.txt.
Does a passing audit guarantee AI citations?
No. An audit identifies technical barriers and diagnostic signals; it does not guarantee that an AI platform will mention or cite a page. Content quality, relevance, authority, and off-site signals also influence AI search visibility.
How can I crawl a site?
Start from the website, a specific XML sitemap, or an uploaded URL list. You can include subdomains, restrict the crawl to selected paths, exclude paths, ignore URL parameters, and optionally check external links.
Can I verify fixes?
Yes. Run another crawl after making changes, then compare the current report with the previous crawl to confirm whether issue counts and affected pages changed.
Can I share the audit with a client?
Yes. A completed audit can generate a public, read-only report link for clients or stakeholders.
Find what is limiting AI search readiness.
Crawl the site, review the evidence, fix the barriers, and verify the change.
Start auditing