Free tool · No signup
What do Google and ChatGPT actually see on your page?
Paste one URL. The check fetches it the way a crawler does, without running JavaScript, and tells you what arrives: title, description, indexing rules, readable text, internal links, structured data, and which search and AI crawlers your robots.txt lets in.
One page per check. Public URLs only. Nothing is stored.
Why I built this
On October 5 a reader audited this site and found that the raw HTML of sideroad.co had one crawlable link to my builds per page, and the homepage told any crawler that skipped JavaScript it was still week one. Google was fine. Bing, and through Bing ChatGPT and Copilot, saw a hub with nothing in it. I fixed it that evening by pre-rendering the build lists into plain HTML.
This check is the quick version of that audit, for one page at a time. If it turns up problems across a whole site, CrawlKind is the full kit: 23 checks over up to 50 pages, run from Claude Code, with every fix shown as a diff you approve. It is $49 once, and it is also included in All Access.
What each result means
Content in the raw HTML is the one most AI-built sites fail. Frameworks that render everything in the browser send crawlers an empty shell. If this fails, look into pre-rendering or static generation for your important pages.
Crawlable internal links is how a crawler finds the rest of your site from this page. Links added by JavaScript after load do not count for crawlers that skip it.
robots.txt shows the rule for Googlebot, Bingbot, GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot on this exact path. Blocking AI crawlers can be a fine choice; blocking Google or Bing usually is not.
FAQ
What does the crawl check look at?
One URL, the way a crawler that does not run JavaScript receives it: the HTTP status and redirects, the title and meta description, noindex instructions, the canonical tag, the main heading, how much readable text and how many internal links arrive in the raw HTML, Open Graph tags, JSON-LD structured data, and whether robots.txt blocks Google, Bing, or the main AI crawlers.
Why does JavaScript matter for ChatGPT and AI search?
Google renders JavaScript, but Bing mostly reads the raw HTML, and ChatGPT and Copilot search through Bing. Most AI crawlers skip JavaScript too. If your content and links only appear after scripts run, those systems see a nearly empty page.
Is the check free? Do you store the URL?
Free, no account. The URL is fetched once to run the checks and the results come back to your browser. Nothing is saved on our side.
What if the check finds problems?
Each finding says what is wrong and why it matters. To fix them in your own codebase, CrawlKind is a $49 kit for Claude Code that runs 23 checks across up to 50 pages and turns the findings into changes you review before they land.