Is Your Brand in the AI Training Data? How to Find Out (and Why It Matters)

A manufacturing client asked me a question last spring that I couldn’t answer on the spot: “Does ChatGPT even know we exist?” Not whether it recommended them. Whether it had ever seen them at all. I gave him a careful non-answer, went back to my desk, and realized most of our clients were sitting on the same unasked question.

It turns out there’s a practical way to get close to an answer, and it starts with one public dataset most marketing leaders have never heard of. Before a model can describe your brand from memory, information about it has to appear in the material the model learned from. Common Crawl is one source of that material, but labs also use other web crawls, licensed data, and third-party mentions. Let me show you how to check, what the check can and can’t tell you, and what to do with the result.

What “being in the training data” actually means

A large language model learns language, facts, and associations by processing an enormous body of text before it ever answers a question. That body of text is its training data, and a large share of it comes from crawling the public web. When a model “knows” that your company makes industrial pumps or runs twelve dental clinics, that knowledge traces back to pages it saw during training.

The best-known source of that web text is Common Crawl, a nonprofit that has been crawling the web and publishing the results openly for years. Its archives have been a core ingredient in many well-known models; OpenAI’s GPT-3 paper, for example, listed a filtered version of Common Crawl as the largest part of its training mix. Not every model discloses its sources, and labs add their own crawls on top, but Common Crawl is the closest thing the industry has to a shared, inspectable record of what the web looked like to a training pipeline.

That’s why we treat it as a proxy. We can’t open up a model and see its training set. We can open up Common Crawl and see whether you’re in it.

Why it matters for how AI describes you

There are two ways an AI assistant ends up talking about your brand. One is memory: what the model absorbed in training. The other is live retrieval: searching the web at the moment of the question and pulling in current pages. Most assistants now do both, and they lean on each in different situations.

Training-data presence shapes the first layer. It’s the model’s baseline sense of who you are, what you do, and who you’re similar to, the picture it reaches for when it doesn’t search, and the prior it brings even when it does. A brand that’s thin or missing in that layer tends to get vague descriptions, outdated facts, or confused with a similarly named company. Live retrieval can paper over some of that, but you’re starting from behind.

“Common Crawl can show whether a public archive captured your site. It can’t show whether a particular model learned from it.”

How the free check works

Our AI Training Data Visibility Check runs your domain against Common Crawl’s public data and reports three things. First, presence: whether Common Crawl has actually captured your site, and how much of it. Second, authority: where your domain sits in Common Crawl’s web graph, measured with Harmonic Centrality and PageRank, two ways of scoring how well connected and referenced a site is across the crawled web. Third, trend: how those numbers have moved across recent crawl releases, so you can see whether you’re gaining ground or quietly slipping.

It takes a minute and costs nothing. I’d encourage every marketing leader to run it before reading another article about AI search, because it replaces an abstract worry with a concrete number.

What the check doesn’t tell you

I want to be straight about the limits, because overselling a free tool is a fast way to lose a reader’s trust. The check measures your presence and standing in Common Crawl. It does not query ChatGPT, Gemini, or Perplexity, so it can’t tell you whether those assistants mention you, cite you, or describe you accurately. It also doesn’t generate an AI Readiness Score; that score comes out of our paid AI Readiness Audit, which tests how the major engines actually represent you and scores your site across eight categories.

Think of the free check as a smoke detector, not a building inspection. A thin or missing result is a reason to investigate; it can reflect crawl selection or the releases checked as well as access problems. A good result is reassuring, but it isn’t proof that AI engines are describing you well.

The most common culprit: you’re blocking the crawler

When a client’s result comes back thin, the cause is very often a single line they didn’t know about. Common Crawl collects pages with a crawler called CCBot, and plenty of sites block it, either deliberately in robots.txt or accidentally through a CDN or firewall rule that treats unfamiliar bots as threats. It’s one of the easiest crawlability problems to miss, because nothing on your site looks broken. Some teams added those blocks during the early wave of anxiety about AI scraping without weighing the visibility trade-off. Others never chose it at all; a security setting chose it for them.

Blocking is a legitimate business decision for some publishers. For most companies that want AI to describe them accurately, it works against them. If you find a block, the fix is usually a short conversation with whoever manages your robots.txt and firewall, which can make your pages eligible for future collection, without guaranteeing inclusion or timing. That access layer is the heart of Technical SEO & AI Crawl Readiness; we covered why crawlers that can’t reach or render your pages never see your content in Crawlability and the “Googlebot Doesn’t Interact” Rule.

What to do with your result

If you’re well represented and trending up, good. Keep the foundation healthy and move your attention to how AI engines actually describe you, which is a different question. If you’re thin, missing, or trending down, start by checking for crawler blocks, then look at whether your most important pages are linked and reachable enough to be collected at all.

Either way, the free check answers one question: was your site captured in the Common Crawl releases we checked? The next question, whether the engines cite you, describe you accurately, and choose you over competitors, needs real testing. That’s where an AI Readiness Audit comes in, and it’s sequenced inside our S.T.A.R.^SM framework so the fixes it surfaces land in the right order.