WTF is RAG? How AI search actually finds your content
RAG is how AI search finds your page, picks a passage from it, and quotes it in an answer. I break down the process and what it means for how you write and structure your content.

TL;DR
- RAG (retrieval-augmented generation) means an AI tool looks things up before it answers, then writes from what it found.
- AI tools don’t read your whole page when they answer. They match the question to individual passages, so one strong paragraph can get you quoted.
- For ChatGPT and Claude, a lot of that matching and ranking happens inside an ordinary search engine, so good GEO is mostly good SEO.
- Answer each heading in its first sentence and name things in full, so the passage still makes sense when it’s pulled out on its own.
- RAG cuts down on made-up answers but doesn’t end them. If retrieval finds nothing good, the AI can still fill the gap.
Early AI chatbots had a chad-like reputation for making things up and saying them with total confidence. Ask for a source and you could get a real-looking citation to a paper that didn't exist.
A language model is a pattern-matching machine. It predicts the most likely next word, so when it doesn’t know a fact, it still writes whatever best fits the pattern of a right answer. To the model, a fluent wrong answer and a fluent right one look the same.
Training made this problem worse. OpenAI's September 2025 research on why language models hallucinate points out that models were graded on accuracy alone, so a lucky guess scored points and "I don't know" never did. Their example is someone's birthday: there's no pattern to learn a random date from, so the model guesses, with a 1-in-365 chance of being right.
RAG is the fix most AI search tools use now: look it up first, then answer from what was found. Once you understand how it works, it’s a lot easier to tell which AI search advice is worth following.
What is RAG?
RAG stands for retrieval-augmented generation. The AI looks things up (retrieval), then writes its answer from what it found (generation).
A model with nothing to check against fills gaps with a guess that sounds right. Retrieval gives it something to check against.
The term and the method come from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research (now Meta AI). The idea had roots before that. Search-then-answer systems for question answering existed for years, like Facebook’s DrQA in 2017, which pulled Wikipedia passages to answer questions. Google’s REALM came out a few months before the RAG paper in early 2020 and also combined retrieval with a language model. The RAG paper named the approach and framed it in the way that stuck. It became a mainstream term in 2023, when companies started using it to ground ChatGPT-style chatbots in their own documents and search results.

RAG is different from the search you’re used to. With Google, you search, get a list of links, and click through to read the pages yourself. With RAG, the AI tool reads the pages for you and writes one answer, quoting a paragraph or two from the pages it used. So instead of your whole page competing for a spot in a list of links, a single paragraph from it is competing to be quoted, and the person asking may get their answer without ever visiting your site.
The process runs in two parts: what happens to your page before anyone types a question into an AI tool, and what happens after they hit enter.

Part 1: Before someone types a question
All of this happens days or weeks before anyone types a question. The AI tool (or the search engine it uses) collects your page, breaks it into pieces, and stores them so they’re ready to search later.
Crawling
First, a bot has to fetch your page. OpenAI's is called OAI-SearchBot, and OpenAI's crawler docs say sites that block it won't be shown in ChatGPT search answers. Other tools use their own bot or their search partner's. An AI tool can only retrieve pages that were already fetched and stored, so a page that was never crawled can't be cited.
Tokens
A token is the small piece of text a model actually reads, usually part of a word. OpenAI puts one token at about four characters of English, or three quarters of a word. Models work with numbers, not letters, so each token becomes a number.
Tokens set the limits on how much of your page an AI tool takes in. In the next step, your page gets cut into pieces sized in tokens. And when someone asks a question, the model can only read so many tokens at once, which is why it reads a shortlist of passages instead of whole pages, and why a long page it does open can get cut off before the end.
Chunking
Next, the page is split into passages called chunks, often a few hundred tokens each. A 400-token chunk is roughly 300 words, or two or three paragraphs.
Splitting the page up makes matching more precise: a whole page covers several ideas, so its overall meaning is blurry, while a passage about one idea can match a question closely.

Whether you should write with chunks in mind is a separate question, which I dive into in Should you chunk your content for AI search?
Entity mapping
The named things in each chunk, like brands, products, people, and places, get linked to the specific thing they refer to. This is called entity mapping.
A chunk cut out of a page loses the sentences around it that made a name clear. Say a chunk from a CRM comparison page reads “Close integrates with Gmail.” On the full page, it’s obvious that’s Close, the CRM. On its own, “Close” could be a word, a different company, or anything else with that name. Entity mapping links it to Close the CRM, so the fact gets attached to the right company. When it links to the wrong one, an AI tool can build a wrong answer out of real sources.
Google says AI Mode can “tap into fresh, real-time sources like the Knowledge Graph,” its database of facts about people, places, and things. That’s as far as the public evidence goes: no platform has published how it uses entities to match chunks to questions.
Storing it two ways
Finally, each chunk is stored twice: in a keyword index, and as an embedding, which is a list of numbers that captures what the chunk means. The two support two different kinds of search in Part 2.
Part 2: When someone types a question
Everything in Part 2 happens in the few seconds after they hit enter. The AI tool decides whether it needs to search, turns their question into searches, ranks what comes back, reads the best passages (and sometimes whole pages), then writes its answer from them.
Deciding whether to search
First, the model decides whether it can answer from what it learned in training or needs to look things up (unless you've turned search on yourself). Searching takes longer, so tools skip it when they think they already have an answer. They don't all decide the same way. Jonathan Clark found that Claude searched in 36.6% of prompts, against about 90% for ChatGPT, though no sample size was given, so treat it as one person's findings. When a model wrongly decides it already knows, it answers from memory, and that's where made-up facts can come from.
Rewriting your question
One question often becomes several searches, which Google calls query fan-out. Ask for "a CRM for a small sales team that works with Gmail" and the tool might search for CRMs for small teams, CRMs with a Gmail integration, and recent reviews, then combine what it finds. Splitting the question up lets the tool cover every part of it and draw on more than one source.
Searching and ranking
From the AI's point of view, this is one step: searches go in, and a ranked shortlist with snippets comes out. Who runs it depends on the tool.
| AI tool | Where it searches |
|---|---|
| ChatGPT | Third-party search providers OpenAI doesn't name (widely reported as Bing), plus its own crawler |
| Claude | Brave Search, plus turbopuffer (a vector and full-text search database) since May 2026, though its exact role isn't public |
| Google AI Mode and AI Overviews | Google's own search index |
| Gemini | Google Search |
| Perplexity | Its own web index, built to search by keyword and meaning and to rank passages within pages |
So for ChatGPT and Claude, a lot of the ranking happens inside an ordinary search engine before the AI sees anything. That's the mechanism behind the point I made in SEO vs GEO: good GEO is mostly good SEO. It's also a reason to check how you rank in Bing and Brave, not just Google.
None of these companies publish how they rank results. What follows is the standard way search systems find and rank passages, but no AI tool has confirmed doing it exactly this way. Their search partners also weigh plenty of other signals, like links, freshness, and quality.
Searching two ways: keywords and meaning
Search systems usually run two kinds of search side by side, which is called hybrid search. Each one scores the chunks against the question and returns its own ranked list, best match first. Each one also covers the other’s blind spot: keyword search misses a paraphrase, and meaning search can match something close but wrong, like a company with a similar name.
Keyword search (BM25)
Keyword search scores a chunk by how often the question's words appear in it and how rare those words are elsewhere, adjusted for length. The classic formula for this is BM25, from the 1990s, and it's still the default in search software like Elasticsearch. Google's own ranking has used far more than this for years, so think of BM25 as traditional keyword search, not "how Google works."
Meaning search (dense vector search)
Meaning search, also called dense vector search, compares the question's embedding with each chunk's, so "running shoes" can match "sneakers for jogging."
How chunking affects search
Both kinds of search score your page one chunk at a time, not as a whole, so the way your page got chunked affects how well it matches. Say someone asks an AI tool, “How much does Close cost?” If your pricing section became one chunk about Close’s pricing, that chunk is a good match. If the same chunk also covers Close’s integrations and support, the pricing part gets diluted, and a competitor’s chunk that’s only about pricing will match better. If the chunk just says “It costs $49 a month” and the name “Close” ended up in a different chunk, nothing in it says what costs $49, so it won’t match at all. And if the price is in one chunk and the plan details are in the next, neither one fully answers the question.
The tool decides where to cut, not you. Splitting on headings and paragraphs is the common default, so your structure probably shapes the boundaries, but no platform publishes its chunking rules. The effect is biggest where search works on passages, like Perplexity or Google’s passage ranking. A search partner may rank whole pages and pull a snippet instead.
Reciprocal rank fusion
The two searches score chunks on completely different scales. A keyword score might be 12.4 and a meaning score might be 0.83, so you can’t just add them together. RRF ignores the scores and only looks at each page’s position in each list: first, second, tenth, and so on. A page near the top of both lists ends up at the top of the merged one. It’s like two friends who rate restaurants differently, one out of 5 stars and one out of 100. You can’t compare their numbers, but if both put the same place at the top of their list, that’s a good sign.
| Page | Keyword rank | Meaning rank | Merged rank |
|---|---|---|---|
| Page A | 1 | 9 | 3 |
| Page B | 2 | 1 | 1 |
| Page C | 3 | 2 | 2 |
Page A wins on keywords, but meaning search barely ranks it, so it drops to third.
RRF can merge any number of ranked lists, not just two. In the fan-out step, one question becomes several searches, and each of those returns its own ranked list too, so RRF can merge them all into one. Microsoft’s Azure AI Search docs, for example, say it uses RRF “anytime there’s more than one query execution,” meaning any time more than one search runs for the same question.
Does AI search actually use it? RRF is the standard method and shows up in nearly every RAG tutorial, which is why so many posts say AI search uses it. For ChatGPT there's a real clue: in July 2025, Metehan Yeşilyurt found settings named rrf_alpha and rrf_input_threshold in ChatGPT's code. That shows setting names, not the formula, and he says himself it isn't necessarily the exact formula. My guess is that ChatGPT uses RRF on its own side to merge the results of its fan-out searches, after the search partner has ranked each one. So: likely for ChatGPT, unconfirmed by OpenAI, and no evidence either way for Claude, Perplexity, or Google.
Reranking
Many systems then have a slower, more careful model re-score the top results. The first searches are fast but rough, so reranking is a double check before anything reaches the AI. It isn't confirmed for any AI platform, though. The same ChatGPT code showed ranking_model: null, and Jonathan Clark, whose Claude search numbers came up earlier, reported that Claude uses Brave’s top 10 results without reranking them.
Reading the shortlist
Only a handful of results get passed to the model, mostly as snippets and passages, because what it can read at once is limited (and measured in tokens).
Picture a research assistant with a highlighter and five minutes. They search, but don't have time to read the pages, so they skim the top results, clip the few paragraphs that look most relevant, and write you a summary from those clippings, with footnotes. If your page made the cut, it's usually because one paragraph did.
Tom Johnson, a technical writer at Google, describes the same thing in Chunking your content for agent readability: search tools return the top-matching chunks with the page title and URL, and "the chunk is what the agent sees first, and sometimes the only thing it sees." The model makes the final call here, deciding what to trust, open, and cite.
Opening some pages (sometimes)
After reading the shortlist, the model may open a few of those pages in full, or search again if the results look thin. It can only open a page that already made the shortlist. OpenAI's ChatGPT-User agent visits pages when someone asks ChatGPT a question, and Anthropic's docs describe Claude using web search to find a page, then its web fetch tool to retrieve the full text (cut off at a length limit if the page is long). No platform says when it does which.
I saw this in my chunking study, where I tracked every page ChatGPT opened while answering 30 questions. It read 813 pages and cited about one in five, so most of what it opened never made it into the answer.
Tom Johnson's example is Google's Developer Knowledge MCP server, which returns chunks from its search tool and has a separate tool for fetching the full document. His warning is useful:
"If a chunk doesn't make sense on its own, the agent might not realize it needs to fetch more."
And even an opened page still gets quoted a passage at a time.
Writing the answer
Finally, the model writes from what it read and cites the pages. The citations tie the answer to text it can point to, so you can check it yourself.
RAG reduces made-up answers, but it doesn't end them. If retrieval comes back with nothing good, the model can still fill the gap. In my hidden pricing study, I asked ChatGPT, Perplexity, and Google AI Mode about SaaS products with no public pricing, and they quoted a price anyway almost every time.
Atomic answers
An atomic answer is a sentence right after a heading that answers it completely on its own, using the heading's words. It's the one habit that helps at almost every step above:
- Chunking: the chunk still makes sense if it's retrieved on its own.
- Entity mapping: naming things in full keeps the right one attached.
- Keyword search: reusing the heading's words matches the words in the question.
- Meaning search: a complete answer sits close to the question's meaning.
- RRF: ranking well in both lists lifts it.
- Reranking: a clear, direct passage holds up when it's re-scored.
- Writing the answer: one self-contained sentence is easy to quote without the model filling gaps.
To write one, answer fully first, reuse the heading's key words, name things in full, then explain. Here's one from my own post on off-site authority (yes, I'm critiquing myself):
An atomic answer helps a passage get found and quoted, but it doesn't guarantee a citation. Whether to go further and size every section to fit a chunk is the debate I get into in the chunking post.
AI search is still search (shocking)
AI search is still search. It looks for your words and your meaning, then quotes the passage that answers best, so the things that help that passage (clear headings, full names, answers up front) are the same things that help the person reading it.
more thoughts













