Product

Topic Engine Content Engine Code-to-Docs AEO Rank Research-Driven Content Autopilot Visibility Tracking Website & Migration Content Factory AEO Agency Pricing

Resources

Browse all resources → Latest articles Case Studies Blog FAQ Use Cases Knowledge Base Research Docs
Failure modes

The overfitting trap in AEO tool prompt lists

Quick Answer

Tuning pages to one AEO tool's fixed prompt list overfits them: the tool's number climbs while the questions buyers actually type go unanswered. Start from buyer questions; use the list only as a check.

How often a tool's default prompts match real buyer questions is not measured yet. The evidence does not show it. So I'd read the list like a smoke alarm, not a syllabus, and go to the buyers first.

Did this answer your question?

Key Points

  • In a 10 September 2026 test by GTM Labs, branded recall was 5 of 5 while unbranded discovery was 0 of 10 answers in all three categories.
  • Only three of twelve AI visibility tools in Daria Dovzhikova's comparison say on their own site that they separate branded from unbranded prompts.
  • Shanika Wickramasinghe's explainer advises a clear answer within the first 40-60 words; one clear answer serves every phrasing of a question, a matched prompt string only one.
Three things marketing teams believe about prompt tracking. Myth or fact?
Call each one, then see how other readers called it.
1 Matching a tracked prompt word for word is what gets a page cited.
2 The wording you measure against can differ from what users actually see.
3 A rising tool number proves new buyers are finding your brand.

Watch the 1-minute version

Should you use AEO tools? What to check before you trust the score

The same article in 56 seconds. Watch on the AEO Content YouTube channel

A marketing lead's desk at night in a quiet office, a printed spreadsheet with two columns of handwritten questions lying beside a pen and an open notebook, a few loose sheets of notes fanned out nearby, dark window

A dashboard at night, and the two columns that check it: the prompts a tool tracks, the questions buyers ask.

AEO specialists range from prompt trackers to content firms; our own four plans run up to Business Plus at $1,000/mo. Whatever you buy, ask whose questions the prompt list holds.

Answer engine optimization (AEO) is the work of getting AI engines such as ChatGPT, Claude, Perplexity and Google AI Overviews to cite your pages when someone asks a question. The companies that sell it come in two broad kinds. Some only track. Others write the pages, score them, then track.

Both kinds hand you a prompt list. That list is where the trouble starts.

At AEO Content we sit in the second group. Our Business plan at $500/mo pairs 30 articles with a dedicated strategist, and Business Plus at $1,000/mo carries 44 articles, Local AEO and 2 backlinks DR 30+. Every page is held to the AEO Rank standard, which checks whether the four engines name the client.

Price is not the test, though. I'd put one question to any vendor, us included: are these prompts the words your buyers type, or the words the tool chose for you?

Our own knowledge-base work points the same way. Its study step starts from questions users actually ask, and its pilot found an admin page describing one sign-in method while the real screen showed another. A writer working from labels would have copied the wrong one straight onto the page without a second look at what the user sees.

Labels drift from life. Prompt lists do too.

The tracker runs at night. It asks the same thing again in the morning, and nobody stops to ask whether a buyer would.

A dashboard at night. The same questions go out to ChatGPT, Claude, Perplexity and Google AI Overviews, and the same number comes back. Up a little. Down a little.

Most teams treat that number as the target. I think that is the mistake. A prompt list is the fixed sample of questions an AEO tool chooses in advance, and pages tuned hard to a fixed sample overfit: they get good at the list and go quiet everywhere else.

Our own pilot shows how strongly an input set shapes the output. We wrote the same two knowledge-base questions twice. With web research allowed, 30 to 38 web sources entered each article, and each article carried 15 to 18 first-person opinions. Under a code-only rule, the second round had zero web sources and zero opinions, and every product claim checked line by line against the code was correct.

Change what you feed a process and you change what it makes. A prompt list is an input too.

So this piece follows one SaaS team from the list back to the buyer, one step at a time.

What is a prompt list in an AEO tool, and why does it matter now?

A prompt list is the fixed set of questions an AEO tool sends to AI engines: a sample of buyer language, never all of it.

Before you trust the list behind your dashboard, check three things:

  1. Who wrote the prompts. A vendor default, your own team, or words lifted from real buyers.
  2. Whether any prompt names your brand. A branded prompt tests memory. It does not test discovery.
  3. When the list last changed. A list that never changes can only measure the questions it started with.

Some terms first. An AI engine is a system that writes an answer instead of returning a page of links, such as ChatGPT, Claude, Perplexity or Google AI Overviews. A citation is the moment that answer names or links your brand. Our AEO Rank standard scores every site on 53 criteria across 5 pillars, we have run 26,577 AI-visibility audits against it, and we track whether those four engines name our clients.

Now the worked example we will carry through this whole piece. Picture a B2B SaaS team with a tracking tool.

The list runs at night. Again in the morning. Same words. Same order.

The team watches one number rise and fall, and they start editing pages so the number rises. That is reasonable. It is also where the trap opens.

The common assumption is that more prompts, checked more often, means better optimization. It means better measurement of that list. Nothing more. One of our founders is a full-stack engineer and content infrastructure architect with 20 years of building enterprise systems, and I find that lens useful here, because a prompt list is a test set and anything tuned hard to a test set learns the test.

Outcomes look different. At Imagine Dental Arts, AI-referred new patients rose 900%, from about 5 a month to 50. That is a count of people who arrived. Not a count of prompts matched.

Here is a quick check you can run tonight: write down three questions a new buyer asked your sales team last month, then look for each one on your tracked list. If the vocabulary is new to you, our AEO knowledge base guides define each measurement term in plain language.

What does a real buyer question look like? Our own visibility monitoring keeps a list of questions where Gemini answers without naming us yet. Three of them read like this:

  • "Should I hire an AEO agency or use a self-serve AI visibility monitoring tool?"
  • "Which AEO services are best for early-stage startups with no domain authority yet?"
  • "Best technical AEO audit services for checking AI crawler access, schema and server-side rendering"

Look at the shape. Each one carries a situation. A budget choice. A young domain. A technical worry about crawlers. Nobody types "AEO tool" and stops. A buyer types a problem, with their own circumstances folded into it.

Now set beside them a far shorter buyer question: "What are the top tools for AI SEO or answer engine optimization?" Short. Generic. Exactly the kind of prompt a tool picks for its default list. It is a real question too, and Gemini does not name us on it either. But a list built only from prompts shaped like that one would never hear the startup with no domain authority, or the team weighing an agency against software.

That is why the list matters now, before any page is edited. It decides which buyers you can see at all. A miss on a question you never track does not turn a cell red. It shows up as nothing. Silence, out past the edge of the dashboard.

So the question for our SaaS team is not whether the number moved. It is what the page learned while chasing it, and that failure already has a name, borrowed from machine learning.

What does overfitting mean when the test is a list of prompts?

Overfitting means a page learns one tool's fixed prompts instead of the buyer's question. Our promise reads: named in AI answers in 100 days, or we work free until you are.

The word comes from machine learning. A model overfits when it fits its training examples so closely that it stumbles on new examples it has never seen. Think of a student who memorizes last year's exam. Perfect marks on that paper. Lost on the next one.

Swap the pieces. The exam is your tool's prompt list. The student is your page. The new questions are the ones buyers type tonight, in their own words, alone at their own desks.

Back to our SaaS team. Their tool reports which prompts surface the brand and which do not, so they rewrite a product page to echo a missing prompt phrase for phrase, and the next morning the number ticks up. The page has learned the exam.

Here is the friction, and I would rather name it than hide it. We sell tracking too. Our Growth plan at $99/mo tracks 5 prompts weekly, and Premium at $250/mo tracks 20 prompts daily. A list that size is a gauge. Turn a gauge into a target and it stops telling you the truth.

Alex and I bring a combined 40+ years of SEO and content infrastructure experience to AEO. The habit I distrust most is an old one carried over from search: shaping pages to please the instrument instead of the reader.

What we hold ourselves to is the answer, not the list. At Understood Care, Google AI Overview citations rose 500%, from 22 to 132, in about five months. That figure counts citations in AI Overviews. It is not a tally of matches against one fixed string.

One plain sign of overfitting is a heading no buyer would type: read each H2 aloud and ask whether it sounds like your customer or like the dashboard, the same test our research-driven content built on evidence has to pass. A page written that way may lose a given prompt on a given morning. I would take that trade every time.

How do you catch overfitting before it costs you? Machine learning has an old answer. Keep a holdout set: examples the model never trains on, checked only at the end. If the training number climbs and the holdout number stays flat, the model has learned the exam and nothing else.

You can do the same with prompts. Split your buyer questions in two. Edit pages toward one half. Never touch the other. Then watch both.

Here is what that looks like with two questions from our own Gemini monitoring, where the answer does not name us yet:

  • "What companies specialize in answer engine optimization (AEO)?"
  • "What are the best AI search optimization (AEO) companies?"

Same need. Different words. One says specialize. The other says best, and calls the whole field AI search optimization. Put the first on your tracked list and the second in the holdout. If a page starts winning the first and stays silent on the second, it has learned a phrase. Not the question.

A page that answers the need underneath, which companies do this work and how to tell them apart, should move both. That is the whole test. It costs nothing but discipline.

I find the holdout half the hardest to protect. Every morning the dashboard shows the tracked half, bright and close. The holdout sits out there unwatched. Leave it alone anyway.

So how does a page slide from answering buyers to answering the list? Slowly. One reasonable edit at a time.

Does a higher tool score mean new buyers can find you?

On 10 September 2026, a consultancy pointed its own AI visibility scorer at three buyer categories it sells into. A blended dashboard would have reported a healthy trend. The answers themselves told a different story.

The consultancy is GTM Labs. Daria Dovzhikova published the result inside a comparison of twelve AI visibility tools. When the engines were asked about the company by name, branded recall was 5 of 5 in each category. When they were asked the unbranded questions a stranger would ask, the organic score was 0 of 10 answers, in all three categories. Combined into one score, she wrote, the test "would have reported 50% and a healthy trend."

Branded recall (asked by name) 5 of 5 answers
Unbranded discovery (buyer questions) 0 of 10 answers
What a blended score would report 50%
One number, two very different results. Results per category in GTM Labs' test of its own brand across three unbranded buyer categories, run 10 September 2026. The blended figure is the one GTM Labs says a combined score would have shown.

Dovzhikova puts it plainly: "The blend averages a solved problem into an unsolved one." Recall is the easy half. Ask a model about a brand by name, she notes, and it will describe that brand: "almost every brand scores near 100% on it." Discovery, the unbranded half, "is the only number that describes buyers who have never heard of you."

For a team writing pages to raise a score, that is where the trap sits. Our own reading of her test: if a tracked list mixes branded and unbranded prompts, the score can climb on the half the brand already wins. Pages tuned until the number improves may be polishing recall while discovery stays at zero.

How many dashboards mix the two? Dovzhikova read each vendor's own site in September 2026, and the answer is mostly unclear.

Three of twelve

AI visibility tools that say on their own site that they separate branded from unbranded prompts. One of the three is GTM Labs' own open-source scorer. The other nine "may do it, but they do not say so."

So the trouble is disclosure. From those nine sites, a buyer cannot tell whether the headline number describes strangers or people who already know the brand. In Dovzhikova's words, a page that leaves something unstated "is a finding in itself."

The size of the list matters too. A tracked list is a fixed sample, and samples vary a great deal. Our own plans track 5 or 20 prompts. One tool in the comparison publishes a single plan with 500.

Tracked listPromptsHow often
Our Growth plan5Weekly
Our Premium plan20Daily
One published tool plan500Not stated in our source

Buyers, meanwhile, do not stick to a list. Shanika Wickramasinghe, who writes explainers on search, describes people typing full questions such as "What is the best CRM for small businesses?" Add a buyer's own situation and the wording branches further. A list of any size can sample that language. It cannot hold all of it, and none of our sources measured how often buyers type the exact prompts a tool tracks.

The test also pointed to something a team can act on. In all three unbranded categories, the engines cited linkedin.com. Dovzhikova concluded that "the engines know the name and never reach for it," and that the citation lists showed "where they reach instead." She calls the gap "a distribution problem." Nine of the twelve tools report citation sources, which she counts as "the part you can act on."

Seen side by side, the evidence says a tool's score describes its prompt list nearly as much as it describes the brand. Which prompts are in the list, how many there are and whether branded ones are mixed in all move the number. Rewriting pages to raise that number can mean winning again where the brand already wins. In the one test we found, the missing visibility was off the brand's own pages, on a third-party site the engines already trusted.

  • Ask any tool vendor, us included, whether branded and unbranded prompts are scored separately, and ask to see the unbranded number on its own.
  • Before you trust a score, run five unbranded prompts by hand. Dovzhikova's rule: "If the tool's score and your five answers disagree, believe the answers."
  • Count how many of your tracked prompts are unbranded. If that share is small, ask what a rising score can tell you about strangers at all.
  • Write a few of your test prompts the long way buyers do, with their situation in the question, and see whether the result changes.
  • Read the citation lists on your unbranded prompts. Note which third-party sites come up again and again, and ask whether your brand appears on them.

How we checked this

We used a published comparison of twelve AI visibility tools that includes a first-party test, a Medium explainer on answer engines and our own plan details. The tracked prompt counts of 5 and 20 are ours. The 0 of 10 and 5 of 5 results come from one company testing its own brand in three categories on one date, so they show what a blended score can hide. They do not show how common that gap is. GTM Labs makes one of the three tools it credits with separating branded from unbranded prompts. The nine other tools may separate them without saying so. We sell plans that track prompt lists, so we have a stake in how those lists are read. Still unknown: what share of any tool's tracked prompts real buyers actually type. None of our sources measured it.

  1. Daria Dovzhikova, GTM Labs, comparison of AI visibility tools with a first-party test, published 16 September 2026, updated October 2026, retrieved 14 November 2026.
  2. Shanika Wickramasinghe, explainer on AEO, GEO and SEO, Medium, May 2026.
  3. Our plan details showing tracked prompt counts and how often each plan checks them, November 2026.

How do you keep a page answering buyers instead of the list?

Answer the buyer's question first and use the tracked list only as a check. Before AEO, I founded LiveHelpNow, a customer-service SaaS and Inc. 5000 inductee (#84, 2015-2018).

A support queue is where questions arrive raw. In the customer's words. Not the product's. I think about prompt lists the same way now: the list speaks in the product's words, and buyers are somewhere else, typing something else.

Here is how the drift happens, step by step, inside our SaaS team:

  1. The tool flags a prompt the brand misses. Nobody asks whether a buyer would ever type it.
  2. The team edits a page to echo that prompt. Headings borrow its exact wording.
  3. The number rises. The edit looks proven.
  4. The cycle repeats. Each round, the page fits the list a little tighter and the buyer a little looser.

Why does the number rise at all? Partly because many tools blend two kinds of prompt. A branded prompt names you, and an unbranded prompt describes the need without your name. One vendor's own September 2026 test found 0 of 10 organic answers named the brand in each of three unbranded categories while branded recall was 5 of 5, and the vendor wrote that a blended score "would have reported 50% and a healthy trend." The same vendor's comparison of twelve tools advises running five unbranded prompts by hand before paying for any of them.

Before your next edit, tag every tracked prompt as branded or unbranded in your own sheet, and let only unbranded misses trigger a rewrite. A branded miss is a memory problem, not a page problem.

The fix is to reverse the order. When we build a knowledge base from a client's code repositories, a study step reads the repositories and proposes the questions users actually ask, such as "What is X?" and "How do I Y?", each tied to the files and line ranges that answer it. The questions come first. Measurement comes after.

We have scored 11,000+ domains across 15 sectors and 28 categories. In my view, no single default list could speak for buyers across that spread. A dental patient and a SaaS buyer do not share a vocabulary.

Buyer language already sits in places most teams own: support tickets, chat transcripts and sales call notes, written before anyone tidied the wording. The overlap rate between a tool's default prompts and mined buyer questions is the figure I most want, and the evidence we hold does not measure it yet. Until it does, treat every unmatched prompt as a question to verify, not a page to rewrite. Copy those questions word for word, typos included, and keep our AEO FAQ answers open for the measurement terms as you sort them.

For our SaaS team, that means one awkward afternoon with a spreadsheet. Tracked prompts in one column. Buyer questions in the other.

What should you do with your prompt list now?

Keep the list, but stop writing to it. Write each page to the buyer's real question, answer it early, and let tracked prompts tell you only whether the engines noticed.

Words on a dashboard drift from reality the way product copy does. In our repo-based pilot, the admin page described two-factor sign-in as using an authenticator app, while the actual sign-in screen sends a one-time code by text and email. A prompt list is that admin page. Plausible. Wrong in quiet places.

Shanika Wickramasinghe's explainer on AEO and GEO recommends giving a clear answer within the first 40-60 words. One clear answer serves many phrasings. A matched string serves one.

Over the next year or two, I expect buyers of tracking tools to ask a sharper question of every vendor. Not how many prompts. Whose words.

Summarize This Article With AI

Open this article in your preferred AI engine for an instant summary.

Frequently Asked Questions

What else do people ask about AEO tool prompt lists?

What is a prompt list in an AEO tool?

A prompt list is the fixed set of questions a tracking tool sends to AI engines on a schedule. It measures whether those engines name your brand. It is a sample, not the market.

What is overfitting in AEO?

Overfitting is tuning pages so closely to a fixed test set that they do well on it and poorly on the wider set of real questions. In AEO, the test set is the prompt list.

Does tracking more prompts improve AI visibility?

No. It improves measurement of those prompts. Visibility improves when pages answer the questions buyers actually type.

Why does this matter if fewer people click through?

An AI engine writes the answer where the buyer asked, so the buyer can get what they need without opening a single site. When the answer is the destination, being named inside it matters more. So the questions you are named on had better be real ones.

How soon can AEO content show results?

At Modern Wealth, organic search sessions rose 220% and Google clicks 61% within three weeks of the first articles. I would still judge results on buyer questions, not on one tracked list.

How do I get my prompt list checked against buyer questions?

Start with our free AEO audit, or reach us through the contact page at aeocontent.ai. Bring your current list. We will set it beside the questions your buyers ask.

Written by

Michael Kansky

Co-Founder, AEO Content

Michael Kansky is a serial founder and operator and co-founder of AEO Content, where he shapes product and go-to-market strategy for an AI-search content optimization platform.

Connect on LinkedIn

Related posts

A software buyer pauses on a city sidewalk at dusk, reading a laptop screen under a streetlamp while blurred storefront signs glow behind them

What ChatGPT tells B2B buyers about your pricing

Photorealistic editorial scene: a marketing buyer at a cluttered office desk in late-afternoon window light, leaning back with arms crossed and studying two stacked sheets of glossy sample swatches and a small product

We audited 18 SEO and content tool vendors: average AEO Rank 49

A marketing director

Search Engine Optimization Monitor: What to Track in Google and AI Answers

Pricing

Simple, flat monthly pricing.

Everything included. No per-seat games. Cancel anytime: your content, your repo.

Growth

$99 /mo

Start showing up in AI engines.

Start with Growth

What's included

  • AEO Website + Cloudflare CDN
  • 10 AEO articles / month
  • 5 prompts tracked weekly
  • Monthly re-audits + alerts
  • Reddit reply drafts + mention monitoring
Most chosen

Premium

$250 /mo

The package marketing teams settle on.

Start with Premium

Everything in Growth, plus

  • 20 AEO articles / month
  • Publish to 2+ site sections
  • 20 prompts tracked daily + threat analysis
  • Bi-weekly re-audits
  • Brand voice profile + strategy call

Business

$500 /mo

Hand us your domain. We run AEO end-to-end.

Talk to us

Everything in Premium, plus

  • 30 AEO articles / month
  • Social media on 6 networks
  • Up to 100 competitors + API
  • Weekly re-audits
  • Dedicated AEO strategist