Run a free AEO audit twice: which half of the result should move

Two runs of the same free AEO audit: the readiness rows should match, the visibility rows may not.

Run a free AEO audit twice and one half should move: the readiness checks that read your site hold still, while the sampled AI-visibility findings shift of their own accord.

A free AEO audit is a one-time scan that reads a site's crawler permissions, schema and content structure, and in some tools samples live answers from engines such as ChatGPT besides. The two halves obey different rules. One reads files that change only when you change them. The other does not.

The second half was described with some precision in a 2025 study by Oshen Davidson, published by AirOps: "every query is a fresh retrieval, every response is a fresh selection." It is remarkable how much hangs on the definition of a hit.

In that 2025 dataset, mentions without citations were 3× more common than citations without mentions. In the same 2025 work, a brand both cited and mentioned on its visible run was 40% more likely to resurface later than one only cited. A tool counting only links will report less presence than one that also counts names.

Practitioners in the AEO community advise running each prompt 5 or 10 times, since a single run of a non-deterministic model wanders considerably. Upon closer inquiry, a recorded miss sometimes proves to be an answer that named the product and not the company. Neither finding touches the readiness rows.

I take the view that the readiness rows deserve the strictness of an inspection, and the visibility rows the patience of a tally kept across many runs. Which rows belong to which half is the first matter to settle.

Which checks in a free AEO audit should repeat exactly on a second run?

Every check that reads your own files and markup should repeat exactly: crawler permissions in robots.txt, llms.txt presence, schema, heading structure and citability signals, unless someone changed the site between runs.

Before comparing a second report with the first, I would put four plain questions to each readiness row:

1. Did the row read something on the site itself, such as robots.txt, llms.txt, sitemap.xml or a schema block?
2. Was anything deployed, edited or republished between the two runs?
3. If nothing changed, does the file still load at the same address when you open it in a browser?
4. If the file loads and the row still flipped, has the checker simply erred?

GPTBot, ClaudeBot and PerplexityBot are either permitted in robots.txt or they are not, and one free checker shared on Reddit tests precisely that among the 18 AEO-specific checks its builder says it runs. A deterministic check is a test that reads the site's own code or content and returns the same verdict whenever those inputs are unchanged. The same checker, by its builder's account, also looks for FAQ, HowTo, Article and Organization schema, for heading hierarchy and answer-first paragraphs, and for author bios, dates and source links. Nothing in that list consults an answer engine. Each item is read from your pages, and a second reading of the same pages ought to agree with the first.

It is worthy of notice how regularly these families recur. Set 5 published walkthroughs of free audit tools side by side and the same groups appear in nearly every one: crawler access, structured data, content structure and signals of citability. One vendor's demonstration scores a page between 0 and 100 across SEO, AEO, schema, performance and content sections; another sorts its fixes into crawler accessibility, content quality and structured data; a third, run on its maker's own landing page, flagged missing H1 and H2 structure and an unclear definition of the service. A fourth builder claims to go beyond robots.txt into content length and copy, and reports lifting their own site to 81/100 after a bit of work. One exception deserves a note, since the first of those vendors describes its performance section as a real-time load test taken at the moment the report runs. I am inclined to think a speed reading of that sort belongs to neither half, and should be read on its own terms.

The common assumption is that any difference between two runs must reflect the site. In the fixed half, a flip without a deploy may just as well reflect the instrument. On that same Reddit checker, one commenter reported that it declared llms.txt missing on a site that had one, and did the same with sitemap.xml. A file is present or absent. When a yes-or-no row disagrees with itself across two runs, the fault lies with the tool rather than the page. Fix the instrument before you touch the page.

At AEO Content we undertake that a client will be named in AI answers in 90 days, or we work free until they are, and a commitment of that kind can only be judged against a site whose readiness rows hold still from one run to the next. Readers who wish to verify any single row by hand will find each check described in our AEO knowledge base. The other half of the report keeps no such discipline, and the cause lies not in your pages but in how the engines choose their sources on each separate occasion.

Why does the AI-visibility half of an audit move when nothing on the site changed?

Answer engines reselect their sources on every run, so a visibility sample can differ from last week's even when not one line of your site has changed.

In 2025, a study by Oshen Davidson, first published by AirOps, found that across more than 45,000 citations only 30% of brands kept their visibility from one run of an identical query to the very next. In that 2025 study, about 20% held across all 5 runs in a series. Yet in the same 2025 data, about 57% of brands reappeared in a later, non-consecutive run, so that most disappearances proved to be absences rather than losses. The study treats this drift as a structural property of how models select citations from a retrieval pool that varies from run to run, and it states the matter plainly: "Reappearance is the rule, not the exception." It is remarkable that the query never changed. Only the draw did. The runs all fell within a 23-day window on a single model, the consumer ChatGPT interface running GPT-4.1, so how far the turnover extends from one month to the next is a question the evidence leaves open.

Practitioners report the same restlessness at a finer grain. GrowthSpree, a B2B SaaS marketing agency, says in an r/aeo discussion that answers shift within 24 hours, "sometimes 9 am versus 6 pm on the same prompt." Another commenter in that thread enumerates the conditions that alter a reply to an identical question:

• Session state: a logged-in account versus a fresh session.
• Memory: switched on or switched off.
• Timing: the hour at which the prompt is sent.

None of these is a property of your website. Each belongs to the moment of asking.

The engines also differ in how closely they lean on the ordinary web index. A study of thousands of questions, described in a 2025 interview, found ChatGPT's citations overlapping with Google results about 35% of the time, against about 70% for Perplexity, and practitioners who run formal tests note that answers vary noticeably even with no intervention at all. A report that pools several engines therefore pools several different rates of drift, each with its own habits and its own season of change.

Not every free audit samples live answers, and the distinction matters when reading a second run. One vendor's free grader, demonstrated in an April 2026 walkthrough, scores how ChatGPT, Perplexity and Gemini characterize a brand based on their training data. A reading drawn from training data answers a different question from a live sample, and ought not to be set against one as though the two were alike. A drop in a sampled row is weak evidence of anything. The sample moved; your site did not.

Owners who would rather follow the sampled half across several weeks than judge it from a single run can see how ongoing visibility tracking works. Which leaves a practical difficulty on the desk: two reports, one visibility figure lower than before, and the need to decide whether anything you did has caused it.

How do you compare two free AEO audit runs to separate real progress from turnover?

Diff the readiness rows against a dated change log, read visibility as a rate across repeated runs, and let booked calls settle the question.

My counsel is to take the two reports in a fixed order, finishing each step before beginning the next:

1. Write down every change made to the site between the runs, with its date: deploys, robots.txt edits, schema, new or rewritten pages.
2. Diff the readiness rows directly, and match each row that changed to an entry in that log.
3. Recheck by hand any changed row with no matching entry, before you credit or blame the site.
4. Carry rows marked unknown, unverified or not run forward as open questions, never as passes.
5. Freeze the visibility inputs: prompt wording, engines, account state and the definition of a hit.
6. Read visibility as a rate across several runs, and set a business outcome beside it.

The fourth step deserves emphasis, because a tidy report invites optimism. One practitioner's published checklist for a free technical audit notes that rows can come back unknown, unverified or not run, and declines to promote any of them to a pass merely because the rest of the report looks healthy. The same checklist observes that a technical audit conducts no ChatGPT searches, and that one answer cannot stand in for an ongoing citation rate. It keeps live AI observations in a separate dated log instead, recording against each entry the prompt, the product surface, the model when shown, the locale, the response and the source links displayed.

The fifth step is where most comparisons quietly fail. In an r/GenerativeSEOstrategy thread, one practitioner described running 20 buyer questions across four engines for a large consumer electronics brand and getting 77. About half the recorded misses were not misses at all, since the answers named the brand's products while the run counted only the company name. With product names added as aliases, the same questions against the same engines the next day returned 94. "Nothing changed in the world," the practitioner wrote. "Only what I counted as a mention." The site did not change. The ruler did. The same practitioner noted that only one of those engines retrieved and cited sources by default, so for most answers the site was not consulted at that moment at all. A second contributor recommends a fixed set of 20 to 50 prompts on a regular schedule, recording mentions, citations, competitors, source types and whether the answer changed.

A frozen definition is worth more than a frequent check. Alex and I bring a combined 40+ years of SEO and content infrastructure experience to AEO, and I am persuaded that the habit which carries over most directly from rank tracking is this one: hold the measuring stick still before reading the measurement. Owners unsure which questions belong in the frozen set can begin from the questions AI is already answering about their market, which our Topic Engine gathers.

Last, keep a business outcome beside the rate. AEO Content's own record includes a site that went from quiet to 10 booked calls in 3 weeks, and calls on a calendar are a steadier witness than any single prompt, however promptly it was checked.

What will matter most in free AEO audits over the next 12 to 24 months?

Separation will matter most: readiness checks judged against deploys, and presence in AI answers reported as a rate across repeated runs of fixed prompts, never as one reading.

I hold that the free audit will come to look less like a report card than a ledger, with one column that moves only when someone ships and another read only as a rate across runs. Three signals point that way already.

My reading of the evidence could be overturned in two ways. If assistants began retrieving and citing sources by default on every query, or if identical prompts stopped returning different answers within a day, repeated sampling would lose much of its purpose and a single run would again mean something.

There is a quieter doubt as well. One commenter in that September 2026 thread admitted to "wondering if I'm measuring real visibility or just visibility for questions I made up." Every tracked prompt is a guess at what buyers type. A rate built on guesses inherits them.

The sources also stop short in time. The strongest dataset here covered one model over a matter of weeks, and nothing in this evidence measures how cited domains turn over from one month to the next. For the readiness half, none of this uncertainty applies: a robots.txt file that nobody has edited should read the same in any season.

What companies specialize in answer engine optimization (AEO)?

AEO specialists span free checkers, agencies and full platforms; the one worth hiring reports site readiness and sampled AI visibility as separate findings.

In 2025, seven in ten brands cited on one run of an identical ChatGPT query did not earn that citation on the next. A single snapshot sold as progress reports the weather of one afternoon.

Among the sources gathered for this piece, the soundest design came from Ethan Smith of Graphite, speaking on Lenny's Podcast in 2025. It uses 200 questions: 100 left completely untouched and 100 changed, each tracked for a couple of weeks before and after the change. The same guest held that a result which reproduces about 10 times probably works. A control group shows turnover with the fixes taken out.

The stakes are not idle. By the same account, Webflow got 8% of its signups from LLMs in 2025.

I expect audit reports to divide before long into a readiness list that moves only with a deploy and a presence rate kept over weeks of runs. AEO Content, which I co-founded, is built to join scoring, content creation and visibility measurement in one loop. Ask any firm you weigh how many runs stand behind each visibility rate, and whether it kept a control group untouched.