How LLMs Change Silently: Invisible Shifts in Fan-Outs and Source Retrieval

TL;DR

When a brand's visibility in AI engines fluctuates, the usual reaction is to look for what needs to be optimized in the brand's own content. But data from the GEO Metrics Research Lab shows that the most frequent cause is not in the content — it is in silent changes in the models' own source retrieval patterns. The internal fan-outs they use to expand queries change. The domains they prefer as references rotate. And all of it happens without any announcement. This article documents three patterns detected in the analysis of ~400,000 responses over 14 months.

When we evaluate AI Search performance, there are two metrics that matter most:

Brand visibility — how often does my brand appear in AI-generated responses?

Retrieved sources — how often does the model use my domain as a source to answer a query?

When these metrics fluctuate, the usual reaction is to assume something needs to be optimized on our own side. Far less often do we consider that the underlying system itself may have changed — modifying which parameters are relevant, which domains it prefers, or how it internally expands queries before generating its response.

GEO Metrics Research Lab data confirms it: LLMs change their retrieval patterns constantly, with small and often invisible updates. And those changes have direct, measurable consequences for which brands get cited and which don't.

What Is Query Fan-Out and Why It Determines Your Visibility

Before getting into the patterns, one step back.

When a user types a question into ChatGPT, Perplexity or any web-enabled AI engine, the model does not search for that question literally. It breaks it down internally into multiple sub-queries — the query fan-out — which it uses to retrieve information from different sources before synthesizing the response.

Those internal sub-queries determine which domains enter the retrieval process. And if the fan-outs change — if the model starts adding or removing terms, prioritizing certain source types or reformulating queries differently — the cited domains change with them.

The problem: those changes are invisible to the user. There is no changelog. There is no announcement. The final response may look similar — and yet be built on a completely different source universe.

Pattern 1: When Fan-Outs Add a New Term and Everything Changes

In the 14-month data analysis, we detected a pattern that has repeated at different moments and for different terms: the model starts adding a specific term to its internal fan-outs, which redistributes which domains are retrieved.

The mechanism is consistent:

1. The model adds a new term to a significant proportion of its fan-outs — typically as a suffix or as a modifier of the original query

2. That term directs the retrieval process toward a specific type of domain

3. Domains favored by that term gain citation share proportionally

4. The term may persist, decline or stabilize at a new baseline level

What makes this pattern especially relevant for GEO strategy is that the change in citation can be significant — and can occur without the model having changed version or any visible update having happened.

The direct implication: if your domain benefits from this type of new term in the fan-outs, your visibility rises without you having done anything. If it competes with the domains that term favors, your share can fall for the same reason. In either case, without monitoring the fan-outs, you won't know why.

Pattern 2: Reference Domains Are Not Static — They Rotate

Another consistent pattern in the data: the domains models use as reference sources — the equivalent of "who do I ask when I don't know something" — are not static. They rotate.

In the 14-month analysis, we observed how a type of domain can go from having a residual share in the reference category to becoming the most cited domain in that category — while the one that historically dominated loses share in a sustained way.

This type of rotation does not respond to changes in the domains themselves — in many cases, the content has not changed. It responds to adjustments in how the model evaluates which sources are most appropriate for the reference function within its retrieval process.

What this means for brands: the source type that positions you well as a reference in an engine today may not be the one that positions you well in six months. Not because you have gotten worse — but because the model has readjusted its criteria for what counts as a trustworthy reference.

The only way to detect it in time is to monitor not just whether you are cited, but what type of domain is gaining share in the reference category of your sector — and how that distribution evolves over time.

Pattern 3: Changes Between Versions of the Same Model Are Larger Than They Appear

When an AI engine releases a new version, the usual coverage focuses on model capabilities — reasoning, context window, speed. Much less on how the source retrieval process changes.

Research Lab data shows that fan-out and retrieval differences between versions of the same model can be substantial — with changes in the frequency of use of specific search operators, in the length and structure of internal sub-queries and in the preference for source types.

These differences have direct consequences for which domains are retrieved. A model that adds the site: operator with high frequency in its iterative fan-outs is prioritizing specific domain first-party content over aggregated or referenced content. A model that adds terms like "official" or "documentation" with high frequency is prioritizing primary sources over secondary ones.

For brands, this means that a new model version can radically change the citation profile — without any action having been taken by the brand and without any degradation in content quality.

The operational consequence: analyzing changes between model versions should be part of the standard GEO monitoring process — not a one-off exercise after a visibility drop has already been observed.

Why Content Is Not Always the Cause of Visibility Changes

The three patterns above point to the same conclusion: a significant portion of AI visibility fluctuations does not originate in owned content.

It originates in the system.

Fan-outs change. Reference domains rotate. Model versions reconfigure retrieval criteria. All of that happens continuously, without announcements, and directly affects which brands are cited and which are not.

A GEO strategy that assumes a visibility change is always a signal that "something needs to be optimized on our own side" is misdiagnosing the majority of cases.

The correct diagnosis requires separating two types of change:

Internal change — something has changed in the domain's content, technical infrastructure or source authority. The correct intervention is to optimize.

Systemic change — the model has changed its fan-out patterns, retrieval behavior or source preferences. The correct intervention is to adapt the source strategy to the engine's new profile.

Without data that allows distinguishing between the two types, the response will always be the same — and will be incorrect half the time.

What to Monitor to Detect Systemic Changes

Standard AI visibility monitoring measures the output — what the model mentions in its final response. To detect systemic changes, the process must be monitored too:

Internal fan-outs — which terms and structures the model uses to expand the query before retrieving sources. Changes in fan-outs are the earliest signal of a systemic change in progress. GEO Metrics' free Fan-out de consultas a IAs Chrome extension extracts these fan-outs in real time for ChatGPT, along with Bing AI Performance grounding queries and Google Search Console data. → Chrome Web Store

Cited domains by category — not just which URL cites you, but which type of domain is gaining or losing share in each source category (editorial, social, reference, niche). GEO Metrics' Citation Intelligence module monitors this distribution daily across the 9 main models.

Temporal source evolution — compare the domains cited today with those from 30, 60 and 90 days ago for the same prompt set. Systemic changes are detected over time — not in a one-off snapshot.

Version-to-version comparison — when an engine releases a new version, run the same prompt set across both versions and compare the fan-outs and retrieved sources. The differences are the most direct signal of what has changed in the retrieval process.

The Question That Should Guide Every Visibility Analysis

When a brand's visibility in an AI engine changes, the first question should not be "what did we do wrong?".

It should be: "has the system changed, or have we changed?"

Answering that question requires data from both sides — from the model's output and from the retrieval process. Without both, the diagnosis is an estimate. And estimates lead to interventions that may be correct or incorrect for reasons that have nothing to do with the quality of the work.

LLMs change silently. Monitoring needs to be granular enough to hear those changes before they become visibility drops that can no longer be attributed.

Data: GEO Metrics Research Lab #03 · ~400,000 responses · 7 engines · 14 months · March 2025 – May 2026

Monitor changes in your cited sources → trygeometrics.com