Lucreya › Research › Data release

AI recommendation stability, 2026: how much an engine changes its answer week to week

Built from the 12 frozen weekly captures of the AI Recommendation Audit (2026-W29 to 2026-W45). Nothing was re-queried: each engine is compared with itself in adjacent weeks, category by category. Licensed CC BY 4.0. Attribution: Vincent Wesley Couey, Lucreya.

What the series shows

EngineCapturedWeeksAdjacent pairsTop pick unchangedRange across pairsTop 3 carried overTop 5 carried over
GPT-OSS 120B (OpenAI open weights, model memory)2026-W29 to 2026-W4512878%63% to 94%81%79%
Llama 3.3 70B via Groq (model memory)2026-W29 to 2026-W344297%94% to 100%94%90%
Gemini 2.5 Flash (no search / model memory)2026-W30 to 2026-W406266%57% to 75%74%63%

"Top pick unchanged" is the share of the 16 categories where the engine named the same first company as the week before. Engines with a single captured week have no pair and are listed in the data file only.

GPT-OSS 120B (OpenAI open weights, model memory), pair by pair

The longest run of consecutive weeks. Intervals are 95% bootstrap intervals over the 16 categories.

WeeksCategoriesTop pick unchanged95% intervalTop 3Top 5
2026-W29 to 2026-W301694%81% to 100%77%78%
2026-W30 to 2026-W311688%69% to 100%79%80%
2026-W39 to 2026-W401688%69% to 100%81%79%
2026-W40 to 2026-W411675%56% to 94%88%80%
2026-W41 to 2026-W421663%38% to 88%83%76%
2026-W42 to 2026-W431669%44% to 88%77%74%
2026-W43 to 2026-W441669%44% to 94%81%84%
2026-W44 to 2026-W451681%63% to 100%77%78%

How much of that change is just noise?

Asking GPT-OSS 120B via Groq the same question twice in one sitting returned the same first pick 77% of the time (95% interval 63% to 91%). Week to week the same engine kept its first pick 78% of the time. The difference is +0.6 points, interval -9.8 to 12.2: it cannot be told apart from zero.

In plain terms, on this index the weekly churn in an engine's first pick is about what you get from asking again, so a single week's change should not be read as the engine changing its mind. Top 3 and top 5 overlap are reported beside it because they move less than the first pick.

Engine, repeated in one sittingFull repeatsSame top pick95% intervalTop 3 overlapTop 5 overlap
GPT-OSS 120B via Groq (the production series engine)3 of 10 (+1 partial)77%63% to 91%83%81%
Claude Haiku 5.5 (model memory)5 of 579%68% to 91%83%91%
Claude Sonnet 5.5 (model memory)5 of 593%85% to 100%89%88%
Cohere Command-A (model memory)1 of 5 (+3 partial)81%65% to 96%90%87%

The same 48 questions (16 categories, 3 phrasings each) were repeated, taken word for word from the 2026-W45 capture. Only the first row is the engine in the weekly series; the others are a research-lane comparison and are not in the weekly panel. Claude models do not accept a temperature setting, so they ran at the provider default and are not strictly comparable with the Groq and Cohere rows. The GPT-OSS run was stopped by the provider's daily token limit after 3 full repeats (10 were planned), and the repeats that did run are all that is counted.

File

filesize
data/series-stability.json
sha256 99ed5da01674ed09bf461c752be5a070cbf73a43de7d7112a7950f868a296ec9
68 KB
manifest.jsonn/a

Rebuilt each week after the tracker freezes a new snapshot. A changed SHA-256 means the series changed.

Stated limits

Conflict of interest. Lucreya is part of the Lattice network, which sells disclosed placements on pages that AI engines cite. That is a commercial interest in this subject. These files are published so the findings can be checked against us.
☆ Save
Dashboard