Lucreya › Research › Data release
Built from the 12 frozen weekly captures of the AI Recommendation Audit (2026-W29 to 2026-W45). Nothing was re-queried: each engine is compared with itself in adjacent weeks, category by category. Licensed CC BY 4.0. Attribution: Vincent Wesley Couey, Lucreya.
| Engine | Captured | Weeks | Adjacent pairs | Top pick unchanged | Range across pairs | Top 3 carried over | Top 5 carried over |
|---|---|---|---|---|---|---|---|
| GPT-OSS 120B (OpenAI open weights, model memory) | 2026-W29 to 2026-W45 | 12 | 8 | 78% | 63% to 94% | 81% | 79% |
| Llama 3.3 70B via Groq (model memory) | 2026-W29 to 2026-W34 | 4 | 2 | 97% | 94% to 100% | 94% | 90% |
| Gemini 2.5 Flash (no search / model memory) | 2026-W30 to 2026-W40 | 6 | 2 | 66% | 57% to 75% | 74% | 63% |
"Top pick unchanged" is the share of the 16 categories where the engine named the same first company as the week before. Engines with a single captured week have no pair and are listed in the data file only.
The longest run of consecutive weeks. Intervals are 95% bootstrap intervals over the 16 categories.
| Weeks | Categories | Top pick unchanged | 95% interval | Top 3 | Top 5 |
|---|---|---|---|---|---|
| 2026-W29 to 2026-W30 | 16 | 94% | 81% to 100% | 77% | 78% |
| 2026-W30 to 2026-W31 | 16 | 88% | 69% to 100% | 79% | 80% |
| 2026-W39 to 2026-W40 | 16 | 88% | 69% to 100% | 81% | 79% |
| 2026-W40 to 2026-W41 | 16 | 75% | 56% to 94% | 88% | 80% |
| 2026-W41 to 2026-W42 | 16 | 63% | 38% to 88% | 83% | 76% |
| 2026-W42 to 2026-W43 | 16 | 69% | 44% to 88% | 77% | 74% |
| 2026-W43 to 2026-W44 | 16 | 69% | 44% to 94% | 81% | 84% |
| 2026-W44 to 2026-W45 | 16 | 81% | 63% to 100% | 77% | 78% |
Asking GPT-OSS 120B via Groq the same question twice in one sitting returned the same first pick 77% of the time (95% interval 63% to 91%). Week to week the same engine kept its first pick 78% of the time. The difference is +0.6 points, interval -9.8 to 12.2: it cannot be told apart from zero.
In plain terms, on this index the weekly churn in an engine's first pick is about what you get from asking again, so a single week's change should not be read as the engine changing its mind. Top 3 and top 5 overlap are reported beside it because they move less than the first pick.
| Engine, repeated in one sitting | Full repeats | Same top pick | 95% interval | Top 3 overlap | Top 5 overlap |
|---|---|---|---|---|---|
| GPT-OSS 120B via Groq (the production series engine) | 3 of 10 (+1 partial) | 77% | 63% to 91% | 83% | 81% |
| Claude Haiku 5.5 (model memory) | 5 of 5 | 79% | 68% to 91% | 83% | 91% |
| Claude Sonnet 5.5 (model memory) | 5 of 5 | 93% | 85% to 100% | 89% | 88% |
| Cohere Command-A (model memory) | 1 of 5 (+3 partial) | 81% | 65% to 96% | 90% | 87% |
The same 48 questions (16 categories, 3 phrasings each) were repeated, taken word for word from the 2026-W45 capture. Only the first row is the engine in the weekly series; the others are a research-lane comparison and are not in the weekly panel. Claude models do not accept a temperature setting, so they ran at the provider default and are not strictly comparable with the Groq and Cohere rows. The GPT-OSS run was stopped by the provider's daily token limit after 3 full repeats (10 were planned), and the repeats that did run are all that is counted.
| file | size |
|---|---|
data/series-stability.jsonsha256 99ed5da01674ed09bf461c752be5a070cbf73a43de7d7112a7950f868a296ec9 | 68 KB |
| manifest.json | n/a |
Rebuilt each week after the tracker freezes a new snapshot. A changed SHA-256 means the series changed.