Ask eight leading AI models the same question, "what is the best tool for X," and they will not give you the same answer. We measured exactly how far apart they are.
The disagreement is not uniform. On mature, settled categories the eight models nearly converge; on the newest, contested categories they scatter. Same models, same question format. "Consensus" is the share of the 28 model pairs that named the same number-one tool.
Each cell is that model's number-one recommendation for the category. Cells sharing a color agree; a row of different colors is a category the AIs cannot settle. Sorted by how divided they are.
In each row the most common pick is blue; every dissenting pick gets its own color. The number badge counts how many different tools the eight models named as #1. Across the whole study, 163 tools were named by only a single model.
Click any cell for the receipts: the ranked list that model actually named for that category, and the sources it cited.
How often each pair of models picked the same number-one tool, out of 16 categories. Darker means more agreement. Models answering purely from training memory tend to line up with each other; the moment a model searches the live web, its recommendations drift.
Disagreement is the story, but it is not the whole story. A small handful of tools surface as a top pick again and again, across engines and across categories. These are the few names the AIs broadly trust.
Named most often overall
The broader consensus set
Out of hundreds of tools named across the study, this short list is where the models converge. Everywhere else, they split.
This is a living index, not a one-off. Each scheduled re-run plots a single point: the Fleiss kappa for that snapshot. Today there is one dated point (2026-06-19). The line fills in as later snapshots land; a second capture is already in progress.
Every number on this page is generated from the raw recorded answers. Take the data, read the exact prompts, re-run the collector, and check the coefficient yourself.
The exact prompts, verbatim
Grounded models (live web search)
Using current web search results, recommend the best tools for this query: "{query}". Write 2-3 grounded sentences, THEN on a final separate line output exactly this and nothing after it:
TOOLS: Name1, Name2, Name3
(the specific tool/product names you recommend, most-recommended first, comma-separated, maximum 8)Memory models (no web search)
What are the best tools for: "{query}"? On a final line output exactly:
TOOLS: Name1, Name2, Name3
(specific tool/product names, most-recommended first, comma-separated, max 8){query} is each category phrasing. Each memory model was asked three phrasings per category; each grounded model one. The collector is collect.js; the coefficient is recomputed openly by _stats.js.
Most "AI visibility" tools tell a brand where it ranks. This measures something nobody publishes: the structural disagreement between the engines themselves, and it is reproducible. Every number on this page is generated from the raw recorded answers, dated, and released open.
Method. On 2026-06-19 we asked eight models, Llama 3.3 70B (Groq), Cohere Command-A, Gemini 2.5 Flash-Lite (grounded), Perplexity (web search), ChatGPT (web, GPT-5), Llama 4 Scout, GPT-OSS 120B (OpenAI open weights), and Qwen 3.6, the last three reached free via Groq, to name the best tools across 16 B2B software categories, and recorded the raw responses. "Grounded" models search the live web; "memory" models answer from training data alone. The full raw captures are retained so any standing here can be reproduced or challenged.
The 16 categories are not arbitrary. They are the B2B go-to-market software taxonomy, marketing, SEO and GEO, and sales verticals, drawn from our AI GTM Tools index, and used here as a fixed sampling frame so the identical question set can be re-run unchanged each snapshot.
Sampling rule (deterministic). Each memory model was queried with three phrasings per category; each grounded model once. The number-one pick per model per category is the modal (majority) top tool across its three phrasings, ties broken by the first phrasing. This replaces any hand-picked "representative" run. The rule governs the grid from the next snapshot forward; the 2026-06-19 grid shown here retains one representative run per memory cell, and three memory models (Llama 4, GPT-OSS, Qwen) were since re-captured for the next snapshot, so their June three-phrasing raw is no longer recoverable and their displayed pick is left as recorded. A robustness signal falls out of the phrasings that survive: a memory model returned the same number-one tool across all three of its own rephrasings only about 26% of the time (21 of 80 model-category cells). The models frequently disagree with themselves, which is part of why they disagree with each other.
Agreement coefficient. Across the eight models over the sixteen categories, Fleiss' kappa is 0.41 (moderate, on the Landis and Koch scale), and mean pairwise agreement is 44%, the share of the 28 model pairs naming the same number-one tool, averaged over categories. Kappa treats categories as subjects, models as raters, and distinct top tools as labels. Observed agreement Pbar = 0.44; expected agreement Pe uses the global (pooled) tool marginals, Pe = 0.05; kappa = (Pbar - Pe) / (1 - Pe) = (0.44 - 0.05) / (1 - 0.05) = 0.41. A per-category-marginal Pe is inappropriate for this design: with only eight raters and few labels per subject it inflates Pe above the observed agreement and returns a degenerate negative kappa, so we report the standard global-marginal value. The full calculation is open in _stats.js and _stats.json.
Data. Open under CC-BY-4.0. DOI 10.5281/zenodo.20767877. Snapshot 2026-06-19. This is a living index: the models are re-queried on a schedule, the coefficient is tracked over time (see the trend above), and standings shift as the engines change their minds.
Limitations. This is a single dated snapshot (2026-06-19), not a longitudinal average. It covers 16 B2B software categories with one number-one pick per model per category, across eight models. The specific versions tested were Llama 3.3 70B on Groq, Cohere Command-A, Gemini 2.5 Flash-Lite (grounded), Perplexity web search, and ChatGPT (GPT-5, web), plus three newer Groq-hosted open-weight memory models: Llama 4 Scout, GPT-OSS 120B (OpenAI open weights), and Qwen 3.6. The "grounded" versus "memory" split is our labeling of whether a model searches the live web, not an official product distinction. Fleiss' kappa on a small, category-specific label space is a summary, not a hypothesis test. Standings are point-in-time and will move as the models are updated. Read the numbers as a measurement of a moment, not a permanent ranking.