New research from SparkToro examines how consistently AI models recommend brands and products and the results should give marketers pause. The study tested repeated, near-identical prompts across major AI systems to see whether the same brands appeared in answers over time. They did not.
Instead, recommendations fluctuated significantly from one run to the next, even when prompts stayed the same. Some brands appeared once and never again. Others rotated in and out with no clear pattern. This inconsistency suggests AI answers are influenced by probabilistic generation, prompt phrasing, and model-side randomness rather than a stable ranking system like traditional search.
For marketers tracking AI visibility, this means single snapshots are misleading. A brand appearing in one AI answer does not indicate durable visibility, authority, or preference. Likewise, disappearing from an answer does not necessarily signal a loss in performance.
The research argues for a more cautious approach to AEO measurement. Instead of treating AI recommendations like rankings, marketers should focus on broader patterns over many prompts and time periods, and pair AI visibility tracking with qualitative review. AI answers can be informative, but they are not yet reliable scoreboards.
