Researchers Just Validated the Biggest Problem in AI Search Measurement

A newly published research paper from Alibaba researchers may have unintentionally highlighted one of the biggest challenges facing brands trying to measure performance in AI search. The paper, <em>Improving Item Discoverability in e-Commerce Search via Related Intent Generation</em>, is not about Answer Engine Optimization. It is focused on improving product discovery in AI-powered shopping systems. But hidden within its proposed architecture is an insight with significant implications for how brands evaluate AI visibility.

<h2>AI Is Beginning to Search Across Intent Networks</h2>

For decades, search engines treated each query as an isolated request. A user searched for 'best running shoes' and the system attempted to retrieve the best answer for exactly that query. The Alibaba researchers propose something fundamentally different. Instead of searching immediately, their system first generates a collection of related consumer intents before retrieving products. A simple search for 'best running shoes' might expand internally into concepts such as marathon running, trail running, stability shoes, lightweight shoes, wide feet, injury prevention, beginner runners, and budget options. Rather than treating these as separate searches, the system recognises them as part of the same underlying decision space. In other words, AI begins reasoning across a network of related intents instead of responding to a single prompt.

<h2>Why This Matters for AI Search Measurement</h2>

Today's AI visibility tools generally evaluate brands using a relatively small number of prompts — for example, 'best running shoes,' 'top running shoes,' 'best shoes for runners' — and calculate visibility based on whether a brand appears within those responses. But if modern AI systems are increasingly expanding a user's intent internally, a fundamental question emerges: are we measuring the right thing? If AI is effectively evaluating brands across dozens of related consumer intents, then a handful of manually selected prompts may only capture a small portion of the decision landscape. A brand may perform exceptionally well for general running shoes while disappearing entirely once the conversation shifts toward stability, injury prevention, or marathon performance. Those differences remain largely invisible when measurement is limited to isolated prompts.

<h2>From Individual Prompts to Prompt Domains</h2>

The research suggests that marketers may need to rethink the basic unit of measurement. Instead of asking 'How did my brand perform for this prompt?', a more useful question may become 'How does my brand perform across the entire family of related intents surrounding this consumer decision?' This shifts the focus from measuring individual prompts toward measuring what might be described as a Prompt Domain — a collection of semantically related prompts representing the broader decision space associated with a consumer need. As AI systems become better at understanding intent rather than simply matching keywords, evaluating entire prompt domains may become increasingly representative of how brands are actually surfaced.

<h2>Questions the Industry Should Be Asking</h2>

Although this research focuses on e-commerce retrieval, its implications extend well beyond shopping. The study provides additional evidence that AI search is becoming increasingly intent-driven rather than query-driven. That raises an important challenge for the AEO industry. Should AI visibility be measured across entire Prompt Domains rather than individual prompts? How consistently do brands perform as consumer intent shifts? Which types of intent cause brands to disappear from AI recommendations? What new metrics will be needed as AI search moves beyond keyword matching? The research does not answer those questions. But it strongly suggests they are the right questions to begin asking.

<h3>References</h3>

[1] Alibaba Research, <em>Improving Item Discoverability in e-Commerce Search via Related Intent Generation</em>, 2026.