How to choose the right prompts for AI visibility tracking

Before asking whether a brand is winning in AI search, ask which questions are being used to keep score. A dashboard built around broad “best provider” questions will tell a different story from one focused on price-sensitive customers, detailed comparisons or people considering a switch. The quality of AI visibility tracking starts with the questions selected, how they are grouped and what the resulting numbers are claimed to represent.

A prompt is the question or instruction entered into an AI assistant. AI visibility tracking repeatedly tests a selected set of prompts to examine how brands appear in the answers. Published provider research and academic preprints support covering distinct customer needs, preserving meaningful context and repeating measurements. They do not establish a universal number of prompts that makes a tracker representative of all consumers. The following nine steps combine those findings with proposed working practices, not experimentally validated allocation rules.

1. Define the customer decision before writing prompts

Start with a narrow brief: product, audience, geography, language and decision. “Banking” is too broad for a modest tracker. “Choosing a Canadian personal chequing account” is a more workable starting point. Mortgages, credit cards and business banking require their own coverage. Specify the target and positioning before asking a tool to generate questions; otherwise the tracker may test the easiest topics rather than the decisions the business has chosen to compete for.

Choose the outcome as well. Unprompted inclusion in a shortlist, an explicit recommendation, an accurate explanation of fees and advice to an existing customer are different results. A mention records that an answer names the brand; a citation records a source reference. Neither establishes that a person saw the answer or chose the provider. Peec’s setup guidance recommends organising questions by journey stage, customer context and geography. Use those dimensions to state what the tracker covers and what it leaves out. [1]

2. Build a question bank from evidence of customer demand

Gather more candidate questions than the budget permits tracking. Combine customer interviews and surveys with voluntarily shared, redacted AI conversations, sales and support questions, internal site searches, search-console queries and relevant public discussions. Record the source of each candidate. Current customers alone may miss the concerns of competitor customers, lost prospects and people entering the category for the first time. An AI tool can organise or expand this material, but its invented prompts are hypotheses, not evidence of demand.

Profound describes a proprietary process that retrieves real-user conversations, filters and clusters them, then selects prompts. That illustrates the value of observed behaviour without independently verifying the provider’s coverage. Ask about consent, collection method, country, language, engines and sampling bias before treating any proprietary question panel as the market. [2]

Demand scores need equally clear labels. Ahrefs describes web-search-volume weighting as a modelling choice, not a validated relationship with how often a question is asked in an AI assistant. Peec’s documented prompt-volume score is relative to the industry, uses a one-to-five scale and incorporates web-search interest. Neither is an exact count of ChatGPT users asking the tracked question. A search query can be useful evidence without being a measured AI conversation. [3] [4]

3. Group by decision need and remove redundant questions

Cluster candidates around the customer’s job: avoid monthly fees, retain branch access, compare savings returns or decide whether switching is worth the effort. Keep a representative question and a small number of materially different situations within each cluster. “Which chequing accounts have no monthly fee?” and “What are the best no-fee chequing accounts?” may cover similar ground. “Which no-fee accounts provide access to a local branch?” adds a requirement that could change the appropriate answer.

A Peec study reported in a sponsored Search Engine Journal article analysed 37,804 responses and found differences associated with prompt format and commercial intent, including sensitivity in unbranded middle-funnel questions. Its practical implication is to test meaningful differences without filling the tracker with cosmetic rewrites. Keep paraphrases under a common cluster ID. In a simple equal-prompt average, ten fee-related rewrites and one service question give fees ten times the influence, whether or not customer evidence supports that balance. [5]

4. Write natural questions without steering the answer

Use customer language and enough context to define the decision. Avoid favourable assumptions, promotional wording and instructions to include the sponsor’s brand. Do not add “list 20 providers” or “cite five sources” simply to make a dashboard easier to populate. Peec’s wording study reported that list requests and concise keyword formats could produce more brands than open-ended questions. Requested answer format can therefore change the opportunity to be mentioned. Keep it consistent within comparable groups and test useful variants during the pilot. [5]

Add decision-relevant context such as location, budget, eligibility or service requirements. A persona paragraph full of unrelated demographic detail makes the result harder to interpret. For multilingual markets, validate locally natural questions rather than relying only on literal translations. The goal is to test the intended decision, not to supply every detail that might persuade the model to favour a particular offer.

5. Separate unbranded discovery from branded evaluation

An unbranded question asks AI to supply the options: “Which Canadian chequing accounts suit someone who rarely visits a branch?” A branded question supplies an option: “How does Bank A compare with Bank B for someone who rarely visits a branch?” Both are useful, but their mention rates answer different questions. Peec’s September analysis distinguishes the comparatively predictable appearance of brands already named in a prompt from the more contested outcomes of unbranded questions. Naming a bank cannot establish that it earned discovery. [6]

For a tracker covering one category and market, the following 100-prompt allocation is an illustrative starting design. Each prompt receives one primary bucket; other tags can overlap. The first four buckets are written as unbranded questions in this example. The final two name the brands or incumbent being examined. Adjust the allocation using customer evidence, commercial priorities and the available budget.

Illustrative 100-prompt allocation for one category and market

Primary bucket	Prompts	What it measures
Broad category discovery	20	Which providers enter an initial shortlist
Customer needs and constraints	30	Who appears for distinct situations and requirements
Attributes and trade-offs	20	Which brands are associated with value, service or relevant capabilities
Decision and switching process	10	What information and providers appear when people prepare to act
Branded comparison and validation	10	How named brands are evaluated and compared
Existing-customer loyalty and switching	10	Whether AI advises staying, renegotiating or leaving

This creates 80 unbranded prompts and 20 branded diagnostics, not an estimate of actual query shares. Report those groups separately and show the number of distinct needs covered, not just the total prompt count. The suggested allocation is not a validated representative sample or a weighting scheme for consumer exposure. A tracker spread across many products, countries and languages needs a larger budget, a narrower claim or both.

6. Pilot the set before establishing the baseline

Run candidates through the intended engines and inspect the full answers. Check that each question is interpreted correctly, uses the intended market and produces relevant information. Review duplicates and omitted customer situations before locking the list. Do not drop a relevant question because the sponsor performs poorly, or force a useful informational question to produce a recommendation. Score it on an appropriate outcome, such as factual accuracy or source use.

For consequential questions, try a few natural paraphrases and repeat the runs. Distinguish wording sensitivity from ordinary variation between answers. If a small wording change materially alters the shortlist, keep the variants as a labelled group rather than selecting the most flattering result. A pilot is a chance to improve the measurement design, not to remove unfavourable evidence.

7. Budget for breadth and repetition separately

More distinct questions improve coverage; additional runs improve estimates for the questions already selected. Peec’s September analysis, drawing on approximately 120 million monitored chats from February to August 2026, favoured broader prompt coverage for estimating portfolio-level visibility under its modelled run allocation. Its provider dataset is not a representative audience survey, and wider coverage does not by itself make a hand-picked panel representative. [6]

Graphite’s principal experiment collected 400 API responses for each of 200 entity-comparison prompts. It used half as an estimated reference, not known ground truth, and repeatedly subsampled the other half. It described roughly ten responses as a quick-estimate starting point, with more sampling needed for greater precision. Results differed by entity visibility; a separate completed comparison found differences between API and scraped logged-out ChatGPT and Gemini responses. Ten runs is not a guarantee that a small change is meaningful. The University of St. Gallen preprint <em>Don’t Measure Once</em> also found repeated-answer variation, but its proposed thresholds used a limited ten-run reference dataset in selected Swiss-German campaigns. Those thresholds require methodological caution and are not general Canadian-market requirements. The shared practical conclusion is to repeat measurements, not adopt a universal minimum. [7] [8]

Daily collection, weekly review and longer-window reporting can be a workable operating schedule. Investigate important changes with additional repeated tests. A month of daily collection measures performance over that month; it is not the same experiment as repeating the question under unchanged conditions. Budget explicitly: 100 prompts × three engines × one market configuration × one run per day over 30 days equals 9,000 scheduled responses. Those are still 100 distinct questions, not 9,000 independent customer needs. Successful responses may be fewer, and vendor billing units may differ.

8. Add a separate test of conversations and customer loyalty

Single-turn questions provide a controlled benchmark but cannot reproduce every user’s conversation history, settings or personal context. Demand-Genius reported a September follow-up across four models and eight B2B categories in which earlier conversational framing affected later recommendations. This is vendor research with model-dependent effects and limited scope. It measures generated answers, not real buyers’ behaviour or Canadian financial-product switching. The report itself says the OpenAI result was difficult to separate from natural response variation. It nevertheless supports testing relevant conversation paths alongside isolated questions. [9]

A telecom scenario might start with “My internet bill has increased”, add a synthetic household profile and plan details, then ask “Should I keep this plan, negotiate or switch?” A banking scenario could examine fees, service needs and switching effort. Record whether AI advises staying, renegotiating or leaving, which alternatives it offers and whether the reasoning is accurate. Naming the incumbent makes its appearance expected, not a visibility win.

Use synthetic or consented, redacted information without account numbers or private application details. Check that the research setup genuinely retains history between turns and supports any claimed attachments. If it does not, run a separate module or label the self-contained text scenario as a proxy. It should not be described as an observed multi-turn conversation or bill-upload experience.

9. Freeze the benchmark and document every change

Assign permanent prompt IDs and record wording, source, intent cluster, branded status, market, language, engine, collection surface, scoring rule and version date. Record model versions when available. Run fresh sessions under a consistent memory and history policy. Keep API, browser and conversation experiments distinguishable: Graphite’s interface results show why the collection surface belongs in the methodology. Review engines separately before combining scores and disclose any weighting. [7]

Maintain a stable core for trending and an exploratory set for new needs. Comparable human-survey waves can reveal changing priorities, brand associations and reported AI tasks, informing exploratory questions without silently replacing the original benchmark. Human research is needed to understand people; model-answer tracking describes the tested information environment. Where the core must change, mark the break, compare common prompts and, where feasible, overlap old and new versions before retiring the old set.

Treat collection errors as missing data rather than brand absence. Retain valid answers with no brands and report refusals and no-recommendation answers separately. A visibility rate restricted to recommendation-bearing answers has a conditional denominator that must be labelled. An illustrative statement such as “The brand appeared in 32% of valid responses to this tracked prompt set” is not evidence that 32% of consumers saw the brand. The percentage is a reporting example, not a new research finding.

AEO Updates Takeaway

The benchmark should represent the customer decisions a brand wants to understand, not the questions on which it already performs well. An agency’s deliverable should explain why the prompts were chosen, whose needs they cover, what is omitted, how answers are scored and how comparisons will remain stable. This extends the mortgage AEO playbook’s approach to human research and AI tracking and the earlier warning that the prompt is not the market. Prompt coverage, answer stability and human demand all matter, but they remain different forms of evidence.

References

Research reviewed 29 October 2026. Source verification is faithful extraction from the reviewed publications, not reproduction of proprietary datasets or underlying AI answers. The allocation, operating schedule and Canadian examples are proposed applications.

[1] Peec AI: How do I know which prompts to track?

[2] Profound: Introducing Prompt Research Reports in Profound

[3] Ahrefs Brand Radar Methodology

[4] Peec AI: Setting up your prompts

[5] Malte Landwehr / Peec AI: What Matters In An AI Prompt? Intent or Keywords?

[6] Jan Ehrlinspiel / Peec AI: The key to prompt tracking

[7] Gregory Druck and Ethan Smith / Graphite: Demystifying Randomness in AI

[8] Schulte, Bleeker and Kaufmann: Don’t Measure Once: Measuring Visibility in AI Search GEO

[9] Tom Rudnai / Demand-Genius: Studying how conversational context impacts AI responses across different models