How to measure AI visibility without fooling yourself

The problem with one answer#

Ask ChatGPT "What are the best project management tools for agencies?" twice, and you may get two different lists. AI tools generate each answer fresh, they may search the web differently each time, and they are updated often. So a screenshot showing that AI named you, or did not, is an anecdote, not a measurement.

What to measure#

The most useful single number is the named rate: the share of answers that name your company, across a fixed set of buyer questions.

Named in 18 of 48 answers (38%), across 12 questions, 4 AI tools, asked once each.

Next to it, track:

• Position: how often you are named first, or in the top three.
• Who is named instead: the named rate of each rival on the same questions.
• Accuracy: whether AI describes you correctly.
• Sources: which sites AI cites when it answers.

Choose questions carefully#

• Use questions buyers really ask, taken from sales calls, search terms and support tickets.
• Keep most of them unbranded. "Best tools for X" questions show whether AI recommends you on its own. A question that names you will almost always mention you.
• Cover the buying journey, from problem to comparison to choice.
• Keep the list fixed. Changing the questions changes the score. Add new questions separately so the trend stays comparable.

Get enough answers#

With few answers, the named rate swings widely by chance. If you are named in 3 of 5 answers, your true rate could easily be anywhere from about a quarter to over three quarters. With 3 of 50 or 30 of 50, the picture is much clearer.

Practical rules:

• Aim for at least 40 answers per check: for example, 20 questions on 2 AI tools, or 10 questions on 4 tools.
• Ask each question more than once when you can.
• Show the result as a range while numbers are small. Our How we measure page explains the ranges we use.

Tell real change from noise#

Before celebrating a jump or worrying about a drop:

1. Compare like with like: the same questions, the same AI tools, the same settings.
2. Check the size of the change against the normal swing. A move from 30% to 34% on 40 answers is usually noise.
3. Look for a trend over several checks, not one week.
4. Check what changed in the answers. Did a rival appear in new answers? Did a new source start being cited?

A simple monthly routine#

• Week 1: run your fixed question set and record the named rate, rivals and sources.
• Weeks 2 to 4: make one or two focused changes, such as a new comparison page or an updated review profile.
• Next month: run the same set again and compare.

If you want this done for you, Live AI Monitoring re-asks your questions on ChatGPT, Gemini, Claude and Perplexity every week, shows ranges, and flags a change only when it beats the normal swing. You can try it with sample data in the live demo.