Why one AI answer can't tell you "our brand doesn't show up"

One answer is a sample, not a result

Step on a scale once and you trust the number. An AI answer feels the same: see it once and it seems to be the whole truth. But AI answers behave less like a scale and more like the weather. The same spot gives slightly different readings from hour to hour. So before you conclude "we don't show up," you need to ask several more times under the same conditions and count how many times out of how many you appeared.

Once you know why the answers move, how to ask follows. There are three reasons.

Three reasons the same question gets different answers

① MemoryYour account knows youChatGPT can draw on saved memories and earlier conversations. An account that has talked about your brand a lot will surface your brand more easily. Your customers' accounts won't.

② PersonalizationPast chats shape the replyGoogle says Gemini's personalization from past chats is on by default. That is why the answer on your screen differs from the one on a customer's screen.

③ NatureIt simply varies a little every timeOpenAI explains that identical requests can return different outputs by default, and that even the reproducibility setting doesn't guarantee a perfect match. This wobble remains after you turn all memory off.

① and ② can be switched off; ③ cannot. That is why you still need repetition after fixing the conditions.

Turning off ① and ②: ask in a temporary chat

OpenAI's help page says a ChatGPT Temporary Chat doesn't use saved memories, isn't aware of previous conversations, and doesn't create new memories. It doesn't appear in your history either. In other words, it is the closest you can get to asking a ChatGPT that doesn't know you. One catch: custom instructions still apply if they're enabled, so turn them off while measuring.

Gemini has offered Temporary Chats since August 2025. They don't appear in recent chats or activity, and aren't used to personalize later conversations. Since personalization from past chats is on by default, asking in your usual window mixes your own history into the answer.

Same question, different window

Your usual window Asking an AI that knows you Draws on saved memory Personalized by past chats Your brand may be overrated The window where "oh good, we show up" comes too easily.

Temporary chat Asking an AI that doesn't know you No memory used No knowledge of past chats Closer to what a customer sees Measure only here, with custom instructions off.

Customers ask from accounts that don't know you. Imitating that condition is where measurement starts.

Handling ③: ask the same sentence three times

Even with memory off, answers wobble. So you ask more than once and record "N out of how many." I use three as the default. Three separates "0 of 3" from "2 of 3," and each extra run is still cheap in time. What matters more than the count is fixing the conditions: same model, same sentence, same window.

One measurement, step by step

01Fix the sentenceOne line a customer would typeExample"Recommend organic dog treats in Seoul"
02Fix the windowTemporary chat, custom instructions offWhyTo remove memory and personalization
03Fix the modelCheck the same model nameWhyA different model is a different measurement
04Repeat three timesNew temporary chat, same sentenceRecordMentioned or not, in which position, with what description
05Write it as a ratio"N out of 3"RecordIn a table with date, model and sentence

Repeat this once a month with the same sentence and only then can you say "it changed."

Looking once vs counting three times Illustration

Not real data, an illustration. On the first answer alone the two brands look identical. The difference only shows when you count three.

One more thing. Google says AI Overviews appear only when they add to classic Search and so often don't appear, and that AI Mode and AI Overviews may use different models and methods, so the answers and links they show will differ. So "it wasn't in the AI Overview once" isn't a conclusion either. Look at the same query again on another day.

A three-minute check for today

No tools, just your phone.

1 min · SentenceWrite one line a customer would askOne sentence someone who doesn't know your product would type. Leave your brand name out.
1 min · WindowOpen a temporary chat, check custom instructionsOpen ChatGPT's temporary chat and confirm custom instructions are off in settings. Note the model name.
1 min · RecordAsk three times, write "N of 3"Send the same sentence in a fresh temporary chat each time. Note whether your name appeared and in which position.

Where doing it by hand gets stuck

One question, one AI: three minutes. But with ten sentences customers might ask, both ChatGPT and Gemini to check, and the same conditions to repeat next month, that's sixty temporary chats and sixty rows every month. Sooner or later the conditions slip. A different model, a slightly edited sentence, or one tired look instead of three. From that moment the number can no longer be compared with last month's.

This is where AIOudit helps. Paste your page URL and it lays out, in one pass, how customers and AI read that page and what to fix first. Put it beside the "N of 3" log above and you can see whether the ratio moves after you change the page.

AIOudit cannot guarantee what an AI will answer on a given day. Nor can it see reasons outside the page: price, stock, ads, reputation, season. So the measurement log stays yours to keep.

How do customers and AI read your page? Find out with AIOudit.