Field Notes

ChatGPT Citations vs. the API: What Local Owners Should Know

A new paper found OpenAI's API only partly matched ChatGPT's citations. Here is what that means for how local service businesses track AI search visibility.

Published November 9, 2026 by the Digital Marketing Raleigh team.

Listen to this article:

If you own a pest control company, an HVAC shop, a plumbing business, or a roofing outfit, you have probably heard some version of this pitch in the last year: "We can tell you how often ChatGPT recommends your business." Some vendors now sell dashboards that claim to measure your visibility inside AI assistants. Those reports look precise. They have charts, scores, and week-over-week trends. A recent research paper, covered by Search Engine Journal, raises a fair question about what some of those reports are actually measuring.

According to the coverage, the paper found that answers pulled through OpenAI's API only partly matched the citations that real users see in the ChatGPT interface. In plain terms, the version of the model that a software tool talks to behind the scenes does not always cite the same websites as the chat window that your customers type into. Sometimes the sources overlap. Sometimes they do not.

This article explains what that finding means, why it happens, and what a local service business owner should do about it. We will skip the hype and focus on practical decisions: what to trust, what to question, and where to spend your marketing time and money. We will also be careful not to overstate the research. We have not reviewed the full paper ourselves, so we will stick to the general finding as reported and the reasoning that follows from it.

What the Paper Reported, in Plain Terms

The core idea is simple. Researchers compared two ways of asking OpenAI's models a question. The first way is the consumer product, ChatGPT, which millions of people use in a browser or phone app. The second way is the API, which is the programmatic doorway that developers and software companies use to send questions to the model and receive answers back as data.

Many people assume these two doorways lead to the same room. They do not, at least not exactly. The reported finding is that when you look at which websites get cited as sources, the API results and the ChatGPT results only partly agreed. Some sources showed up in both. Others appeared in one and not the other.

We are deliberately not quoting specific overlap figures here. Research numbers depend heavily on how a study was designed, what questions were asked, which models and settings were used, and when the data was collected. Repeating a number without that context would give you false precision, which is exactly the problem this research points to in the first place. The useful takeaway is directional: API output is not a reliable stand-in for what a real customer sees in ChatGPT.

For a local business owner, that matters because a growing number of "AI visibility" tools rely on API calls to generate their reports. If the API and the consumer product diverge, a report built on API data may be describing a slightly different world than the one your customers live in.

Why an API and a Chat Window Can Give Different Answers

It helps to understand why the two might differ. You do not need a technical background for this. Think of it like calling a company's main phone line versus walking into their front office. You are dealing with the same business, but the person you reach, the information they have handy, and the way they answer can vary.

Here are some general reasons an API response and a ChatGPT response may not line up. These are common, well-understood factors in how AI products are built, not claims about the specific study.

• Different configurations. The consumer app often wraps the model in extra instructions, tools, and settings that a developer calling the API may not replicate. Small differences in setup can change which sources the model reaches for.
• Different search behavior. When an AI assistant cites websites, it usually runs some form of web search first. The consumer product and the API may use different search tools, different search settings, or different rules for when to search at all.
• Personalization and context. A real user in ChatGPT may have memory turned on, earlier messages in the conversation, or a general location the product can infer. An automated API call usually starts from a blank slate with none of that context.
• Randomness. AI models do not always give the same answer twice, even to the same question. Ask the same thing on Monday and Tuesday and you may get different sources. That variation exists in both doorways, which makes any single snapshot less reliable.
• Timing and updates. AI products change frequently. A model update or a change to how search works can shift citations overnight, and the consumer app and the API may not update on the same schedule.

None of this means anyone is doing something wrong. It simply means the API is a different measurement instrument than the consumer product. If you weigh yourself on two different scales, you should not be shocked when they disagree, and you should not treat either one as the final word without checking.

What This Means for AI Visibility Tools and Reports

Let us be fair to the software vendors. Measuring AI visibility is genuinely hard. There is no official "rankings report" from AI companies the way there are some reporting tools for traditional search. Vendors have to build their own methods, and the API is the most practical way to run thousands of test questions automatically. Doing that through the consumer app at scale is slow, and automating the consumer app may run into terms of service problems.

So the API approach is understandable. The issue is how the results are presented and how much weight a business owner places on them. A report that says "You were cited in a certain share of answers this month" sounds like a measurement of what customers saw. If the underlying data came from the API, it is more accurately a measurement of what the API returned to a test script under specific conditions.

That distinction can lead to real mistakes:

• Chasing noise. If your score drops one week and rises the next, you might change your website, your content, or your budget in response to variation that has little to do with what customers experience.
• False confidence. A strong score might lead you to believe you are well represented in AI answers when real customers in your area see a different set of businesses.
• False alarm. A weak score might push you to spend money fixing a problem that may not exist in the consumer product.
• Paying for precision that is not there. Some tools carry a meaningful monthly cost. If the core data only partly reflects reality, you should weigh that cost carefully.

Our view is not that these tools are worthless. Many owners find them useful for spotting broad patterns, such as which competitors' websites keep showing up as sources, or which kinds of questions trigger citations at all. The problem comes when a directional signal gets treated like an exact count.

Questions to ask any vendor selling AI visibility reports

If someone is pitching you an AI visibility product or including it in a marketing package, these questions will tell you a lot about how seriously to take the numbers:

1. Does your data come from the API, the consumer app, or both?
2. If it comes from the API, how do you account for differences between the API and what real users see?
3. How many times do you run each question, and do you report the variation between runs?
4. Do your test questions include location context that matches how my customers actually search?
5. When the AI company updates its models, how do you flag that in your reports so I do not misread a sudden change?

A trustworthy vendor will answer these plainly, including the limits of their method. A vendor who gets defensive or vague is telling you something too.

What Local Service Businesses Should Actually Focus On

Here is the good news. You do not need a perfect AI visibility score to make smart decisions. The things that tend to help a local service business show up in AI answers are largely the same things that help you show up anywhere online. They are also things you control.

When an AI assistant cites sources for a question like "who is a good plumber near me" or "how much does termite treatment cost in my area," it is generally drawing on information that exists on the open web. That includes your website, your business listings, reviews, local news mentions, industry directories, and helpful articles. The clearer, more consistent, and more useful that information is, the better your odds of being represented accurately, whichever doorway the question comes through.

Make your website easy to understand

Your website should state plainly what you do, where you do it, and how to reach you. That sounds obvious, but many local service sites bury this information or spread it across vague pages. Each core service deserves its own page that explains what the service is, who it is for, what the process looks like, and what areas you cover. Write it the way you would explain it to a customer on the phone.

AI systems, like people, do better with clear writing. If your HVAC site has a page titled "Solutions" that never actually says "furnace repair" or names the towns you serve, you are making it harder for any system to connect your business to the questions customers ask.

Keep your business information consistent

Your business name, address, phone number, hours, and service areas should match everywhere they appear. Inconsistencies create doubt, both for search systems and for customers. If one listing shows an old phone number and another shows your current one, any system pulling information from the web has to guess which is right.

This is unglamorous work. It is also some of the most valuable work a local business can do, because it supports traditional search, maps, directories, and AI answers all at once.

Earn and respond to reviews

Reviews are one of the clearest public signals of how real customers feel about your business. Many owners find that a steady flow of genuine reviews, along with thoughtful responses, improves how their business comes across online. Ask satisfied customers for reviews as a normal part of finishing a job. Respond to negative reviews calmly and specifically. Never buy or fake reviews; beyond the ethical problem, it can backfire badly.

Publish genuinely helpful content

When people ask AI assistants questions, those questions are often practical. "Why is my AC blowing warm air?" "What does a roof inspection include?" "Are carpenter ants dangerous to my house?" If your website answers these questions clearly and honestly, you give AI systems and search engines a reason to point to you as a source.

Helpful content does not mean keyword-stuffed articles. It means answering real customer questions the way an experienced technician would. Our own Field Notes blog exists for this same reason: to explain things clearly rather than to fill space.

Build recognition beyond search

Here is a point that often gets lost in conversations about AI visibility. When a customer already knows your name, they search for you directly. They ask an AI assistant about your company by name, or they skip the assistant entirely and call you. Brand recognition shortcuts the entire question of who gets cited.

That is one reason we take memorable branding seriously, including audio branding. A short, catchy jingle on radio, streaming audio, or video ads can plant your name in a customer's head long before they need you. When the furnace quits in January, the name they remember is the name they look up. You can learn more about that side of our work on our jingles page.

A Practical Checklist for Owners

If you want something concrete to work through, here is a checklist you can use this month. None of it depends on a perfect AI measurement tool, and all of it supports your visibility across search and AI answers.

1. Search for your own business in a few AI assistants yourself. Use the consumer apps, the way a customer would. Ask natural questions like "best pest control company in Raleigh" or "emergency plumber near Cary." Note which businesses and sources come up. Repeat this on a few different days, since answers vary.
2. Ask about your business by name. Ask an AI assistant to describe your company. Check whether the information is accurate. If it gets your services, service area, or contact details wrong, look for where that outdated information lives online and correct it.
3. Audit your core listings. Check your main business profiles and directories for consistent name, address, phone, hours, and service areas. Fix anything that does not match.
4. Review your service pages. Make sure each main service has a clear page that names the service, explains it in plain language, and lists the areas you cover.
5. Write down the top ten questions customers ask your team. Your front desk and technicians hear these every day. Turn the best answers into clear content on your website.
6. Set up a simple review request process. A short text or email after each completed job, with a direct link, is often enough to keep reviews flowing.
7. Track what actually pays the bills. Phone calls, form submissions, booked jobs, and revenue by lead source tell you more about your marketing than any visibility score. Ask new customers how they heard about you and record the answer.
8. Treat AI visibility reports as one signal, not the verdict. If you use a tool, look at trends over months, not single weeks, and compare its findings to your own manual checks.

That last item is the heart of the matter. The research on API citations is a reminder that measurement tools have limits. Your phone ringing does not have those limits. If calls and booked jobs are rising, your marketing is working, regardless of what any dashboard says.

Why Manual Spot Checks Still Matter

It might seem old-fashioned to tell a business owner to type questions into ChatGPT by hand. In light of this finding, though, manual checks have real value. When you use the consumer app the way your customers do, you see something closer to what they see. You are not relying on an intermediate layer that may behave differently.

Manual checks have their own limits, of course. Your results may be influenced by your own account history, your location, and the randomness of AI answers. One person running a handful of questions cannot produce a statistically meaningful picture. But combined with other signals, manual checks give you a grounded sense of reality that a purely automated report cannot provide alone.

A reasonable approach for most local service businesses looks something like this: run a small set of realistic questions once or twice a month, in the consumer app, and keep simple notes. Which businesses come up? Which websites get cited? Is the information about your company accurate? Over a few months, you will notice patterns. Those patterns are more trustworthy than any single number.

Ask the questions your customers actually ask

When you run these checks, phrase questions the way a stressed homeowner would, not the way a marketer would. A homeowner with a leaking water heater does not type "top-rated residential plumbing services provider." They type something closer to "water heater leaking from bottom what do I do" or "plumber open now near me." The more realistic your test questions, the more useful your observations will be.

This is also a weakness worth watching for in automated tools. If a tool tests polished, marketing-style queries that real customers rarely use, its results may be even further from reality than the API difference alone would suggest.

How We Approach This at Digital Marketing Raleigh

Digital Marketing Raleigh is a done-for-you local marketing service operated by Petronella Technology Group, Inc. We work with pest control companies, HVAC contractors, plumbers, roofers, and other local service businesses. Our job is to handle the marketing so owners can focus on running their crews and serving customers.

When it comes to AI search, our approach is grounded in the same principle this research supports: be careful about what you measure, and focus on fundamentals you can control. We do not promise specific AI citation rates, because nobody can honestly guarantee how an AI assistant will answer a given question on a given day. What we can do is make sure your business information is accurate and consistent, your website clearly explains your services and service area, your content answers real customer questions, and your brand is memorable enough that customers seek you out by name.

We also believe in clear expectations about results. If you want to understand what we commit to, our guarantee page spells it out. We would rather be upfront about what marketing can and cannot control than sell a dashboard that looks impressive but measures the wrong thing.

When we report results to clients, we lead with the numbers that matter to a service business: calls, form leads, and booked work. Visibility data, including any AI-related observations, sits alongside those numbers as context. It helps explain trends. It does not replace them.

The bigger lesson for any new marketing metric

This finding about API citations is part of a larger pattern. Every time a new marketing channel appears, new metrics and tools follow quickly. Some turn out to be reliable. Others turn out to measure something adjacent to what they claim. Early on, it is hard to tell which is which.

The safest path for a business owner is to stay curious and skeptical at the same time. Learn what the new metric claims to measure. Ask how it is collected. Compare it against things you can verify yourself, like phone calls and customer feedback. And do not rebuild your marketing around a number until you understand where that number comes from.

AI assistants will likely play a growing role in how customers find local service providers. That makes it worth paying attention. It does not make it worth panicking or overspending on measurement tools that may only partly reflect what your customers see. The businesses that tend to do well through these shifts are the ones that keep doing the fundamentals well: clear information, honest content, good service, and a name people remember.

FAQ

What did the paper about OpenAI's API and ChatGPT citations find?

As reported by Search Engine Journal, the paper found that the sources cited in answers from OpenAI's API only partly matched the sources cited in the ChatGPT consumer app. Some citations overlapped, but many did not. The practical point is that API results should not be treated as an exact copy of what real users see in ChatGPT.

Does this mean AI visibility tools are useless?

No. Many of these tools can still reveal useful patterns, such as which websites tend to get cited for certain types of questions. The concern is treating their numbers as precise measurements of what customers see. Use them as one directional signal, compare them with your own manual checks, and focus on trends over months rather than week-to-week swings.

How can I check whether ChatGPT mentions my business?

Open the consumer app and ask natural questions the way your customers would, such as asking for a recommended service provider in your town. Also ask the assistant to describe your business by name and check whether the details are accurate. Repeat this on different days, since answers vary, and keep simple notes so you can spot patterns over time.

What can a local service business do to improve how AI assistants represent it?

Focus on fundamentals: clear service pages that name your services and service areas, consistent business information across listings, a steady flow of genuine reviews, and helpful content that answers real customer questions. These steps support traditional search and AI answers at the same time, and they do not depend on any single measurement tool.

Should I pay for an AI visibility report?

That depends on your budget and goals. Before paying, ask the vendor whether the data comes from the API or the consumer app, how they handle variation between runs, and whether their test questions reflect how your local customers actually search. If the answers are vague, be cautious. For many small service businesses, tracking calls, leads, and booked jobs gives a clearer picture of what is working.

Does brand recognition matter for AI search?

Yes, in a practical way. When customers already know your name, they often search for you directly or ask about your company by name, which sidesteps the question of which businesses an AI assistant happens to recommend. Memorable branding, including audio branding like jingles, can help keep your name top of mind for the moment a customer needs help.

If you want a clear, honest plan for showing up where your customers look, from search results to AI answers, we would be glad to talk it through. Call Digital Marketing Raleigh at 919-348-4912 or reach out through the contact page, and we will help you focus on the marketing that actually brings in calls.