OpenAI's API is not ChatGPT's citations. Stop measuring the stand-in.

If your GEO dashboard is built on Google SERP pulls, OpenAI's Responses API with web search, or both, you are measuring a proxy. A new observational study says that proxy recovers only about a quarter of the exact pages ChatGPT Web actually cites.

On October 11, 2026, researchers Yunxuan Jiang, Shiqi Zhang, Anqi Xing, Xiangyu Chang, and Xiaokui Xiao posted Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations on arXiv (HTTP 200 this fire; abs, HTML, and PDF all opened). Affiliations on the abs page include National University of Singapore, Xi'an Jiaotong University, and SGIT AI Lab. Search Engine Journal covered the preprint on October 18 (OpenAI's API Only Partly Matched ChatGPT's Citations, Paper Finds, Matt G. Southern, HTTP 200 this fire).

The abstract's line that matters for operators:

"Even when combined, they recover only about one quarter of cited URLs."

Source: Jiang et al., arXiv:2610.15382, abstract (opened this fire).

That "they" is Google Search plus the OpenAI Responses API — the two stand-ins many GEO tools and DIY scripts already treat as "what ChatGPT cites."

The short version

• Retrieval is not citation. ChatGPT Web retrieves candidates, then cites a subset. The paper's GEO implication section says you should distinguish retrieval visibility from citation visibility.
• The two common stand-ins miss most exact URLs. Combined Google organic results + Responses API candidates recover 28.3% (chatgpt-5.3-instant) and 26.2% (chatgpt-5.5-instant) of cited URLs, and 50.3% / 48.7% of cited hostnames (Table 4 in the HTML, opened this fire).
• Hostname and entity match better than exact URL. Both proxies align more closely with ChatGPT Web at hostname and entity levels than at deep-normalized exact URLs. Tracking only the precise page URL understates presence.
• Run-to-run variation is large. Across 20 runs per query, 40.8% (5.3) and 44.5% (5.5) of URL candidates appear in only one run; the "stable core" (≥14 of 20 runs) is 9.4% / 7.7%.
• A supervised model on external candidates can look strong and still leave most citations outside the set. AUCs above 0.74 for 5.3 and 5.5 at URL/hostname/entity are conditional on the Google ∪ Responses API candidate pool — not "you can predict ChatGPT citations from the open web with 74% accuracy."
• This is a preprint. SEJ notes arXiv does not peer-review papers. OpenAI and Google did not confirm these figures. Settings in the study predate GPT-5.6 / GPT-6 per SEJ (opened this fire).
• Monitoring is not execution. Proving a stand-in fails is not the same as shipping the page ChatGPT can see and choose to cite.

What the researchers actually did

Piece	What the paper reports (opened this fire)
Query set	80 queries from Search Arena across four intent types (Factual Lookup, Information Synthesis, Analysis, Recommendation) and two languages (40 English / 40 Chinese per SEJ and the paper's Section 2)
ChatGPT Web settings	Four settings from 5.2 Instant through 5.5 Instant; Instant mostly single-round; GPT-5.4 Thinking often multi-round
Main Instant sample	chatgpt-5.3-instant and chatgpt-5.5-instant: 20 independent runs per query → 1,600 runs each
Data window	Majority of collection March 29 – July 29, 2026 (paper Section 2; SEJ same dates)
Google proxy	Up to five pages of organic results via SerpAPI; average ~43 organic results per query (paper: 43.2; SEJ: "about 43")
Responses API proxy	gpt-5.3-chat-latest and gpt-5.5 with web search; 20 invocations per query; candidates from web_search_call.action.sources[]; citations from url_citation; 1,600 calls per model
Match levels	Exact URL (deep_normalize), hostname (subdomains kept distinct), entity

OpenAI's platform documentation for web search in the Responses API (HTTP 200 this fire) describes a product path where the API returns sources and URL citations. That is a real surface. The preprint's point is narrower: that surface is not a faithful stand-in for what ChatGPT Web cites in the product UI.

The recall table that should change how you read dashboards

From Table 4 (URL / hostname full-set visibility), macro-averaged over the 80 queries, for pages/hostnames cited in at least one of 20 ChatGPT Web runs as the paper defines:

Setting	Proxy	URL recall	Hostname recall
5.3 Instant	Google alone (G)	9.4%	26.1%
5.3 Instant	Responses API candidates (A_cand)	22.9%	38.3%
5.3 Instant	G ∪ A_cand	28.3%	50.3%
5.5 Instant	Google alone	6.4%	22.8%
5.5 Instant	Responses API candidates	22.5%	38.1%
5.5 Instant	G ∪ A_cand	26.2%	48.7%

Responses API citations (A_cite) trade coverage for precision: higher precision and better ranking alignment with ChatGPT Web than candidates alone, but lower recall than A_cand. Google alone is the weakest URL stand-in in this table.

Read that the way an operator should: if a tool only asks Google or only asks the API, and then reports "ChatGPT citation share," it is reporting overlap with a set that misses most of the exact URLs ChatGPT Web cited in this study. Hostname presence is less bad — about half recovered when both proxies are combined — and still not the product.

Why this is distinct from our other measurement posts

We have already written that GEO measurement is noisy, that a measurement layer now exists, and that missing ChatGPT UTM tags can make GA4 look like a crash when the problem is attribution. This post is a different failure mode:

• not "the tag fell off the click"
• not "third-party tools disagree"
• the API/SERP stand-in does not contain most of the URLs ChatGPT Web cited

Cloud AI cites who it can see. If your measurement stack never sees the page ChatGPT cited, you will optimize for the wrong universe — the pages Google and the API happen to share — and call it ChatGPT GEO.

What to do instead of trusting the stand-in

1. Label the instrument. In every report, write which surface you measured: ChatGPT Web UI (model + mode + date), Responses API (model + web_search), Google organic, or a vendor black box. Do not collapse them into one "ChatGPT" column.
2. Separate retrieval from citation. Being in a candidate set is not being cited. The paper's Section 6 says the same in GEO language. Your proof loop needs both: "did we appear as a candidate?" and "did we get the citation mark?"
3. Track hostname and entity, not only exact URL. Exact-URL recall is the hard bar. Hostname and entity presence are more stable across runs and more recoverable from proxies — still incomplete, but less misleading than URL-only scorecards.
4. Repeat the prompt. One run is a sample, not a census. The paper's one-off rates (~41–45% of URL candidates) are the reason a single screenshot is weak evidence. Date-stamp model, mode, and run count.
5. Re-check after model or product changes. Authors recommend re-evaluating when the deployed setting changes. SEJ notes every setting in the paper predates GPT-5.6 and GPT-6. Until someone repeats the comparison on current defaults, treat the recovery % as describing March–July Instant settings, not forever.
6. Keep first-party proof. Indexed, crawlable pages; clear task and entity language; consistent site identity. Google's standing AI features and your website guidance (HTTP 200 this fire) still says supporting links in Google AI features require a page that is indexed and eligible for a snippet, with no special markup. That is Google's surface — not ChatGPT — but the operator habit is the same: conditions and proof, not a dashboard promise.

That is the proof loop: name the surface, separate retrieve vs cite, measure at URL and hostname/entity, repeat runs, re-baseline after model changes, keep shipping the page.

What this is not (honesty block)

• Not peer-reviewed. The paper is an arXiv preprint. SEJ (opened this fire): "The paper is an arXiv preprint. arXiv does not do peer review of papers."
• Not an OpenAI or Google confirmation. Neither company confirmed these recovery figures in anything opened this fire. OpenAI news/help returned 403 this fire; we do not invent a product statement.
• Not a claim about every GEO vendor. The paper did not test commercial tracking tools (SEJ). Do not name tools you did not open.
• Not "74% accuracy predicting ChatGPT from the open web." AUCs above 0.74 are for a supervised model ranking candidates already inside Google ∪ Responses API. Cited URLs outside that set never enter the task. End-to-end URL recovery stays in the mid-20%s in Table 4.
• Not current default ChatGPT. SEJ: every setting in the paper predates GPT-5.6 and GPT-6. Figures describe the Instant-focused March–July 2026 window in the study, not tomorrow's product default.
• Not a reason to abandon all API/SERP measurement. Responses API candidates still beat Google alone on recall in this table, and API citations align more closely than Google alone. Use them as partial signals, labeled honestly — not as ChatGPT Web ground truth.
• Not a guarantee that repeating prompts or publishing more pages increases citation probability. The authors treat recurrence as observational, not causal.

Related New Reward Field Guides

• GEO measurement is noisy. Build a proof loop anyway.
• GEO has a measurement layer now. That is not the same as a fix.
• ChatGPT UTM tags are missing. Direct traffic is not a visibility crash.
• Reddit didn't vanish from ChatGPT citations. The share still matters.
• AI visibility tools for ChatGPT and Perplexity: what brand tracking actually shows
• Google is testing a top slot for the official page. Make sure it's yours.
• Bing now shows the AI queries you're missing. That's a gap list, not a fix.

Want the fast read first?

I'm not going to pretend a scan is a strategy. The free AI-visibility score is a quick check: email only, no credit card, a 0–100 read across the engines we track. It will not recover the three-quarters of ChatGPT citations a SERP/API stand-in misses in this preprint. It will tell you whether AI can see you at all, which still comes first.