Same API name ≠ same model — why silent upgrades need your own eval

TL;DR. The name on the slide and the string in the API are labels, not frozen products. A provider can change the checkpoint behind a stable ID, ship different surfaces under one family name, or add a faster tier that is still “the same model.” Pin what you evaluated — and re-test when anything moves.

Three ways the label drifts

Drift	Everyday picture	Recent public colour (not a product claim)
Silent checkpoint bump	Same street address; new tenants moved in overnight	A familiar API string points at a newer official build (e.g. DeepSeek documenting V4-Flash-0731 / V4-Pro-0813 behind stable IDs)
Surface split	Same brand of car; different trim for consumer vs work	One family name across ChatGPT vs Work / Codex-style surfaces with different checkpoints
Latency / infra tier	Same recipe; a faster kitchen line	“Ultrafast” (or similar) tiers that keep the model name and change speed / hardware path

None of these are automatically bad. They are easy to miss if your procurement or pilot notes only say the marketing name.

Why buyers get surprised

1. The demo was last month. The string still matches. The behaviour may not.
2. The RFP copied a slide. “We will use Model X” without a pinned build, surface, or tier.
3. Cost and latency moved first. A silent upgrade or a new tier can change bills and timeouts before anyone notices quality drift.
4. Compliance assumed a fingerprint. Auditors ask “which model?” — “the one named X” is not enough if X is a moving handle.

A calm checklist (bring to the next vendor call)

Clarify	Good enough answer sounds like…
Pin	“We evaluated this ID / checkpoint / surface / tier on this date.”
Notify	“You tell us before the string’s behaviour changes — or we re-test on a schedule.”
Golden set	“Here are 20–50 real tasks we re-run after any change.”
Pass rule	“Pass means citation / tool call / refusal — not ‘sounds fluent’.”
Rollback	“If the new build fails our suite, we stay on the pinned prior route.”

Soft OrchAI bridge (one short beat)

In the OrchAI suite, Library is where you choose and pin model routes under governance, and Eval is the quality habit — suites that catch regressions before a change reaches users. Eval’s shipping status is still Partial (monitoring / CI-gate framing); the literacy point does not wait for a perfect harness: judge the route you actually run, not the name on the brochure.

What this post is not

• Not a ranking of vendors.
• Not a claim that AiHPC evaluated any named third-party build.
• Not a reason to freeze innovation — only to notice when the label moved.

See also

• Strategy notes: DeepSeek V4-Flash · DeepSeek Harness / V4-Pro-0813 · OpenAI Sol/Luna surfaces · Sol Ultrafast
• Sibling literacy: Leaderboard vs your work · Hospital AI beyond the benchmark
• Series home: 科普 direction