This article was written with AI (Claude) and reviewed by an editor. Our company is a paying customer of both Anthropic and OpenAI, and we also use Google models in testing. Item 3 is a post written by the maker of the AI that wrote this article (Anthropic) about its own model; the Ramp post in item 2 contains a sentence saying Anthropic's share grew, the rankings table in "In brief" lists an Anthropic model in second place, and the assistant on PayPal's case-study page uses Claude as its front end. For that reason, we add no assessment to any side and only carry over the numbers and sentences. Figures announced by companies are marked as "announced by the company," and anything from a single source is noted with that source. This article is not investment advice on any specific company.
In NO.04 we wrote "five criteria a small team works by when models change every week," and in NO.46 we wrote about the speed at which the "daily driver" slot changed within two days. This week's numbers are the other side of that — not how fast models change, but what it costs us to change a model. We read one survey, one set of payment data, one company case, and one post in which a model maker says "drop this habit," dividing them into "tied to the model / kept outside the model." We write in three parts: facts, interpretation, and what we would do.
1. VB Pulse — 47% of companies swapped a production model, and 69% of those who did said "weeks or more" (facts)
VentureBeat's research arm, VentureBeat Intelligence, published its October survey ("VB Pulse — Agentic Orchestration") on November 12 as an article and a report page. The sample is small. 100 qualified respondents out of 106 (72% people doing AI work at organizations with 100 or more employees, 28% consultants serving such organizations; 88% came through a survey panel), organization size over 10,000 for 31% and 100–250 for 23%, multiple respondents from the same organization in 18 cases, and the report itself notes that "as a self-selected sample it is hard to generalize to all enterprises."
Here are the numbers. 47% of companies swapped the model in a production AI system within the past 12 months while keeping the app. Among the 45 respondents who had done so, the engineering required beyond changing the model name was — "almost none, a configuration change" 13%, "days" 9%, "weeks" 33%, "a month or more" 36%, "don't know" 9%. Weeks or more adds up to 69% (the article body rounds it to 70%; the report says 69% — this article uses the report's figure). Outcomes were "the new model was the same or better" 56%, "performance dropped and we reworked the app" 33%, "we reverted to the original model" 11% — reworked or reverted combined, 44% (the article says 45%). Compared with the September survey, "a month or more" rose from 12% to 36%, but the article said the respondent mix differed so read only the direction, and noted that the 44% is not a statistically significant difference from September (58%).
The survey did not ask "why did you switch" or "what did you fix." The article gives four examples. (1) Prompts, tools and tests are tuned to one model and have to be reworked (including the "harness" that determines which tools can be called). (2) Evaluation sets must be built from real requests, and gathering that data is the hardest part (Box's Ben Kus). (3) Fine-tuning does not carry over to the next model — PayPal's Len Zhang said "when a new model comes out, you have to fine-tune again on that model." (4) The same model yields very different results in different harnesses — in one benchmark cited by the article, Kimi K3's "median cost per passed task" was $18.34 in one coding harness and $3.28 in another, with the same 63.3% pass rate (an example from a single article). JPMorgan is described as putting each new Claude model through internal approval before employees use it, and Upwork's Sudhanshu Bist said that for a production system he would "probably keep the existing model." On the side of making swaps easier, the article lists gateways and routers (change one line, but check the app separately), keeping business knowledge outside the model (PayPal, in item 2), testing in the harness you will actually run, measuring by "cost per good outcome," and doing what T-Mobile did — running the old and new versions side by side on real calls and scaling only the better one.
2. Ramp: "8% switched providers in September"; PayPal: "same model, 43% to 95%"; Uber: "cost per cleanup down 90%" (facts)
Ramp (a US corporate card and spend management company) published a post from its data team on October 6 titled "Businesses are changing AI model providers more often" (by Ryan Stevens, Director of Applied Science). The share of businesses that switched AI model providers hit a record 8% in September, a monthly rate on a three-month average. The post describes two movements — Anthropic greatly increased its share after the Opus 4 and Sonnet 4 launches in May 2025 and switching declined for the following year, and OpenAI's price cut and new model releases in June 2026 broke that balance. It says there have been nine or more model release announcements from the two companies since June. The author wrote, as a possibility, that "this may be an early signal that models are becoming a commodity." The post does not state the sample size, the criterion for counting a "switch," or the sample composition (Ramp's November AI Index itself notes that "customers skew toward tech and finance" — NO.47). The sentence about Anthropic's share growing is carried over as Ramp wrote it, and we add no assessment.
PayPal's figures come from a single VentureBeat article dated November 10 — a write-up of remarks by two PayPal executives at VentureBeat events on October 8 and October 28, not an official PayPal post. Len Zhang (Vice President and Head of AI Product) said the internal accuracy of an internal data science assistant (one that answers business questions that used to go to the data science team) rose from 43% to 95% with "the LLM unchanged." What changed was the knowledge layer outside the model — "golden queries" that define which data to pull, and how, for which purpose; "analysis playbooks" that describe how to dig into a question, such as login failures or passkey usage; and business definitions such as "what is a new account," given as knowledge instead of leaving the model to infer them from data. How 43% and 95% were measured, and over what period, is not in the article (the 1,000+ questions over 10 weeks and the estimated 1,600 hours saved are a separate sentence, and the article itself added that "question volume does not prove productivity"). In the same article, PayPal said it tested four to five models for a customer-facing commerce agent — large models had latency of around 15 seconds, an 8-billion-parameter model was fast but less accurate, and fine-tuning improved it but not enough — and that it reduced its reliance on fine-tuning because of the cost of retraining every time the model changes (not abandoning it entirely). For reference, Google Cloud's PayPal customer case page (undated) describes the same kind of assistant — Looker's MCP server placed on an internal MCP hub, queried in natural language from Claude Desktop, roughly 5,000 users, with metric definitions fixed by Looker's semantic layer. That page's "nearly 100% accurate against defined metrics" is a vendor-side claim with no baseline or method, and the 43%→95% figure does not appear there. We also could not confirm whether the two articles refer to the same assistant.
Uber appears in a different VentureBeat article from November 10 (the VB Pulse infrastructure survey — 156 respondents, 38% "delayed, scaled back or canceled an AI project because of cost overruns" versus 12% for "compute shortages"). Uber's Uday Kiran Medisetty (Vice President and Distinguished Engineer) said that after benchmarking managed agents on real tasks to choose a model, "cost per cleanup" fell 90%, with quality measured by reverts and incident counts and cost measured "per completed review or cleanup." The article notes that the 90% is a figure specific to Uber's tasks.
3. Anthropic — "drop these seven old prompting habits," one of which comes from a 2022 paper (facts, conflict of interest)
On November 12, Anthropic posted an article on claude.com titled "Using Claude Opus 5.5 as your daily driver for knowledge work" (by Aleksandra Todorova; summarized by The Decoder on November 14). Because it is a post by the maker of the AI that wrote this article about its own model, we carry over only the list and the figures the company stated. Opus 5.5 is the default model on the Pro, Max, Ultra and Team plans, and the post names seven habits to change — (1) adding phrases like "think hard" (said to be unnecessary because this model always thinks); (2) starting the effort level at High (start at the default Medium and raise it only when it falls short); (3) giving instructions one step at a time (state the goal, the output format and when to ask back, all at once); (4) asking for a draft or outline before the finished version; (5) transcribing chart numbers by hand (attach the image or file instead); (6) vague prohibitions like "avoid generic design" (be specific about what to avoid); (7) splitting work so only complex tasks go to Opus and the rest to Sonnet (the post says running Sonnet at a high effort level brings cost per task close to Opus 5.5).
The figures the company stated are as follows, and all are the company's internal tests. Opus 5.5 at Medium matched or beat Opus 5 at High on its own coding and knowledge-work evaluations; removing the "think" phrase made responses start faster with no clear drop in quality; the lowest effort level read dense charts more accurately than Opus 5's highest level; 16 of 18 reports built from saved web snapshots passed an automated check for "fabricated figures and citations"; and a merger-model task took 63 minutes (Opus 5: 93 minutes) at half the cost. The statement from the outside company Factory that it "got the same results as Opus 5 High at Medium with 20–25% fewer output tokens" is also a quotation in the company's post. The price table in the post is Haiku 5.5 0.10/0.50, Sonnet 5.5 2/10, Opus 5.5 4/20, Fable 5.2 8/40 (input/output per million tokens).
We separately checked the origin of habit (1). According to the Wikipedia entry "Prompt engineering," "chain-of-thought," which has the model write out intermediate reasoning steps, was reported in 2022 by Google Brain as a few-shot technique with examples attached, and the finding that a single sentence, "Let's think step by step," is enough to have an effect comes from a paper the same year by researchers at Google and the University of Tokyo (Kojima et al., NeurIPS 2022). The same entry notes that a meta-analysis of some 100 papers found the gains concentrated in math, logic and symbolic reasoning, with small gains on other tasks and lower accuracy on some, and includes the sentence that "the marginal value of elaborate prompt construction is declining" as instruction-following and reasoning improve. In other words, a habit created by a 2022 paper is one the maker is telling us to "drop" in 2026 — but the only figures so far showing that dropping it helps are that maker's own tests.
Two in brief (facts)
MathArena. The math competition evaluation site run by ETH Zurich's SRI Lab and INSAIT added results for Gemini 4 Carbon and Claude Fable 5.2 on November 13 (we opened it ourselves on November 15, and The Decoder reported the same table the same day). The overall "expected performance" (the average normalized score across non-deprecated competitions) is Gemini 4 Carbon (high) 85.4%±3.0, Claude Fable 5.2 (max) 82.8%±5.3, GPT-6.1 Sol (max) 80.6%±2.7, and the highest open model is Kimi K3.1 Pro (max) at 54.3%±5.8. Cost is "the average cost of solving one problem once," at $15.70, $40.20, $1.07 and $3.70 respectively. The gap between first and third is 4.8 points and roughly 15x in cost, and the intervals of first and second (±3.0 and ±5.3) overlap. The Decoder wrote that Carbon's most common failure was "timeout," and that in one case it ran a script for more than 11 hours without checking progress (The Decoder's primary source is an evaluator's X post; we confirmed with the table on the site). Because the model in second place is Anthropic's, we add no assessment of the rankings.
eBay. We opened eBay's User Agreement directly (the current version took effect for existing users on June 28, 2026). Section 3 prohibits the use of robots, spiders, scrapers and other automated means "without eBay's express written permission," and gives as examples, in parentheses, "buy-for-me agents, LLM-driven bots, or any end-to-end flow that attempts to place orders without human review." According to PPC Land (November 14), this wording is not new but has been there since the January 20 revision (effective February 20), and the report that Meta's Muse access to eBay became unstable in early November is relayed from The Verge's November 10 story (which would not open on our end). The eBay spokesperson statement PPC Land carried is that "we have restricted agent access in some cases, including unauthorized scraping and training on our data," and no list of permitted agents or application process has been published. It is a sentence that sits in the same row as United (NO.45, 47).
Reading the three together (interpretation)
First, the cost of changing a model is not the cost of the model but the cost of everything around it. In a gateway, the model name changes in one line, yet seven in ten of those who swapped said "weeks or more," for exactly the reasons the article gives — prompts, tools, tests and fine-tuning are tuned to that model. So "the share of companies switching models often rose to 8%" (Ramp) and "a swap takes weeks" (VB) are not opposites; they are the same cost seen from both sides (interpretation). That said, both numbers have small samples or unstated compositions, so we use them to read "what is tied" rather than "how much."
Second, the only things that carry over to the next model are the ones kept outside it. What PayPal improved was not the model but "the definition of a new account" and "this is how you dig into this question," and those are assets that remain usable when the model changes. Fine-tuning, by contrast, is a cost that disappears along with the model, which is why PayPal reduced its reliance on it. In NO.05 we wrote that "instructions given to AI are a liability, not an asset," and with this week's numbers we cut that sentence once more — instructions with a model's name baked in are a liability; business definitions, worked answers and test sets are assets (interpretation). Uber's 90% is also a number that came not from "choosing the model well" but from having a benchmark made of real tasks (the caveat that it is specific to Uber is carried over as the article gave it).
Third, habits are tied to the model too. "Think step by step" is a sentence a 2022 paper created, and four years later the maker wrote that the sentence slows its model down. Whether that is true is something we have to measure ourselves — the figures in the maker's post are all the maker's tests, and that company is the company behind the AI that wrote this article. But the question itself — "which of the habits baked into our prompts came from which model?" — is valid regardless of the model (interpretation). MathArena's table is the cost side of that question — a 4.8-point score gap at 15x the cost, with first and second place overlapping in their intervals. NO.40's "a one-point gap is within the interval" and NO.46's "price per task, not price per token" show up in the same shape in math problems.
What we would do — three things for a team running apps, commerce and AI channels this week (in practice)
- Run one "model swap rehearsal" and write down the time it took. Take the three tests from NO.17 and NO.46 (ingredient-label judgment, app copy, Ria's script) and actually switch them from our current default model to one other model, then record the time in four columns: "changing the model name / fixing prompts / fixing tool calls / re-aligning tests." We do not have to go all the way — whichever column takes longest is our list of "what is tied." Next to the result, write "cost per good outcome" the way the VB article counts it.
- Build a box that sits outside the model. Separate SafeList's judgment criteria (say only as much as "no avoided items were found" — NO.03), our commerce terminology (what "new customer," "cancellation" and "return" mean), and 20 worked answers for ingredient-label judgment into separate files rather than prompts, and change the prompts so they only point to those files. Add one line to NO.05's "delete instructions every quarter" check — flag "any sentence that presupposes a model's name or a specific model's quirks," and delete those sentences first at swap time.
- Test the version with "think" phrases removed against our own numbers. A maker's recommendation is only a recommendation, so run the prompts for the three tests with and without "think hard"-type sentences at the same effort level, 20 runs each, and record accuracy, time to first token and output token count. Whichever way it comes out, keep it in our table as two rows: "announced by the company / confirmed by us." For commerce, just one line — after looking at the eBay and United wording, check whether our own terms have a clause on "end-to-end automated ordering without human review" and, if so, whether an application route for permission is written in, and attach it to the draft of the "why" column in NO.45.
What we did not pick this time
The three The Decoder posts of November 13–14 on Anthropic's IPO roadshow, CEO remarks and a former researcher's claims are investment-decision material with a conflict of interest, so we did not use them. Analysis of Nvidia's executive reshuffle (The Decoder, VentureBeat) is outside our topic. Runway's release of Praxis-1 weights (Apache 2.0, robot control) and OpenAI's robotics team reassignment come from The Decoder alone, so we leave only the headlines. Anthropic's Claude in Google Meet (VentureBeat, November 12) and self-hosted Claude Managed Agents (Help Net Security, November 12) are conflicts of interest, so we note only the headlines. Help Net Security's "a pinned MCP server may not be pinned" (November 13) and its "stolen Claude API keys" article are security incidents, so headlines only — once an official notice comes out we will cover them as a follow-up to NO.32 without describing the methods. PPC Land's report that "the EU proposed a fine of 6% of global revenue for paid AI recommendations" is on hold because we could not open the official document. The two Skift articles of November 11 (hotel chain MCR's CEO on "banning AI bot bookings" and Citi on "AI companies should pay for mistaken bookings") and the November 9 "check of AI crawler directives on 50 travel sites" are follow-up candidates for NO.37 and 45 as we did not open the full text. The title of another October VB Pulse report — "78% of agents' confident wrong answers came from missing or conflicting business definitions" — points the same way as item 2, but we did not open the full text, so we note only the title. As of November 15, Meta's newsroom still has no post on Muse's phone feature (NO.47).
What we could not confirm in this story
- The VB Pulse swap group is 45 respondents. "Weeks or more 69%" and "a month or more 36%" are shares within those 45, and the report described the difference from September as "direction only" and "not significant." The article's 70%, 45% and 55% differ in rounding from the report's 69%, 44% and 56%.
- The Kimi K3 example in which cost per task split into $18.34 and $3.28 depending on the harness is one benchmark cited in the VentureBeat article, and we did not open that benchmark.
- Ramp's "8% in September" has no sample size, counting criterion or sample composition in the post. That Ramp's data consists of corporate card payment records is an explanation from other Ramp posts (NO.43, 47) and is not stated in this one.
- PayPal's 43%→95% comes from a single article relaying remarks at VentureBeat events (no confirmation from other outlets apart from republishing sites), with no measurement method, period or test set. The Google Cloud case page's "nearly 100%" is a vendor-side claim, and we could not confirm that the two articles describe the same assistant.
- The figures in Anthropic's post are all the company's internal tests with no independent confirmation. Factory's 20–25% is an outside company's statement quoted in the company's post.
- We saw MathArena's table ourselves, but the details of which effort level and settings each model ran at, and the "11-hour timeout" case, are only in The Decoder (relaying an evaluator's X post).
- We read only the first 100,000 characters of the eBay User Agreement (the remaining 27,000 unread), and the Muse access instability is PPC Land's relay of The Verge's report. A list of permitted agents exists nowhere.
What we read alongside
We opened and checked sources of different kinds directly. They are listed separately as company announcements and official documents, survey and evaluation organizations and data, media reports, wiki, and our earlier articles.
Company announcements and official documents:
- claude.comUsing Claude Opus 5.5 as your daily driver for knowledge work (Anthropic, November 12)in English; the seven old habits, the company's internal test figures and price table (conflict of interest)
- cloud.google.comPayPal — conversational analytics with Looker MCP Server (Google Cloud customer case, undated)in English; internal MCP hub, Claude Desktop, roughly 5,000 users, semantic layer ("nearly 100%" is a vendor claim)
- ebay.comeBay User Agreement (version effective June 28, 2026)in English; Section 3 wording "buy-for-me agents, LLM-driven bots, or any end-to-end flow that attempts to place orders without human review"
Survey and evaluation organizations and data:
- venturebeat.comVB Pulse — Agentic Orchestration, October 2026 (VentureBeat Intelligence)in English; 100 respondents, 47% swapped, of 45 weeks 33% and a month or more 36%, same or better 56%, rework 33%, revert 11%, sample caveats
- ramp.comBusinesses are changing AI model providers more often (Ramp Economics Lab, October 6)in English; 8% in September (three-month average), the two shifts of May 2025 and June 2026, "may be a commoditization signal"
- matharena.aiMathArena leaderboard (ETH Zurich SRI Lab · INSAIT, updated November 13)in English; definitions of expected performance and cost per problem, Carbon 85.4/15.70, Fable 5.2 82.8/40.20, Sol 80.6/1.07, Kimi K3.1 Pro 54.3/3.70
Media reports:
- venturebeat.comSwapping AI models is easier said than done: for most companies, it takes weeks, VB Intelligence finds (VentureBeat, November 12)in English; survey write-up, prompts/tools/tests/fine-tuning, harness example (single source), JPMorgan, Upwork, Box, T-Mobile
- venturebeat.comPayPal says its AI assistant went from 43% to 95% accuracy without changing the model (VentureBeat, November 10)in English; write-up of event remarks (single source), golden queries, playbooks, business definitions, reduced reliance on fine-tuning
- venturebeat.comAI's biggest bottleneck is cost: companies blame stalled projects on overruns three times as often as on compute shortages (VentureBeat, November 10)in English; VB Pulse infrastructure survey (156 respondents), Uber's cost per cleanup down 90% (Uber-specific)
- the-decoder.comAnthropic says outdated prompting habits are slowing down Claude Opus 5.5 (The Decoder, November 14)in English; summary of Anthropic's post
- the-decoder.comGemini 4 Carbon takes the MathArena crown but costs 15 times more than GPT-6.1 Sol (The Decoder, November 14)in English; leaderboard summary, "timeout" failure case (relaying an evaluator's X post)
- ppc.landMicrosoft's Exchange outage stalls offices as eBay shuts out robot shoppers (PPC Land, November 14)in English; eBay terms timeline (January 20 revision, effective February 20), Muse access instability (relaying The Verge), spokesperson statement
Wiki:
Our earlier articles:
- apholdings.krWeekly AI news — Google changed three models and Search in five days (AP News NO.47)
- apholdings.krThe "daily driver" slot appeared — Hydrogen and price per task (AP News NO.46)
- apholdings.krBlock the customer's agent, let the store in, and the model for everyone (AP News NO.45)
- apholdings.krCustomers can't bring their own agent (AP News NO.43)
- apholdings.krThe scoreboard got probabilities (AP News NO.40)
- apholdings.krWhich AI for which job — the test method we use (AP News NO.17)
- apholdings.krInstructions given to AI are a liability, not an asset (AP News NO.05)
- apholdings.krWhen models change weekly, five criteria a small team works by (AP News NO.04)
- apholdings.krWhen AI reads the ingredient label for you (AP News NO.03)