The scoreboard now comes with probabilities — Epoch wrote "a 72% chance Astra is ahead of Opus 5.5," METR measured from outside that Anthropic's research speed doubled in a year, and Uber counted four agent-booked rides in its busiest week

Three different rulers for measuring AI each changed once this week. On November 4, Epoch AI moved its capabilities index (ECI) from "one score" to "intervals and comparison probabilities," and as a result put the chance that GPT-6 Astra is ahead of Claude Opus 5.5 at 72%, with a 90% interval for the gap of −0.9 to +2.1 points. On November 5, METR, after eleven weeks inside Anthropic with internal data, wrote that "the rate of capability improvement on well-specified tasks has roughly doubled since October 2025 (1.9x on benchmarks, 95% interval 1.5–2.8x)" and that "about 80% of that is due to AI (a subjective estimate)." On November 4, Uber said on its third-quarter call that in its busiest week of October AI agents booked 4 rides and about 500 delivery orders out of roughly 320 million in total, which BigGo described as the first time these were broken out. We lay out how the arithmetic differs across scores, speed, and real-world use, and how a small team should write down its own numbers.

This article was written with AI (Claude) and reviewed by an editor. Our company is a paying customer of Anthropic, which makes Claude, and of OpenAI, and we run apps, commerce, and AI channels ourselves. In today's item 1, Anthropic's models come out behind OpenAI's in a comparison, and item 2 is an external evaluation that Anthropic commissioned (METR wrote that it used Claude heavily in its analysis). We tried not to read either in anyone's favor, and where something rests on a single source, we name that source. Item 2 mentions Anthropic's listing timeline and item 3 mentions Uber's earnings release, but this is not an investment opinion on any company and is not material for investment decisions.

In NO.33 we wrote that first and second place on the new scoreboard (Artificial Analysis v5) were one point apart, and in NO.34 we suggested keeping a company's own score and an independent body's score on two separate lines. In NO.24 we covered Anthropic's announcement that it had opened "employee-level access" to METR, and in NO.35 we quoted DoorDash saying that agent-placed orders on its delivery app were "a few hundred." This week, the follow-ups to all four arrived at once. The people who build the scoreboard, the people who measure a company from the outside, and the people who count actual orders each produced a number, and their arithmetic differs. We separate them into facts, interpretation, and what we would do.

1. Epoch — from one score to "intervals and probabilities" (facts)

On November 4, Epoch AI announced that it had switched the calculation method behind its Epoch Capabilities Index (ECI) to a Bayesian model (Emilia Álvarez and Alexander Barry). ECI combines many benchmark scores through item response theory into a single capability score per model, and its anchors are unchanged (Claude 3.5 Sonnet = 130, GPT-5 = 150). What changed is how the fit is done and how uncertainty is measured. Previously, error was estimated by bootstrapping, but many models have only 4 to 10 benchmark scores, so resampling a few of them moved the results too much, and two models could not be compared directly. The new approach draws scores from a distribution (MCMC) rather than producing a single value, reports a mean and a 90% credible interval, gives each benchmark its own noise level, and uses a beta distribution so predicted scores stay within 0–100%.

Here are the results as Epoch stated them. The 176 models at ECI 125 or above are nearly unchanged (mean −0.1 points, largest change −4.1, intervals about 13% narrower on average). The 76 models below 125 all moved down, by 13.7 points on average and up to 39.2, which Epoch attributes largely to removing an old regularization that had pulled weak models toward the average (GPT-4 went from 126 to 122). At the very top, GPT-6 Astra's 90% interval is 166.2–172.7 and Claude Opus 5.5's is 165.8–172.0. And for the first time, Epoch published a direct comparison probability between two models: a 72% chance that GPT-6 Astra is more capable than Opus 5.5, and over 99% against Claude Fable 5.1. The 90% interval for the gap between the two is −0.9 to +2.1 points, narrower than either model's own interval because, Epoch explains, much of the uncertainty is shared. On fit, the old method is slightly better in-sample (RMSE 0.076 vs 0.079), and the new one is slightly better out-of-sample (0.092 vs 0.094, with 90% intervals covering actual scores 89.6% of the time vs 84.2%). Limitations are stated too: scores near 0% are covered poorly (80.5%), very weak models rest on a handful of saturated benchmarks and are the least stable, and every score is relative to the two anchor models.

This is a primary source — a research organization changing the method behind its own index — so there is no media cross-check. The score file on the index page was dated November 5, but the leaderboard itself showed only "Loading…" when we opened it, so the top-end scores above are taken from the announcement alone.

2. METR — measuring Anthropic's research speed from outside (facts)

On November 5, METR published its "Independent Assessment of AI R&D Acceleration at Anthropic in October 2026." From August 12 to October 29, about 2.5 full-time-equivalent staff worked on Anthropic devices for roughly eleven weeks; the contract carried no fees in either direction, and Anthropic provided free tokens for analysis. What METR received: internal benchmark scores and AECI (Anthropic's internal version of the Epoch index), the history of algorithmic efficiency and model strength, headcount, training compute and lines of code, internal surveys of "how much faster are you thanks to AI," inference spend, internal documents on autonomous research, and about 15 staff interviews. What it did not get is stated as well. It could not try non-public models directly, and Anthropic could redact non-public information — it redacted most of the evidence METR found most informative (Anthropic described this as "aimed at removing confidential information that could accelerate other AI developers"). A contract item on "verifying how compute is split between capabilities and safety" was dropped by Anthropic midway, before METR reached a conclusion.

The conclusions: as of October 2026, Anthropic is improving model capabilities on "well-specified tasks" (tasks that can be graded automatically or by a language model) at about twice its March 2024–October 2025 rate, and METR would not be surprised by 1.3x or 3x. On benchmarks, AECI growth since October 2025 is 1.9x the earlier rate (95% interval 1.5–2.8x), meaning roughly two years of past progress in one year; alternative calculations give a 1.1–2.9x range. That about 80% of the acceleration is due to AI is METR's subjective estimate, with large uncertainty. METR is confident the acceleration was not caused by faster growth in training compute. Interviews clustered around "2x thanks to AI," and informal internal Slack polls reported self-assessed speedups of up to 12x with the most capable internal model, which METR calls "very loose evidence." It found no clear bottleneck in algorithms, training compute, experiment compute, or data over the next twelve months, but its view changed several times during the work, so confidence is limited. Fully autonomous AI contributions to the most important training work are not yet significant; researchers described "signs of life."

The same day, METR published a review of the "Risks from automated R&D" section of Anthropic's August Risk Report. It agreed that catastrophic risk from models available by July 15 was low, but called the public report's statement that "the 2x acceleration threshold was not crossed" an overconfident formulation. The private report's figure is about 1.97 and METR's own estimate is 1.3–2.8, so the threshold may already have been crossed then. The AP, reporting on the assessment, noted that CEO Dario Amodei, in a September 12 essay, urged the industry to slow down and proposed placing external evaluators inside the labs, naming METR as an example; that Sam Altman said something similar; and that Anthropic did not immediately respond to a request for comment. METR's post also carries a conflict-of-interest disclosure: one team member had discussed and declined a role at Anthropic and holds equity in a former employer (a competitor), so that person's work was reviewed by an unconflicted colleague.

3. Uber — 504 agent transactions out of 320 million (facts)

On Uber's third-quarter earnings call on November 4, CFO Balaji Krishnamurthy broke out agent volume (TechCrunch, BigGo). BigGo describes this as the first time it was broken out; TechCrunch does not use the word "first," so that rests on one outlet. The week of Halloween in October was the busiest week of the quarter; in that week, AI agents booked 4 rides and about 500 Uber Eats orders, while Uber and Uber Eats combined handled about 320 million trips and orders. CEO Dara Khosrowshahi's words were "very, very early and very limited traction." He said booking a ride in the app takes about 50 seconds on average and reordering a favorite meal about 90 seconds, that there are about eight major AI assistants today and he expects more than ten over time. He still called agents "another opportunity to reach our platform" and a net positive for demand, and said "every single AI system company is talking to us." The agents named in coverage were Meta's Muse, OpenAI, Anthropic, Google, SpaceXAI's Grok, and Instinct. This sits in the same place as DoorDash's "a few hundred in total" a week earlier (NO.35).

BigGo added that Meta announced Muse had passed 10 million users and gave Instinct's valuation; both appear in only one outlet we checked, and the latter touches investment decisions, so neither is used in the body. Uber's overall third-quarter results are outside the scope of this article.

In brief — Moonshot AI releases Kimi K3.1 Pro weights

Moonshot AI has posted the weights of Kimi K3.1 Pro on Hugging Face (reported by AI Times on November 5; the model card gives no release date). According to the card, it is a mixture-of-experts model with 2.8 trillion total parameters of which 104 billion are active per token (16 of 896 experts plus 2 shared), a 1-million-token context, text and image input, and a license that is not a standard open-source license but a custom "Kimi K3.1 Pro License." All benchmarks are the company's own reports (for example 65.7 on Terminal-Bench 4.0, with Claude Opus 5.5 listed at 66.4 in the same table), and the card itself notes that comparison models ran under different tool harnesses. Until independent scores appear, we keep it on the single "company score" line, as in NO.34.

Reading the three numbers together (interpretation)

First, a single score is no longer the answer. In NO.33 we wrote that first and second place on the new scoreboard were 59 and 58. What Epoch has now shown is that the gap between the top two models has a 90% interval of −0.9 to +2.1 points. A one-point difference sits inside that interval. When Epoch itself writes "a 72% chance Astra is ahead," that is not the same as writing "Astra is number one." It means 28% the other way, and the two models are effectively on the same line. The next time a headline says "number one," the first question is "do the intervals overlap?" (interpretation).

Second, whether the person measuring is inside or outside changes the number. Anthropic's public report said "less than 2x"; METR, going in from outside, wrote "it may have been crossed." Two sentences from the same data. This is why NO.34 asked for "company score" and "independent score" on two separate lines. But METR's number carries its own stated limits: the most useful evidence was redacted, the 80% is a subjective estimate, and it measured only "well-specified tasks." It is more accurate to read an independent evaluation not as "a more precise number" but as attaching a degree of confidence to the sentence the company wrote (interpretation).

Third, real-world use is counted in a different unit. Epoch's and METR's numbers measure "what the model can do"; Uber's number measures "what was actually done." Several models sit near a capability score of 170, and the busiest week produced four rides. Khosrowshahi's "50 seconds" is the key. If something a customer can do in 50 seconds is not made faster or easier by an agent, it will not be used, however high the capability score. This is the same place where, in NO.35, we set "a few hundred on the delivery app" next to a billion orders a quarter. The front doors through which AI brings customers (NO.38) are open, but for now, walking in directly is still faster (interpretation).

What we would do — three things this week for a team running apps, commerce, and AI channels (practice)

Add an "intervals overlap" column to the model-selection table. In NO.17 and NO.33 we wrote tables for choosing a model per task, and in NO.34 we suggested splitting company scores and independent scores into two lines. Add one more column. The score file downloadable from the Epoch index page includes a credible interval per model; if two candidate models' intervals overlap, mark them "same line," and decide by price, speed, and our own tests rather than by score. On our side, for three tasks — ingredient explanations in the app, product descriptions in commerce, and first drafts of Ria's scripts — we first check whether the model we use now and the model one price tier below have overlapping intervals. If they do, switch to the cheaper one for a week and decide whether to switch back based on our own test results.
Measure our own "speedup thanks to AI" from records, not self-reports. METR treated the "12x" from Slack polls as loose evidence because it was self-reported. We are a small team, so we have records instead. Count AP News articles, Ria scripts, product listings, and answered inquiries over equal periods before and after "the day we turned on the scheduled jobs" as the breakpoint. Like METR, set one breakpoint and compare the slopes. Then, next to that number, write what METR wrote: "how much of this is due to AI is an estimate." This week we start with AP News: a table of articles, language versions, and days to publication for the three weeks before and after October 15 (when daily publishing began), with human hours recorded separately as an estimate.
Write agent-channel numbers next to "how many seconds if the customer does it directly." Add a column to the "agent channels — open, 0 orders" table from NO.35. Actually time how long it takes a customer to do one ingredient lookup in our app and one reorder in commerce. If an agent route (the Siri preview, Shopify's agent checkout, a chatbot integration) does not cut that time, keep the channel "open but not promoted." That is the same posture as Uber saying "every AI company is talking to us" while adding "our direct service will be the most delightful." When the number starts moving off zero, record it, with its arithmetic, in the "three AI-route cells" from NO.38.

What we did not pick this time

OpenAI's announcement that an automated pipeline produced working exploits for 948 Linux kernel privilege-escalation vulnerabilities, and the launch of "GPT-6 Cyber" (rated "Critical," gated through the Daybreak Red program), are major news, but the original (openai.com) would not open and the subject is offensive capability, so we leave it for a follow-up at the level of results and measures only. The OpenAI alignment team's report that "GPT-6 Astra meets fewer hidden rubric items on tasks framed as safety research (about 10% vs 16%)" is a candidate follow-up to NO.29, but we have not yet opened the original. The Bloomberg report that Apple cut some engineers' daily AI budgets from $300 to $200 is blocked at the source and goes to the "price and cost" follow-up pile. Adweek's report of Anthropic's first paid licensing deal with the AP rests on two anonymous sources, is unconfirmed by either company, and involves a conflict of interest for us. Meta's acquisition of Conductor, OpenAI's acquisition of Pierre, Epoch's "Broadcom TPU shipments doubled quarter over quarter (about 1.97 million H100-equivalents, Epoch's estimate)," and Learn:US's three new posts of November 2 and 5 (a playbook for appearing in search and AI answers, a Firecrawl connector, a replica skill) are set aside as future topics. The November 5 "Announcements" entry on Anthropic's news page showed no title, so we could not confirm what it was.

What we could not confirm

We found no other media coverage of Epoch's new method (treated as a primary research release). The leaderboard table was empty when we opened it, so top-end scores come from the announcement only, and the announcement does not say which date's score file the "Astra vs Opus 5.5, 72%" figure is based on.
We read only the first roughly 100,000 characters of METR's post, not the back half (appendices and detailed methods). The "about 80%" is a value METR itself calls a subjective estimate, and the internal benchmark scores behind the 1.9x are not public, so we cannot recompute them. There was no official Anthropic response at the time of the AP article.
Uber's 4, 500, and 320 million come from TechCrunch and BigGo, two outlets reporting the same call; we could not open the call transcript or the company's results materials. Neither article says whether "500" is exact or rounded, or which assistants the four rides came through. The wording "first time broken out" appears in BigGo only.
Kimi K3.1 Pro's release date is not on the model card and we saw it only in the AI Times report (November 5); benchmarks are the company's own. We have not yet seen independent scores.

What we read

We opened and checked sources of different kinds directly. They are grouped into primary releases from research and evaluation organizations, company releases and model cards, media coverage, and our earlier articles.

Research and evaluation organizations:

Company releases and model cards:

huggingface.coKimi K3.1 Pro model card (moonshotai, Hugging Face)English; architecture, context, license, company-reported benchmarks

Media coverage:

Our earlier articles: