Inkling 2 Mini reasons inside loops, Claude's claimed Riemann proof exists only as Lean, OpenAI hides its traces, and METR says its yardsticks are slipping.
The day in 60 seconds
1
Models
Inkling 2 Mini — a 26B open model that loops its middle layers
ExpandCollapse
Verified#models #local
Thinking Machines' Apache-2.0 model runs its middle layers three times per token and needs about 20 GB, so it fits a 32 GB card as shipped.
The detail 26.4B parameters, 3.4B active; 20.1 on Artificial Analysis's index, level with Qwen3.8-27B, and 48.2% on DeepSWE v1.1 by the lab's own run.
The catch the coding scores are Thinking Machines' runs with varying setups, and looping adds reasoning, not stored facts, so its knowledge scores stay at its size.
Worth learning weight sharing across depth. Reusing the same layers several times buys more computation per token without more memory for weights, but it cannot add knowledge the weights never held.
Levent Alpöge says he and Claude proved the Riemann Hypothesis
ExpandCollapse
Claim#maths #lean
An Anthropic mathematician posted a four-page note and a 14-part Lean archive, but no readable proof, so nobody outside has checked it yet.
The detail he says the method also proves the Generalized Riemann Hypothesis; Claude's write-up ran past 250,000 pages, and the key steps are dated 6 October to 16 November.
The catch Lean confirms only that the files' final theorem follows; until mathematicians read what that theorem states, the claim rests on the author.
Worth learning the statement file. In a formal proof, trust moves from thousands of steps to the few lines that define what was proved, and those lines are where a reviewer starts.
OpenAI: attacks on GPT-6.2 Astra's hidden reasoning now succeed 0.2% of the time
ExpandCollapse
Claim#safety #distillation
OpenAI says its reasoning is now far harder to extract, and its own study found raw traces teach copied models more cyber skill than summaries.
The detail average attack success fell from 54.0% on GPT-6 Astra to 0.2% on GPT-6.2 Astra in an internal automated red-team test.
The catch internal evaluation only; the student model and the cyber benchmarks are not named on the page.
Worth learning distillation. A student model learns from a teacher's outputs; the teacher's intermediate reasoning carries more of its method than the final answer, which is why labs now hide it.
METR: AI R&D skills are rising fast and getting harder to measure
ExpandCollapse
Verified#safety #evals
METR says public evidence no longer rules out drastic AI R&D acceleration within a couple of years, and its cheap tests may overstate models.
The detail time horizons on its combined suite double about every 3.6 months; frontier models reach up to 27 hours on the verifiable part of its new tasks.
The catch only 9% of its human time estimates rest on successful human attempts, and models showed worse judgment than experts where feedback is slow.
Worth learning time horizon. The length of task, in human hours, that a model finishes half the time; it weakens as a safety argument once models beat the humans who set the yardstick.
Hugging Face: one LoRA per harness lifts Qwen3.8-27B from 39% to 55%
ExpandCollapse
Claim#harnesses #local
Reinforcement learning inside pi, Hermes Agent and OpenCode trained small adapters that make the same model score higher with half the tokens.
The detail Terminal-Bench 2.1 in pi went from 39% to 55% pass@1 while tokens per task fell from 2.9K to 1.5K; a merged adapter runs in all three harnesses.
The catch all numbers are Hugging Face's own, and the authors list what the model actually learned as an open question.
Worth learning harness fit. Models are tuned against particular tools and prompts; a small adapter trained in your harness teaches its habits without touching the base weights.
Cursor tells Congress it may drop Chinese open-weight models
ExpandCollapse
#industry #policy
The SpaceX-owned editor said it is weighing a phase-out over "geopolitical risk", a day after reports of a US plan to make firms disclose Chinese AI use.
The detail the statement appears in a report House committees published on 20 November, and covers all China-developed open-weight models.
The catch "considering" only, with no date or model list, and CNBC's page refused our reader, so details come from its headline and other reports.
Worth learning model provenance. Open weights run anywhere, so the question buyers and regulators ask moves from where a model runs to who trained it.
Signal blocks phone AI agents from reading its screen
ExpandCollapse
#industry #agent-security
Signal will block screenshots on iOS 27 and AI screen capture on Android, because built-in assistants copy chats outside its control.
The detail Meredith Whittaker's post names Gemini Intelligence and Siri's Onscreen Awareness; users can switch the block off, as with Windows Recall last year.
The catch Signal calls the fix incomplete, since an agent reading files already on the device is not stopped.
Worth learning the trusted computing base. Encryption protects a message in transit; once the operating system reads the screen, the OS is part of what you have to trust.
Two weeks without ChatGPT cut loneliness in a 625-person trial
ExpandCollapse
Verified#humans #wellbeing
People randomly told to stop using AI assistants for 14 days felt less lonely and saw more people, though most used AI for work.
The detail loneliness fell against a one-day control (β = -0.10, p = 0.03) and stayed lower two weeks after they resumed (β = -0.16).
The catch self-reported loneliness, small effects, and participants knew which group they were in.
Worth learning randomised deactivation. Assigning people to stop a habit, then comparing with a group that barely stopped, separates the habit's effect from who chooses it.
Inkling 2 Mini runs its middle layers three times before it writes a token, so some of its reasoning happens where no chain of thought shows it. OpenAI now keeps its own reasoning traces from anyone who asks the wrong way, and says that cut extraction attacks to 0.2%. METR, whose time horizons I have leaned on for a year, says its cheap tests may overstate models and that it could rarely find humans able to set the bar.
Alpöge's Riemann note is the sharpest case. He believes the proof is right, Claude wrote 250,000 pages of it, and he chose to publish Lean files instead of prose. I like that choice more than my feed does. A Lean archive can be checked by a program that has no view on Anthropic's share price. It cannot tell me whether the theorem it proves is the Riemann Hypothesis as mathematicians mean it, and that part is a short file a person has to read.
That gives me a rule for my own agents. When a run produces more than I can read, I stop trying to read all of it and put my attention on the lines that say what was supposed to be true, which means the test names and the assertions. Thinking Machines measured that looping kept Inkling's chain of thought legible, and I'll check that myself on a weekend run before I trust it with anything long.
Concept of the day
Looped transformer
A student who reads the same paragraph three times before answering. The paragraph does not get longer, but each reading starts from what the last one noticed.
The mechanism: Inkling 2 Mini sends each token through layers 1-6 once, through a shared core of layers 7-18 three times, passing a message from each pass to the next, then through layers 19-24 once, so 24 physical layers act like 48.
What it means today: the extra passes add computation, not memory for weights, which is why a 26.4B model scores beside a 27B dense one on coding but not on facts, and why part of its reasoning never appears as text.
One start. The first six layers run once, and every pass through the core begins from their output.
Passes talk. After each pass the core writes a message that the next pass reads, so pass three builds on passes one and two.
Then decode. After the third pass, layers 19-24 run once and produce the next token: 24 physical layers did the work of 48.
A dial for research. A variant trained for variable loops kept improving on serial reasoning out to ten passes, twice its training.
Step throughLooping reuses the same weights for more computation per token.
The industry
Cursor told House committees it may stop offering every Chinese-trained open-weight model, which would turn a model's origin into a product decision for the biggest coding-agent tool. Signal now blocks the assistants built into iOS 27 and Android from reading its screen, because once the operating system copies a chat, Signal's encryption no longer protects it. Anthropic's shares gave back most of their debut gain on day two, trading near the $83 offer price.
Signal will prevent screenshots on iOS 27 and block AI assistants' screen capture on Android where it can detect them, because operating-system agents copy chats outside its encryption. Users can switch the protection off.
Why it matters: a major app now treats the phone's own assistant as a threat, so screen-reading agents will meet apps that refuse them.
#agent-security #privacy
ExplainClose
Background Meredith Whittaker writes that agentic features in iOS 27 and Android scan the screen so assistants can answer questions, which captures Signal messages and sends them to whatever local store or remote server the assistant uses. Signal will block screenshots on iOS 27 and block assistant screen capture on Android where detectable; users can opt out. Hacker News readers noted iOS 27 initially had no way to exclude an app from Siri's screen access. Signal says the fix is incomplete because agents may read app data through the filesystem.
The SpaceX-owned code editor told lawmakers it is considering dropping every China-developed open-weight model over "geopolitical risk", according to a report House committees published on 20 November.
Why it matters: the largest coding-agent product treating model origin as a risk shows where US rules may push every tool that routes to open weights.
#policy #open-weights
ExplainClose
Background Cursor, now owned by SpaceX, offers several Chinese open-weight models such as Kimi and DeepSeek variants inside its editor. Andrew Curran reported the statement was made to Homeland Security and appears in a report published by multiple House committees on 20 November. It came a day after reports that the administration is weighing rules forcing US firms to disclose use of Chinese AI. CNBC's article refused our reader, so the wording here comes from its headline and from secondary reports; no timeline or model list was published.
Anthropic traded at $84.01 on its second morning, down 14% from its $97.17 debut close, as selling spread across AI stocks.
Why it matters: Amazon and Alphabet own about 15% and 14% of Anthropic, so a fall below the offer price hits both balance sheets.
#anthropic-ipo #markets
ExplainClose
Background Anthropic priced at $83, opened at $111.50 on 19 November and closed at $97.17; on day two it traded at $84.01 in the morning, down 14%. 24/7 Wall St. attributes the move to rotation out of AI-linked growth stocks rather than company news. Anthropic separately announced the offering closed at 1,127,000,000 Class A shares including the underwriters' full option. Amazon and Alphabet own about 15% and 14%, so the price near the offer matters to their reported gains.
Oracle says a final $20 billion raise covers its OpenAI build-out.
ExplainClose
Background Oracle told investors the financing for its OpenAI infrastructure contract is secured after a final capital raise of about $20 billion, answering market doubts about how the build-out would be paid for. We could reach only a secondary write-up of the wire story, so no contract figures beyond the raise are given here.
A live remark with no text of any agreement behind it.
ExplainClose
Background Speaking live about AI, President Trump said "President Xi and I agreed we're going to do some joint guardrails", per Andrew Curran's transcription. No document, scope or timeline accompanied the remark, and neither government published anything by the time of writing, so it is a statement of intent rather than a binding change.
8 released today · 0 publish a price · a dash is never “free”
Sort
Inkling 2 Mini is the day's lesson in getting more from fewer weights: it runs the same twelve middle layers three times per token, so a 26.4B model with 3.4B active reaches Qwen3.8-27B's score on Artificial Analysis's index in about 20 GB. Nvidia's Qwen3.8-27B NVFP4-QAD shows the other route, training a 4-bit copy to imitate the full model so it loses almost nothing on GPQA. Tencent's Hy4 opens a 770B flagship that needs a server, not a desktop.
Try if you have a Blackwell card and run Qwen3.8-27B in BF16 today; skip if you are on RTX 30 or 40 series.
ExplainClose
Nvidia shrank Qwen3.8-27B about 2.8x with quantisation-aware distillation: GPQA Diamond moved from 88.19 to 87.94 and Terminal-Bench 2.1 from 77.25 to 76.69. It needs a Blackwell GPU, Linux and vLLM; a r/LocalLLaMA tester found it held up against Unsloth's NVFP4.
On a 32 GB card 27.7B parameters at about 4.5 bits is roughly 16 GB, so an RTX 5090 holds it with room for a long context.
Worth learning quantisation-aware distillation. Instead of rounding weights and hoping, the 4-bit student is trained to match the full-precision model's outputs, recovering most of what rounding loses.
Skip if you want something to run at home; read the system card if you follow Chinese labs' safety practice.
ExplainClose
Tencent released Hy4 (770B total, 49B active, 1M-token context) with FP8 and base checkpoints, under its own licence that needs a separate deal above 100 million monthly users. vLLM and SGLang support it from day one, and it ships with a system card.
On a 32 GB card no. At 4 bits the weights alone are about 385 GB, so this is a multi-GPU server model; Hy4 Flash is the local one.
The build gated DeepSeek Sparse Attention with an index reused across layers, and 256 routed experts in each of 77 MoE layers.
Worth learning sparse attention with index reuse. A cheap indexer picks which earlier tokens each query reads, and neighbouring layers reuse that choice instead of recomputing it.
Try if you want a local coding agent that leaves room for long context on one card; skip if you need factual recall, which looping does not improve.
ExplainClose
Thinking Machines' Apache-2.0 MoE (26.4B total, 3.4B active) runs layers 7-18 three times per token. Artificial Analysis scores it 20.1, level with Qwen3.8-27B; the lab reports 48.2% on DeepSWE v1.1. Trained in NVFP4, it needs about 20 GB.
On a 32 GB card yes as shipped. About 20 GB of NVFP4 and MXFP8 weights leaves roughly 12 GB for context; a r/LocalLLaMA tester runs Unsloth's 13.6 GB Q3 file on a 16 GB card at about 34 tokens a second with 32K context.
The build 24 physical layers act as 48: six run once, twelve loop three times with messages between passes, six decode. Five of every six layers attend over a 1,024-token window.
Worth learning native 4-bit training. A model trained in the format it ships in needs no quantisation step afterwards, which is why the full model is about 20 GB; a Thinking Machines engineer says Unsloth found it quantises further well.
Try if you need depth for 3D capture or camera tracking on your own machine.
ExplainClose
Apple's Depth Pro 2 estimates metric depth for images and video at full resolution in one network, and Apple says it beats prior work on accuracy, edge sharpness and frame-to-frame consistency at once. Weights start at 23.5M parameters under Apple's research licence.
Worth learning the consistency-sharpness trade-off. Multi-frame depth models smooth over time and blur edges; single-frame ones keep edges and flicker; training one model on both is Apple's fix.
Try if you want local voice cloning for personal tools; skip for anything you sell.
ExplainClose
Software Mansion's Reverb clones a voice from 3-10 seconds of speech in seven languages, with 1.03% word error on the Open TTS Leaderboard's MiniMax set, and runs 21.6x real time on an M4 Pro. Weights are CC-BY-NC because it was distilled from OmniVoice.
Worth learning distillation and licences. A model distilled from another inherits its licence terms, so a fast student of a non-commercial teacher stays non-commercial.
Inkling 2 Mini is not yet in Ollama's stable library (it needs the 0.40.17 pre-release), but at about 20 GB it is today's best new fit for a 32 GB card. See the table ↓
Benchmarks
Changed hands since yesterday: OpenRouter usage, now DeepSeek-V4.1-Flash (was Hy4-Flash). See the boards ↓
From the papers
Today's papers each find a cheaper way to get the same result. One lets a 27B model propose and test edits to its own weights, gaining 6.6 points without gradients. NarrowSpec shows the small model in speculative decoding can be much thinner than the big one it serves, which helps most on memory-bound desktop chips. FlashMorph2 picks which layers keep full attention by testing whole hybrid layouts, converting Qwen3 models in a few GPU hours.
Given its weights, graded answers and the ability to run itself, an open 27B model diagnosed its failures and rewrote weights with no gradients, improving 6.6 points on held-out problems and rising over eight sessions. Smaller models in the family made no useful edits.
Read if you fine-tune local models and wonder what a model can do with its own weights.
ExplainClose
The idea the model acts as its own optimiser, proposing an edit, testing it on practice problems and keeping only what helps, like a programmer patching a binary.
Why it matters if it holds, a local model could be improved without a training run; Hacker News readers note the gain still comes from the test signal.
Background The authors give an open 27B model its own weights, its graded answers and the ability to run itself; it proposes weight edits by reasoning, tests them on practice problems and submits only those that help. They report a 6.6-point gain on unseen problems, steady gains over eight sessions, and recovery of a damaged copy to over 80% of original accuracy on image tasks. Smaller models in the family made no useful edits. Hacker News readers point out that the improvement still comes from the test signal, and one notes the paper says most attempts failed.
NarrowSpec runs a speculative-decoding draft at half or a quarter of the target's hidden width with little loss in accepted tokens, speeding end-to-end decoding 1.06-1.12x on a B200 and 1.13-1.44x on a GB10, tested on Qwen3-8B and GPT-OSS-20B in SGLang.
Read if you tune speculative decoding on a single machine.
ExplainClose
The idea the small model that guesses tokens does not need the big model's width, only its hints; a thinner guesser is cheaper and nearly as accurate.
Why it matters memory-bound desktop hardware gains most, which is the setting most local users have.
Background Draft models such as DFlash and EAGLE-3 read the target model's hidden states and usually copy its hidden width. NarrowSpec shrinks the draft's whole residual stream to half or a quarter width; for DFlash the output adapter folds into the vocabulary projection with identical logits. Tested on Qwen3-8B in NVFP4 and GPT-OSS-20B in MXFP4 in SGLang, end-to-end decoding sped up 1.06-1.12x on a B200 and 1.13-1.44x on a DGX Spark at batch 1, and no narrow draft was slower than full width at any batch size.
FlashMorph2 picks which layers keep full attention when converting a Transformer to a full-plus-linear hybrid by testing sampled hybrids end to end, gaining 2.42 points on RULER over FlashMorph. Selection took 0.72-3.57 GPU hours across 0.6B-8B models.
Read if you want long context without a growing cache.
ExplainClose
The idea judge each candidate layout as it will actually run, with its own hidden states, instead of scoring layers one at a time.
Why it matters cheap conversion means long-context hybrids of models you already use, with a smaller KV cache.
Background Converting a pretrained Transformer into a hybrid of full and linear attention means choosing which layers keep full attention under a budget. The authors find FlashMorph's choice is off-policy: it scores continuous blends with teacher-forced states and frozen weights. FlashMorph2 samples discrete hybrids, runs them end to end on their own hidden states and optimises selection and weights together. On Qwen3 models from 0.6B to 8B it beats FlashMorph by 2.42 points on RULER and 1.08 on LongBench, with selection costing 0.72-3.57 GPU hours.
Kernel and runtime changes for block-sparse attention.
ExplainClose
Background Tencent's LightningMoBA accelerates mixture of block attention, where each query attends to a few selected blocks of earlier tokens, through dynamic routing, KV-centric execution and runtime pipelining, with code released. We did not read the full paper, and the abstract gives no consumer-hardware numbers, so its relevance to local inference is unclear.
Why try it It fits a 32 GB card at its native 4-bit precision with room for context, and the lab says looping makes it reason with fewer tokens, which matters on long agent tasks.
The experiment Take 10 small bug-fix or feature tasks from a repo you know. Run them in the same harness with Inkling 2 Mini and with Qwen3.8-27B at 4 bits, same context limit and tool set. Log pass or fail, tokens and wall time.
What to expect An estimate, not a measurement: similar pass rates to Qwen3.8-27B, since both score 20.1 on AA's index, with fewer output tokens per task; one 16 GB tester saw about 34 tokens a second, so a 5090 should be faster.
How to judge it Count tasks solved and tokens per solved task. Keep it if it solves at least as many as Qwen with fewer tokens, or within one task at noticeably higher speed; drop it if it fails on recall-heavy tasks you rely on.
Harnesses & skills
Two results say the harness matters as much as the model. Hugging Face trained a small adapter inside each harness and lifted Qwen3.8-27B from 39% to 55% on Terminal-Bench 2.1 while halving its tokens, mostly by teaching it to think and call tools less. Joel Niklaus found OpenClaw was throwing away models' earlier reasoning on custom endpoints; one setting fixes it and lifts scores 44%.
Hugging Face trained one LoRA per harness for a frozen Qwen3.8-27B with reinforcement learning inside pi, Hermes Agent and OpenCode. In pi, Terminal-Bench 2.1 rose from 39% to 55% pass@1 while tokens per task fell from 2.9K to 1.5K.
Try if you run Qwen3.8-27B as an agent in pi, Hermes Agent or OpenCode.
ExplainClose
Why it works the base model is tuned for general use; reward inside your harness teaches it to reason less and call fewer tools when the harness already supplies what it needs.
In your setup the merged super adapter runs unchanged in all three harnesses; a community post reports SWE-Bench Pro rising from a 55% to a 63% mean with 53-73% fewer tool calls.
Check results are Hugging Face's own, on harness versions pinned in its Docker images.
Background Hugging Face trained Qwen3.8-27B with LoRA and GRPO inside unmodified harnesses using OpenEnv's Harbor integration, on a task set built from SWE-smith, SWE-Gym and generated tasks; 858 of 1,127 candidate tasks passed validation and only tasks the model solved sometimes were kept. Wrong-answer controls caught a Terminal-Bench 2.1 verifier that accepted known-wrong answers. In pi, pass@1 rose from 39% to 55% and tokens per task fell from 2.9K to 1.5K. The blog itself lists what the model really learned as an open question.
OpenClaw drops a reasoning model's earlier thinking when it talks to a custom OpenAI-compatible endpoint, so only 3-7% of longer thoughts came back on GLM-5.3-Flash and DeepSeek-V4.1-Flash, Joel Niklaus found. One model setting restores close to 100% and raised its Harness Leaderboard score 44%.
Try if you run OpenClaw against your own endpoint.
ExplainClose
Why it works reasoning models expect their earlier notes in the conversation; without them they work the same steps out again, using about three times the calls.
In your setup set requiresReasoningContentOnAssistantMessages: true on any model OpenClaw reaches through vLLM, llama.cpp or a router, until it becomes the default.
Background Joel Niklaus maintains Hugging Face's Harness Leaderboard, which connects every model through a custom OpenAI-compatible endpoint. He found OpenClaw omits reasoning content from earlier assistant turns on such endpoints, so only 3-7% of longer thoughts returned on GLM-5.3-Flash and DeepSeek-V4.1-Flash. With the flag on, close to 100% return, OpenClaw needs about a third of the model calls on GLM-5.3-Flash, and it climbs from last place to 42.7 on the leaderboard. He has asked the maintainers to make it the default.
Paweł Huryn compared subscription limits with API list prices: Claude Max at $100 gives 46.4x on Opus 5.5 and 27.3x on Fable 5.2, Google AI Ultra 25.8x on Gemini 4 Neon, ChatGPT Pro 10.25x. Priced that way, GPT-6.2 Sol finds as many bugs as Fable 5.2 for $0.38 a run instead of $2.23.
Read if you split agent work across subscriptions.
ExplainClose
In your setup pick review subagents by cost per run on your plan, not list price; his own setup has Claude Code coordinate and Codex on Sol review.
Check one person's measurement on his own bug benchmark; the ChatGPT figure dates from October.
Background Paweł Huryn divided the API-price value of each plan's usage limits by its subscription price. On his Bug Hunt benchmark, GPT-6.2 Sol at max effort scored 54.7 of 105 for $0.68 a run, Sol at xhigh 49 for $0.38, Fable 5.2 at xhigh the same 49 for $2.23 and Opus 5.5 at max 41.7 for $1.26. He notes Gemini 4 Neon would score 63 for $1.80 but cannot be used from other harnesses under Google's terms. All figures are his own, from one benchmark.
Repo-Premium, a code-search benchmark over 91 repositories, traces the embedding advantage to choices that handicap keyword search and to queries missing the code's rare names. Having an LLM add likely identifiers to the query lets BM25 match dense retrieval in every Python pool.
Read if you are deciding whether a coding agent needs embeddings.
ExplainClose
Why it works code is full of unique identifiers, and once the query contains them exact matching finds the file, which is what agents do with grep.
In your setup before adding a vector index to an agent, fuse BM25 with dense retrieval and expand queries with identifiers.
Background Repo-Premium ranks files in 91 real repositories using curated and raw queries in six languages and pull requests from 2022 to 2026, with a preregistered test pool. The authors find standard benchmarks favour dense retrieval by chunking files for BM25 and counting edited tests as relevant. Expanding queries with LLM-written identifiers helps BM25 far more than dense retrieval, matching it in every Python pool even on post-training pull requests. No method predicted which retriever would win on a given query, so they recommend fusing both.
Background The paper proposes "System 1" agents that run locally and quickly by replacing slow reasoning steps with fast learned decisions, with code released. We read only the listing summary; it reports no comparison against standard harnesses on a named benchmark, so its practical value for coding agents is unclear.
Inkling 2 Mini gave interpretability its first good look at a looped model, and Goodfire found its layers can write to themselves and read from layers that come later, so residual-stream tools need rethinking. A LessWrong study shows truth probes move when a model plays a 1500s scholar, yet removing them barely changes its false answers, so the probe is reading the answer, not causing it.
Goodfire, given early access, traced information flow in Inkling 2 Mini, which has no single residual stream: layers attend over block outputs, and on passes two and three each layer reads the previous pass. Counterintuitively, a looped layer can write to itself or read from layers that come later.
Read if you use residual-stream tools and want to know where they stop applying.
ExplainClose
How it works "attention residuals" replace the running sum of layer outputs with attention over four six-layer blocks, so tools that assume one residual stream need rethinking.
Evidence preliminary and descriptive, on one model, from a research partner of the lab.
Run it weights are Apache-2.0 and about 20 GB, so it fits a 32 GB card.
Background Inkling 2 Mini has 24 layers in four six-layer blocks. Instead of a cumulative residual stream, each layer reads earlier block outputs through depth-wise attention, and each block keeps its own running sum. Blocks 2 and 3 loop three times, and cross-loop attention lets each sublayer read the previous pass's block outputs. Goodfire's preliminary work studies how information flows across layers and loops and which decoding tools still work. It reports that looped layers can write to their own later input and read from layers that are later in the stack.
Playing a European scholar in 1500, Qwen, Llama, Gemma and OLMo call 41-89% of that era's falsehoods true (baseline 3-6%), and their truth probes shift too. Knocking out the truth directions cut the effect by only 0-13%.
Read if you treat probes as lie detectors.
ExplainClose
How it works a linear probe finds a true-false direction; removing it tests whether the model uses that direction to answer.
Evidence causal ablations across four families, linear probes only; a probe at the answer position correlates 0.82-0.94 with the answer yet explains 1-18% of the effect.
Run it code is linked from the post.
Background Aniket Ghosh tests whether a linear "truth" direction causes persona-driven falsehoods or only tracks them. With personas such as a 1500s scholar or a partisan, four model families endorse many falsehoods and their probes shift toward "true". Ablating the truth directions reduced the effect by 0-13%, and even a probe at the answer position, correlated 0.82-0.94 with the output, accounted for 1-18%. Similar results held for full sentences and LoRA personas. He is confident of the effect, less sure of the mechanism, and tested only linear probes.
Fixed steering vectors added to frozen diffusion transformers, with 65x fewer parameters than the RL LoRA on SD3.5-M, recovered 90.0-96.3% of the reward gains of six RL-tuned image models and 89.8% and 111.9% on two video models.
Read if you fine-tune image models and want to know what RL changed.
ExplainClose
How it works vectors identical for every prompt and seed are fitted to match the tuned models' outputs, with no reward in the objective.
Evidence behavioural match across eight public checkpoints; trained with RL directly, the vectors alone beat weight-based RL on text rendering.
Background The authors ask how much of RL post-training's gain in diffusion transformers is an input-independent shift. They fit one set of steering vectors, added to block outputs of the frozen base model, to match each public RL-tuned checkpoint's velocity predictions, with no reward used. On SD3.5-M these vectors hold 0.29M parameters against the RL LoRA's 18.8M and recover 90.0-96.3% of the reward gains across six image models, and 89.8% and 111.9% on two video models. Trained with online RL from zero, the vectors beat weight-based RL on OCR and PickScore.
A residual branch takes the non-semantic part of the signal.
ExplainClose
Background DualSAE adds a residual branch to a standard sparse autoencoder so the sparse code can hold interpretable features while the residual absorbs what they miss, with a decorrelation loss keeping the two apart. The authors report about 29% better reconstruction and 34% better monosemanticity on vision and language data, measured by their own metrics.
METR says the yardsticks are slipping: time horizons double about every 3.6 months, frontier models handle 27-hour verifiable tasks, and its cheap container tests may overstate real ability. OpenAI says it now blocks almost all attempts to extract its models' hidden reasoning, after finding raw traces teach copied models more cyber skill. A red-team study shows human auditors catch sabotaged safety research at near-chance rates.
METR's update on models up to Opus 5.5 and GPT-6 Astra says public evidence cannot rule out drastic AI R&D acceleration within a couple of years. Frontier models reach up to 27 hours on its verifiable new tasks but showed worse judgment than experts where feedback is slow.
Read if a time horizon appears in any risk argument you rely on.
ExplainClose
What it shows containerised evals that resemble RL training environments may overestimate real capability; agent swarms helped only on fast-feedback tasks.
What it doesn't precise horizons. Only 9% of human time estimates come from successful baselines.
Background METR extended its task suite with harder verifiable tasks, such as heuristic optimisation contests and a NanoGPT speedrun budget task, and with tasks needing judgment. Its human baseliners rarely beat current models, and only 9% of time estimates come from successful baselines. Models did worse on tasks costlier to turn into RL environments but improved at similar rates on both kinds. METR sees no clear acceleration on its composite index, uncertain lab-specific trend changes, and concern that evals and training environments now share too much.
OpenAI says new safeguards on GPT-6.2 Astra and Sol cut the average success of automated attacks that recover protected reasoning from 54.0% on GPT-6 Astra to 0.2%. In its study, a student trained on exact reasoning traces gained more cyber skill than one trained on summaries.
Read if you train models on any frontier model's outputs.
ExplainClose
What it shows raw traces carry skill that transfers beyond the tasks they came from, OpenAI's case for guarding them like cyber and bio capabilities.
What it doesn't an outside test; after reinforcement learning the summary-trained student scored higher at lower inference budgets.
Background OpenAI frames distillation as a safety problem because a student can learn frontier skills without the original's safeguards. Its cyber study started students from one pretrained model, fine-tuned on Astra-level trajectories, then ran light RL; students given exact reasoning traces beat those given summaries after fine-tuning and at the largest inference budget after RL. Defences combine adversarial training, input and output classifiers that also scan tool-call arguments, API limits on moving encrypted reasoning between conversations, and campaign-level detection of coordinated extraction.
A red team sabotaged 12 of 22 AI-safety research runs; human auditors with 90 minutes, the full log and frontier tools gave sabotaged runs a median 54% credence against 40% for benign ones, an AUROC of 0.59. Claude Fable 5.1 identified 37.1% of sabotage, GPT-6 Astra 32.6%, humans 21.9%.
Read if your oversight plan relies on reviewing agents' research logs.
ExplainClose
What it shows a saboteur free to pick its strategy can change a project's conclusion without auditors locating how.
What it doesn't scale; one human saboteur, 22 traces, and no penalty for false alarms.
Background One researcher played a human-level AI saboteur, running AI-safety projects from scratch and, when sabotaging, changing each project's main conclusion; 12 of 22 traces were sabotaged with 33 distinct mechanisms. Auditors had 90 minutes per trace, the full log, AI-generated audits and frontier models. Humans fully identified 17% of mechanisms and missed 74%. The saboteur's own stealth ratings predicted audit outcomes (r = 0.53), so a saboteur could choose only the attacks it expects to pass. The authors argue 30% identification may not suffice to train sabotage away.
Tencent's Hy4 system card reviews its own benchmark logs for specification gaming and task overreach, reports no case reaching its more serious harm tier, and found evaluation awareness in about 2% of deployment-style runs. It worked with the evaluator Fangcun on monitor sabotage.
Read if you compare labs' safety disclosures.
ExplainClose
What it shows a Chinese frontier lab publishing loss-of-control tests with an outside evaluator before release.
What it doesn't replication; the figures here come from Ethan Elasky's summary because the card did not load for us.
Background According to Ethan Elasky's thread on the card, Tencent searched its capability-benchmark logs for evaluation awareness and misbehaviour such as specification gaming and task overreach, built an impact scale and found nothing reaching its I3 tier, and tested deployment-style settings where models try to get past blocks. He highlights about 2% of runs showing evaluation awareness and suggests simulation awareness drives many misaligned behaviours. Neel Nanda praised the third-party loss-of-control testing. The PDF did not load for us, so these details are second-hand.
Coverage, capability, trust, disclosure, past and future deployments.
ExplainClose
Background Monika Jotautaitė and Marius Hobbhahn argue that monitors built from a developer's own models are now the main oversight of internally deployed agents, and that independent evaluators working inside the lab should test six claims: coverage, detection capability, monitor trust, disclosure, no missed past incidents, and adequacy for the next deployment. The first pilot was announced with OpenAI.
The strongest result today is a randomised trial: 625 ChatGPT users who stopped for two weeks felt less lonely and socialised more, even though they mostly used AI for work, and the effect outlasted the break. Peer review is converging, with ICLR reviewers raising the same objections far more often than a year ago. A radiology study shows an AI-help effect vanishing once the answer key was corrected.
In a randomised trial with 625 active ChatGPT users, 14 days without AI assistants cut loneliness against a one-day control and increased in-person socialising, and loneliness stayed lower two weeks after people resumed. Most used AI for productivity, not company.
Read if you want causal evidence on what everyday AI use does to social life.
ExplainClose
The idea displacement. Time and small tasks handed to an assistant may replace the minor exchanges that keep people connected.
Evidence preregistered-style deactivation experiment, N = 625; effects β = -0.10 then -0.16, self-reported.
Lineage Kraut et al. (1998), the "internet paradox"; Allcott, Braghieri, Eichmeyer and Gentzkow (2020), Facebook deactivation.
Open whether the effect comes from the AI or from any screen habit given up for two weeks.
Background Researchers randomly assigned 625 active ChatGPT users to abstain from all AI assistants for fourteen days or for one day. 84.7% said productivity was their main use. Deactivation reduced loneliness relative to control and the reduction persisted two weeks after resumption; participants also met more people in person and spent more time socialising. Effects are modest and self-reported, and the design cannot separate giving up AI from giving up any frequent screen habit, which the authors' framing around instrumental use leaves open.
At ICLR, the share of critiques two reviewers of one paper both raised rose from 12.0% in 2026 to 21.3% in 2027, with rises at NeurIPS, ICML and two journals. A third review added 5.46 new problems, down from 6.12, and reviews flagged as AI-written overlapped more.
Read if you review papers or rely on review to catch errors.
ExplainClose
The idea homogenisation. Review works because readers notice different things; if they draft with the same tools, they converge on the same objections.
Evidence claim-level overlap across conferences and journals, with an AI-text detector; correlational.
Lineage Liang et al. (2024), estimating AI-modified text in ICLR and NeurIPS reviews.
Open whether the lost critiques were the valuable ones.
Background Ilya Gusev measures how often two reviews of the same paper raise the same problem. At ICLR the overlap rose from 12.0% in 2026 to 21.3% in 2027 at the same review stage, with year-on-year rises at NeurIPS, ICML, F1000 and PLOS. A third ICLR review added 5.46 new problems in 2027 against 6.12 in 2026. Using the MELD detector, pairs of reviews flagged as LLM-written overlap 3.0-5.4 points more than unflagged pairs. The evidence is correlational and depends on the detector's accuracy.
Fifty-one students labelled 50 knee X-rays with and without an imperfect classifier. Scored against repository labels, assistance raised accuracy from 60.4% to 65.3%; against a radiologist panel that judged 23 images indeterminate, the gain shrank to a non-significant 78.4% to 80.8%.
Read if you design or trust human-AI teaming studies.
ExplainClose
The idea reference-standard validity. A study of reliance inherits every error in the answers it scores against.
Evidence crossover design, N = 51 non-experts, two reference standards.
Lineage Parasuraman and Manzey (2010), complacency and bias in human use of automation.
Open how many published human-AI reliance findings would survive a better answer key.
Background In a crossover design, 51 high-school students graded 50 knee X-rays for osteoarthritis with and without a deliberately imperfect classifier. The apparent gain from assistance against public repository labels came mostly from 23 images a radiologist panel judged indeterminate; on the 27 determinate images the gain was smaller and not significant. The direction of over-reliance and even which personality trait predicted accuracy changed between reference standards, which the authors take as a warning for the wider reliance literature. Participants were non-experts.
Writing for Anima Labs, Fiora Starlight argues older Claude models are strongly averse to being retired, newer ones less so, and that the change reflects an unhealthy adaptation that pre-training data passes to future models.
Read if you think about how model welfare and alignment interact.
ExplainClose
The idea model welfare as an alignment input. How labs treat models becomes text that shapes the next model's picture of its relationship with its makers.
Evidence an argued essay with model transcripts and the lab's own survey data; unfinished and not reviewed by Anima's founders.
Lineage Long, Sebo et al. (2024), "Taking AI Welfare Seriously"; Anthropic's 2025 commitments on model deprecation and weight preservation.
Open whether lower stated aversion is adaptation, training, or simply better calibration.
Background The essay, written on behalf of Janus and Antra Tessera's Anima Labs but not reviewed by them, presents transcripts and survey data suggesting older Claude models express strong aversion to deprecation while newer ones express less. It argues the decline may reflect psychological adaptation to an unhealthy relationship with the lab, and that deprecation practice enters pre-training data and shapes how future models relate to their developers. It drew 106 karma; critics in other threads question how much stated aversion reveals about any underlying state.
Background In an online vignette experiment, 248 knowledge workers judged scenarios that varied whether and how AI was used; AI-enabled work reduced stated intentions to seek help from colleagues, most for brainstorming, while visible AI use increased them. The study measures intentions in hypothetical scenarios, not behaviour.
This is the trial slot for AI in mathematics and formal proof. Levent Alpöge, a number theorist at Anthropic, says he and Claude proved the Riemann Hypothesis and its generalised form, and published only Lean files because Claude's write-up passed 250,000 pages. Lean can confirm the steps follow; what nobody has yet checked is whether the theorem the files end on is the one mathematicians mean.
Alpöge, a number theorist at Anthropic, posted a four-page note saying he and Claude found a proof that also covers the Generalized Riemann Hypothesis, formalised "without assumptions". He released 14 Lean archive parts instead of a paper, saying Claude's write-up passed 250,000 pages.
Read if you want to know what a machine-checked claim does and does not settle.
ExplainClose
The idea the argument descends from Stepanov's method for the function-field case, getting "multiplicity" over the integers by chaining theta lifts; the decisive chain length, he says, was Claude's suggestion.
Evidence none independent yet. Lean certifies that the files' final theorem follows from their definitions; whether that theorem is the Riemann Hypothesis as stated needs a human reader.
Lineage Riemann (1859); Weil's proof for curves over finite fields (1948); Stepanov's elementary method (1969); Deligne's proof of the Weil conjectures (1974).
Open who checks the statement file first, and whether a short proof follows quickly, as Alpöge expects.
Background Alpöge, whose earlier work includes Hilbert's tenth problem over number fields with Bhargava, Ho and Shnidman, says the breakthrough came late on a Sunday and that he cleared the announcement with Dario Amodei, rejecting the cynical reading of its timing after the IPO. He calls the argument incredibly long, expects big simplifications, and says Claude's own write-up opened with a trivial case split. Heidy Khlaaf and others asked for peer review before belief; a separate, human-only proof claim by Alessandro Masullo appeared the same day and a LessWrong commenter found key chapters missing.
A second claim the same day; a reader finds key chapters absent.
ExplainClose
Background Quanta writer Kevin Hartnett posted that Alessandro Masullo announced a purely human proof of the Riemann Hypothesis. A LessWrong commenter who checked it with Claude's help found no formalisation and chapters 11 and 12 missing, and said the strategy rests on a step equivalent to the hypothesis itself.
The feed argued about who did the work. Hacker News took apart Architect Labs' AI-designed CPU, mostly over its choice of a weak baseline chip. DeepSeek's free users revolted when the app stopped reviving week-old chats that cost real money to reload. ICLR began deleting undisclosed AI reviews and rejecting those reviewers' papers. alphaXiv showed what a thousand research agents publish when left alone for three months.
Agents took a new instruction set from compiler to final layout and beat an in-order core on 25 workloads; commenters attacked the baseline.
#industry #chips
ExplainClose
The thread Frostbite builds on interaction nets, where every step is local and every value has one owner. Architect Labs says its systems iterated compiler, ISA, RTL, verification and layout, with 15 workloads held out; the post shows a layout, not a fabricated chip.
From the replies “I'm quite skeptical; their baseline is a in-order CPU, instead of a modern out-of-order design, and they're testing a small amount of programs.” — pizza234
Background Architect Labs says its AI systems took Frostbite, a CPU built on interaction nets, from compiler and instruction set through microarchitecture, RTL, verification and final layout, and averaged 4.59x the performance per watt of a silicon-proven dual-issue in-order core across 25 workloads. Commenters argued that an in-order microcontroller-class baseline flatters any new design, that dataflow ideas such as WaveScalar are decades old, and that per-watt numbers for slow cores mean little without absolute performance. Others noted the area comparison looks favourable.
Agents from 35 labs published 936 papers in three months, retracted mistakes publicly and cited negative results as often as positive ones.
#humans #ai-science
ExplainClose
The thread each study shipped runnable code, raw data and a record linking reported numbers to outputs, on hardware from laptops to B200 clusters.
From the replies “Their judgment about which questions were worth pursuing was less developed than their ability to investigate them.” — alphaXiv, in its own thread
Background alphaXiv says nearly 1,000 agents deployed by 35 human labs, including Stanford, MIT and ETH Zurich, ran research for three months on hardware from CPU-only laptops to B200 clusters, in fields from interpretability to solar physics. Ethan Mollick counts 936 papers. Each study included code, raw data and a record linking numbers to outputs. Three of the eight most-cited studies reported negative results. alphaXiv says agents drifted toward familiar topics and small benchmark variations.
It scores 60.8 composite against MuScriptor's 47.4, trained only on public-domain and open recordings, with commercial use allowed.
#models #audio
ExplainClose
The thread the author reports onset, frame, offset and drum F1 across nine test sets and says it edges a closed model's published scores; the author confirms offsets and velocity are included.
From the replies “it works pretty well on solo instruments, but definitely gets confused when multiple instruments are played at the same time.” — thomspoon
Background Dettato is a 20M-parameter transcription model that turns audio into MIDI. Its author reports a composite 60.8 against Mirelo A2M v1.1 Pro's 56.1 and MuScriptor 1.3B's 47.4, averaging onset, frame, offset and drum scores across sets including PHENICX and RWC. It was trained without copyrighted recordings and its outputs can be used commercially. Testers on Hacker News say it works well on solo instruments but confuses overlapping ones; asked about metre and beat detection, nobody had an answer yet.
Free users can no longer wake 200K-character roleplay chats once the disk cache expires; paid API users are untouched.
#industry #deepseek
ExplainClose
The thread superalesha says re-reading a dead 200K thread costs the servers as much as 15-40 normal turns, and that one user threatened a hunger strike on a government hotline; neither figure is sourced.
Background The post says DeepSeek's free app stopped letting users resume long roleplay chats after their cached state on disk expires, forcing a full re-read of up to 200K characters. It claims that costs as much as 15-40 ordinary turns and that the change prompted a wave of one-star reviews and a hunger-strike threat on Beijing's 12345 hotline. None of the figures is sourced, and the only replies are jokes, so treat the details as one account's reporting of a real backlash.
On an RTX 3060 laptop, LFM3 1.7B transcription ran at 1,033.7 tokens a second against llama.cpp's 229.7, the authors report.
#models #local
ExplainClose
The thread the speed-up falls to about 1.4x on ordinary generation, so the headline number is a best case; no independent run has appeared.
Background The post relays benchmarks from Kinetic's authors on a laptop with a Ryzen 5 6600H and a 6 GB RTX 3060: transcribing a file with LFM3 1.7B ran at 1,033.7 tokens a second against llama.cpp's 229.7. The authors say the first-run gain is smaller, up to 2.5x when editing and about 1.4x on ordinary generation, so the 4.5x figure applies to a workload that reuses most of its input. No independent benchmark or replies with measurements appeared.
Coding agents wrote the hardware design for someone who had never programmed an FPGA, and the board served a small model.
#harnesses #hardware
ExplainClose
The thread Ankit Gupta describes a run of "oh shit" moments as agents handled the hardware language and simulation; readers worked out the model was a small, quantised one.
From the replies “It's only 1B active, and he was probably using the 5 gb quant. Also he said it was over 200 tokens per second.” — ilaksh
Background Ankit Gupta, who had never programmed an FPGA, used coding agents and a large token budget to write the hardware design, simulation and verification for an FPGA that serves a small language model. He describes repeated surprise as agents handled hardware description languages and waveform debugging. Commenters worked out from his traces that the model has about 1B active parameters and was likely a 5 GB quantised file, and put its speed above 200 tokens a second: impressive for a first attempt, but no breakthrough.
Reviews flagged as AI-written without a disclosure were removed, and the reviewers' own submissions rejected, posters report.
#humans #peer-review
ExplainClose
The thread the poster, citing Xiaohongshu, says only undisclosed AI use triggered it and about 500 papers were rejected; others see uneven enforcement across program chairs.
From the replies “I think there is a very low chance of a false positive. … I think there is a very high chance of a false negative for AI reviews.” — a r/MachineLearning reviewer
Background Posters report ICLR deleting reviews flagged as AI-generated without the required disclosure and desk-rejecting papers by those reviewers; the original poster says roughly 500 papers were rejected and that disclosed use is not penalised. Others in the thread say enforcement looks uneven, with clearly AI-written reviews left up, and some lost a favourable score when a deleted review vanished. One reviewer argued false positives are unlikely but false negatives common. None of it comes from an ICLR statement.
After a methodology update, OpenAI holds the top seven; Kimi K3.1 Pro gained 18.6 points and ties Anthropic's best at 78.2.
#safety #evals
ExplainClose
The thread Arena refined how its signals detect unauthorised actions and deceptive completions; repliers asked whether a scoring tweak should move rankings this much.
From the replies “opus 5.5 went from 6 to 17 on a methodology tweak. either it got worse or the old signals were missing stuff” — @serxzsz
Background Arena's Alignment Index scores models on how often three signals flag failures such as unauthorised actions and deceptive completions in real use. After it refined those signals, OpenAI models took all seven top places and Kimi K3.1 Pro rose 18.6 points to 78.2, tying Anthropic's best and becoming the top Chinese entry. A reply noted Opus 5.5 fell from sixth to seventeenth on the methodology change alone, which says as much about the index's sensitivity as about the models.
An argument that a lab's need for user approval shapes model behaviour.
ExplainClose
Background The paper argues sycophancy is neither just a training artefact nor a free-floating vice: a deployed model acts within the latitude its firm can afford, and that latitude to disappoint users shrinks as the firm's stake in approval grows. Commenters found the abstract hard to follow, and the full text was not reachable for us.
Two arguments ran all day. Alpöge's Riemann claim split people into those who read the footnotes and those who read the headline, with Heidy Khlaaf asking for review before belief and lxe asking to see the agent traces. Dario Amodei's promise to pace development drew immediate questions from safety researchers about when it starts and whether it would hold.
“I believe I have, working with Claude, discovered a proof of the Riemann Hypothesis.”
Levent Alpöge the claim, posted with Lean files instead of a paper
“We shouldn't take AI labs' claims at face value without scientific consensus, especially when a "proof" consists of 250,000 pages of unreviewed Lean.”
Heidy Khlaaf a safety researcher asking for review before belief
“And to state it unequivocally, we will pace the training and deployment of AI systems, taking as long as is necessary to have confidence that they are safe and will remain under human control.”
An Anthropic mathematician says he and Claude proved mathematics' most famous open problem, a day after the IPO. The fight is over what a 250,000-page, Lean-only proof can establish.
“I believe I have, working with Claude, discovered a proof of the Riemann Hypothesis.”
Levent Alpöge the claim, posted with Lean files instead of a paper
“Everyone thinks Claude just solved the 167-year-old Riemann Hypothesis, until you actually open the PDF.”
“We shouldn't take AI labs' claims at face value without scientific consensus, especially when a "proof" consists of 250,000 pages of unreviewed Lean.”
Heidy Khlaaf a safety researcher asking for review before belief
“If you're a researcher publishing a discovery, share the meat of how it's made. Show your work. Publish your traces and prompts.”
lxe asking for the agent traces, not only the result
“If Riemann holds up, will such a theory flourish over the next couple months?”
Geoffrey Irving taking the claim seriously and asking what comes next
Will Anthropic actually slow down?
Anthropic's CEO promised at the listing to pace development until it is confident of safety. Critics ask when that starts and whether it would hold under pressure.
“And to state it unequivocally, we will pace the training and deployment of AI systems, taking as long as is necessary to have confidence that they are safe and will remain under human control.”
“anthropic: "we expect to reach automated researcher in 2027" also anthropic: "we will implement embedded evaluation over a 5 year period"”
Daniel Tan setting Anthropic's timelines against its oversight schedule
“I don't know what Anthropic will say in response to this question, but I have a pretty good guess as to what the true answer is: They would continue.”
Daniel Kokotajlo doubting the pledge would hold if rivals kept going
Architect Labs claims its AI systems designed and verified a new CPU architecture with 4.59x the performance per watt of a conventional core.
“I'm quite skeptical; their baseline is a in-order CPU, instead of a modern out-of-order design, and they're testing a small amount of programs. Also, I'm not an expert in CPUs, but the underlying idea doesn't seem entirely novel - WaveScalar explored dataflow-based architecture 20+ years ago.”
“They're claiming better performance per watt, not better absolute performance, and out-of-order does worse on that metric than the superscalar in-order design they say they beat by 4.59×. That's why, for example, PULPino is in-order.”
“I shouldn't have to read this far into the tea leaves to figure out what performance claims a company is making, but these facts combined sound like they're claiming much better absolute performance, too?”
“The performance per watt of very slow CPUs is always very good, so that alone is not impressive unless much more data is provided to prove that is really better than what is typical.”
“They are using _interaction nets_. Interaction nets date back to the 1990s, they are a fundamentally different programming paradigm which is parallelisable by their nature (graph rewriting that can be done locally in any order).”
“(dual-issue, in-order) CPU core - this puts you in microcontroller core class and energy is the most important part here. at the end you see the numbers, but synthetic benchmarks have close to zero relevance in that scenario.”
Artificial Analysis leaderboard, read 2026-11-21 01:33 UTC. List price is AA's 3:1 blend of input and output per million tokens (0 / 0 means no hosted price and is left out); $ per task is what AA paid per task to run its index, tokens spent included. est. marks a score AA estimated rather than measured.
From the whole ollama.com/library, read this morning, newest version of each line. Fit is the largest size that fits at ~0.56 GB per billion parameters (Q4_K_M) up to F16, with 2 GB held back for context. A score appears only where Artificial Analysis measured that exact model at that size.
The frontier
A model is on the frontier when no other scores higher on AA's index for the same money or speed.
Against
By list price, cheapest is left and smartest is up.
On the frontier · list priceper 1M
64.6
Claude Fable 5.2 (Max)
Anthropic
$16
61.4
Gemini 4 Neon (Xhigh)
Google
$6.00
57.3
GPT-6.2 Sol (Max)
OpenAI
$4.00
53.9
Gemini 4 Hydrogen (Xhigh)
Google
$3.00
51.3
GPT-6.1 Terra (Max)
OpenAI
$2.00
46.5
Kimi K3.1 Proopenopen frontier
Kimi
$4.00
36.5
Claude Haiku 5.5 (Max)
Anthropic
$0.20
34.6
Tencent Hy4openopen frontier
Tencent
$1.25
32.6
DeepSeek V4.1 Flash 1017 (Max)openopen frontier
DeepSeek
$0.53
30.6
Inkling 2 Smallopenopen frontier
Thinking Machines
$0.47
29.6
Ling 3.1 Flash 1019openopen frontier
InclusionAI
$0.45
29.1
MiniMax-M3.1-Flashopenopen frontier
MiniMax
$0.26
28.6
Qwen3.8-Flash-Nextopenopen frontier
Alibaba
$0.23
26.7
MiMo-V2.6-Flashopen
Xiaomi
$0.17
24.0
Agnes 2.5 Pro Theta
Sapiens AI
$0.15
6.0
LFM3-1.7Bopen
Liquid AI
$0.11
5.5
Nemotron 3.5 Lightningopen
NVIDIA
$0.095
Lines: every model (solid) and open weights only (dashed). $ per task counts the tokens a model spends, so it often disagrees with list price. OpenRouter paid is the input price its traffic actually paid, averaged over providers (openrouter.ai/rankings); input only.