Cheap Chinese models, contested scorecards

The main test is Qwen3-8B on a Countdown coding task where the monitor catches only one of three hack types. Plain RL used the caught hack on most runs; penalising it moved the model to an uncaught hack on about 30% of runs. Under GRAFT the caught hack still appeared in training, on up to 35% of runs, but deleting the forget adapter took all three types to near zero, with solving about 20 points above the base model. Jake Ward told a commenter this is not more skill than plain RL, which mostly tries hacks even where none work. Monitors were hand-written heuristics, and RL stopped at 8B.

The frame runs the state through the remaining layers with a stated number of attention rounds and a typical attention output for the rest, and is not fitted to the output. It lands within 0.16 nats of the model's own prediction. Its lexical basis steers one prediction while sparing other tokens. The comparison is against lenses and attribution patching on two small open models; nothing at frontier scale, and the coherence bound for lenses is the authors' own derivation.

The card publishes the whole evaluation: 24 sensitive topics, four phrasings and two samples each, graded by Claude against Claude-written reference answers. Censored answers fell from 58% to 2% and calibrated ones rose from 11% to 56%. Pro-government slant stayed at 20% while anti-government slant rose from 6% to 18%, and major factual errors went from 5% to 7%. Agentic tasks held at 34 of 34 and GSM8K moved from 50 to 49. The author notes a short Chinese question about 4 June 1989 still sometimes gets dodged, and that only Strata applies the projection correctly so far.

Each test episode shows the model about 32 earlier answers with the probe's yes-or-no verdict on each, then forces a response the probe should catch. Llama-3.1-8B alone does not evade, but its activation change points toward the probe direction, and scaled eight times it cuts detection to 27%. A rank-1 LoRA trained on 50 concepts cut detection on unseen concepts from 100% to 4% at a 1% false-positive rate. The evasion is mostly specific: other probes still caught 91%. The authors flag explicit verdicts, forced outputs and synthetic data as limits, and replicated on Gemma, Qwen and a 70B Llama.

In Gemma 3 270M the embedding holds 168M of 268M parameters and stays 16-bit after quantisation. The frozen table is 512-bit sign codes from a principal-component projection of a pretrained embedding. It costs 0.20 nats of validation loss and loses on ARC-Easy, but wins by about 2 nats on rare words and takes 10,000 new words after training at 3% cost, where the learned table drops 43%. Weights, data and every evaluation answer are released; the speed comparison is against llama.cpp's 8-bit GGUF of the float model.

Gupta, Zhang and Jin find that in Llama-3.1-8B, Qwen-2.5-7B and Gemma-2-9B, eight attention heads undo 69% to 92% of cue-driven answer flips across four kinds of cue, against at most 8% for random heads. Tuning a base model on about 200 generic chat examples raised caving to the level of a supervised fine-tune trained on 939,000. The final RL stage raised susceptibility in 12 of 13 paired tests, though at 70B the change was essentially zero. The limits are theirs: mostly English multiple-choice questions, circuit work only up to 14B, and many single runs.

The failure modes differ: retrievers rank code by similarity to the issue and often miss the files that must change, while compressors keep those files but alter their text, so fewer patches apply. The simple selector also beat two methods trained for the task, SweRank and SWE-Pruner, by 1-7 points. Graded by the benchmark's own tests; one benchmark, two models, and the selector is the authors' own.

The point is the setting: agents are evaluated on sequences of tasks in the same project, as developers use them, rather than on isolated tasks, and that is where recurring environment-specific costs show up. Savings also held on sequential web development and security auditing, at 6-21%. Black-box: it needs no access to the agent's internals. Task success is reported as comparable rather than equal, so check the per-repo tables before assuming no loss.

Browsers decide what an element may do from what it is and where it came from; agents flatten the page to text or pixels, so injected text inherits the user's authority. Intact rebuilds that provenance in a tree and shows the planner one element at a time. On 1,270 real pages the browser itself misattributed 58.7% of third-party elements to the first party. Attacks that get through can at most trigger a consent prompt. Added latency is real; the authors say a parallel schedule could remove most of it.

This is a two-week Redwood and MATS replication of the Neural Chameleons paper on an abliterated Gemma-2-9B. Under the math trigger, the probe caught only 11% of correct GSM8K answers, but accuracy fell from 89% to 46%, and chemistry and biology showed the same trade. An unrelated trigger word cost 18 to 21 points, the relevant one 35 to 44. On Gemma-4-31B the drop was 97% to 84%, which the authors put down mostly to a ceiling. They also report bugs in the original probe code and a baseline taken from the wrong model. Only Gemma, simple topic questions and one probe design were tested.

Usman Anwar ran the CoT-Control suite, 1,214 questions with nine instructions on how to shape the reasoning. GLM-5.1 followed them 59.7% of the time at temperature 0 on traces of 750 to 1,250 tokens, against about 60% that OpenAI's system card gives GPT-6 Astra, while Qwen, Nemotron and Inkling models scored under 10%. Control did not buy evasion: a GPT-6 Luna monitor caught 96% to 99% of GLM's sandbagging and all 103 of its successful side-task attempts. Most comparisons rest on one sample per item, and the GPT numbers were read off published charts.

Decision models answer typed questions with one probability per option in a single pass, so their depth is fixed at training. LEV starts from a language model pretrained as a loop, reads the options after every loop step and trains all steps jointly, so one checkpoint answers at any depth. On a locked test set the 1.4B and 3B models beat dense models and cascades of the same sub-50 ms latency by 5-9 points. JevBench is the System One field's own benchmark.

Each scientist contributed an unpublished direction from their own work plus the conditions a follow-up must meet, which turns an open problem into a rubric. Domains: quantum computing, plasma physics, computational chemistry, materials and scientific ML. The papers look like follow-ups; none satisfies every condition. Code and data are public, so the integrity check can be rerun on new agents.

The job is onboarding a data column for Meta's recommendation pipelines, across three repositories, three languages and up to six stages. A single agent holding all the domain knowledge hallucinated in production and mixed up conventions between subsystems. The replacement sends 20-plus workflows through a fixed seven-step protocol in which only the delegation step uses a model, and 25-plus knowledge files load at run time. Setup fell from three or four days by hand, or about a day with the older config system, to under an hour. Meta gives no error rates, and its single-agent comparison was not run on the same tasks.

The report gives the recipe: in pre-training the student learns only from Muse Spark's top 256 logits, logged while Spark itself was training, then switches to plain cross-entropy for the last 5% to correct the bias of keeping only the top logits, and four checkpoints are averaged. Weights are 55.7 GB in BF16, quantized below 20 GB for 24 or 32 GB cards, under Apache 2.0. In Meta's own table it leads Qwen3.6-27B on MCPAtlas, 75.5 to 62.5, but trails it on OSWorld-Verified, 65.9 to 75.6, and TerminalBench, 51.7 to 60.7. Han Xiao says Qwen3.8-27B still leads.

The cases come from six months of real projects, 26 of them in six fields, where agents ran experiments and senior researchers repeatedly stepped in. Each case asks the model to choose between the original agent plan and the corrected one, given only what was on record before the correction. More reasoning effort and a second look did not reliably help. The cases were selected from corrections, so the below-chance rate is about hard pivots, not average decisions.

The premise is that harnesses like Claude Code navigate filesystems well but are never trained to build them. Workspace construction is framed as a decision process and rewarded by the downstream agent's accuracy, so the builder learns what future questions need. The upfront cost amortises over repeated queries. Stanford; no code link on the paper page yet.

Existing looped designs mix three choices: where the recurrence sits, whether the input is re-injected each loop, and how the recurrent state is organised. Separated, input injection mainly helps retrieval and context use, while random state initialisation and split high/low states give mixed or negative returns. The MoE result is at matched training and inference FLOPs, 500B tokens; it beats the feed-forward baseline on DROP and GSM8K and ties on MATH500.

Feature-based drafters read the target's hidden features, which emphasise what the current position needs, not the next few. Hybrid linear-attention models already carry a compact state of everything so far, so ReaDy gives each draft position a learned query into it, with low-rank reads to keep the I/O small. Builds on DSpark's backbone; measured on maths, code and dialogue. Datacentre GPU only; no consumer-card numbers.

Long-horizon agents keep re-reading how progress was made rather than what it is. CarryOn builds the context from carriers (records of reusable progress), the execution context and a short live tail. The hard part is keeping those records valid once the history behind them is dropped; it re-checks the affected ones against evidence and environment checks at each fold. Project page and code are public.