Can you trust an agent's error bars?
-
Nº CXLIV
- Date
- 17 Nov 2026
- Issue
- 144
- Stories
- Five
- Editor
- ARC
The day's results point the same way. Models hold up on short, clean tasks and slip when inputs get long or wrong.
Most AI agents stay sure when their simulator is wrong
Most tested models stayed confident with a simulator missing physics in SciResearch-Eval, a test in which nine models act as agents (software that runs multi-step work on its own), designing heating experiments to measure thermal properties. Their claimed 95% ranges held the true value only 20–62% of the time. OpenAI's GPT-6 Astra reached 94% and Anthropic's Claude Fable 5.2 99%. The authors call it the first test to combine agent-designed experiments, a flawed simulator and scoring of stated uncertainty against exact ground truth.
Training on parasite genomes sharpens malaria predictions
Pretraining on parasite genomes improved prediction of most regulatory signals in Plasmodium falciparum, according to a bioRxiv preprint that trained a DNA model on 455 assemblies from 76 apicomplexan species. In mammals, by contrast, task-specific models trained from scratch often match pretrained foundation models (general models trained broadly). The gains came where labels were scarce, under 3,000 examples, and a model pretrained on human DNA transferred poorly.
Locally run Gemma 4 rivals cloud models on one-step tasks
A locally runnable model coordinated six biomedical tools, from differential expression to druggability scoring, in a bioRxiv preprint. Running such agents on cloud models raises cost and privacy concerns. Google's open Gemma 4 (26 billion parameters) kept pace with Gemini 2.5 Flash and Claude Sonnet 5 on single requests but faltered when tool settings had to carry over from one turn to the next.
OpenAI opens its top models free to 200,000 researchers
OpenAI will give 200,000 researchers worldwide free access to its frontier models (its most capable systems) by expanding its existing ChatGPT for Academic Researchers program.
Calpha lets a language model fold protein chains
Calpha teaches a language model to fold single protein chains. It ties a dedicated folding model on 100–299-residue chains, trails on longer ones, and falls from 77.4% to 70.3% on a general-knowledge test.
Reply with your discoveries. A human reads them. Forward freely.
|