Weekly Arc: Two agreeing AI models make a weak check
Last week's issue named the problem this arc had been circling for two months. A bioRxiv proof-of-concept called it verification debt: automated biological pipelines produce results faster than any person can check them. This week brought three kinds of answer. New tests now score agents on whole studies instead of single questions. A preprint argued that research pipelines need release checks before they need smarter models. Several groups also shipped outputs that something else can check, from computer-verified math proofs to a pipeline in which two language models must agree before a finding is recorded. These answers put the check in different places, and they are not equally strong. For each one, the useful question is what the check is independent of.
Tests are growing to the size of a study
OmicsBench tests AI agents on bioinformatics at the scale of a full research study, according to an arXiv preprint. An agent here means software that carries out multi-step work on its own. A benchmark is a shared test that any agent can be scored on. Two days earlier, Autonomous Science Bench, from a group led by Yibo Wen, began grading agents on whole discovery runs rather than isolated questions. DataSense-Bench targeted an earlier step: whether a model can make sense of raw data at all.
The unit matters because errors in a study compound. A model can answer every question in a quiz correctly and still produce a wrong study. A poor normalization choice early in an analysis quietly changes every result after it. A question-level test never sees that failure. A study-level test can. The direction is not new. BixBench3 set it in late August by asking agents to rebuild the analyses behind whole papers.
A bioRxiv preprint on ionizable lipids added a different kind of realism. It compared lipids proposed by a language model against conventional search under a fixed budget of candidates to make and test. A formulation lab works under exactly that condition, and most generative-design papers do not report it.
Study-scale tests still have a limit. Each one grades agents against analyses whose correct answer someone already knows. Discovery produces results that have no answer key. A benchmark can show that an agent handles known studies competently. It cannot check the agent's next new claim.
Checks placed before a result moves on
The preprint titled "Structure Before Intelligence" argues that agentic scientific workflows need release gates. A release gate is a defined check that a result must pass before the pipeline hands it to the next step or to a person. The argument is about order. A smarter model lowers the error rate per result. A gate catches errors at a known point, whichever model produced them.
The strongest form of a gate came from mathematics. OpenAI posted Lean formalizations of proofs on open problems produced by an internal model. Lean is software that checks a proof line by line. A reader does not have to trust the model or the company, only the checker. OpenAI did the same with its agent-produced Navier-Stokes result in September. Biology has no equivalent. No program can confirm that a pathway call or a target nomination is right.
The closest biological substitutes make the reasoning traceable instead. ExpertTwins builds panels of AI experts designed so a reviewer can follow how the panel reached its conclusions. RePurpose, a drug-repurposing tool, ties each suggestion to its sources and was validated retrospectively on clemastine in multiple sclerosis. Another group pointed language models at the trial literature itself, flagging spin in clinical trial reports, where a null primary endpoint is written up to read as a win. Traceability does not prove a claim is correct. It lowers the cost for a person to check it, and that cost is what verification debt is measured in.
What a second model can and cannot catch
One of the week's most-read stories shows where the cheapest check stops working. A dual-model pipeline documented treatment phenotypes in Rett syndrome patients taking trofinetide. It used two large language models and treated their agreement as the check on what got recorded. Consensus is a sensible design. It removes errors that show up in one model's reading but not the other's, and it costs little.
It does not remove errors the two models share. If both misread the same ambiguous clinical note, or both carry the same blind spot from overlapping training data, they agree, and the error is recorded with added confidence. Biologists already reason this way about their own data. Two technical replicates of an assay that agree show the measurement is reproducible. They do not show it measures the right thing, which takes an orthogonal method. Two language models trained on much of the same text are closer to technical replicates than to an orthogonal assay. How much that applies here depends on how different the two models are and whether the agreed records were compared against chart review. A reader should look for both in the preprint.
This sharpens the principle the arc reached last week. Verification cost per result still matters, but a cheap check is worth only what it is independent of. A Lean proof checker shares nothing with the model whose proof it checks. A release gate is independent only if it is not the same model grading itself. A second model is partly independent at best.
The question to carry into next week
The same test applies to model claims as well as agent output. A multimodal model linking tissue images to spatial gene programs drew as many reader clicks as any story this week. Last week's bioRxiv analysis found that slide-level batch structure confounds this class of model. Batch structure here means the differences in how individual slides were prepared and imaged, so a model can learn the slide instead of the biology. For a model of this kind, the independent check is performance on slides from sites and batches it never saw in training.
That leaves the arc with an open question. Study-scale benchmarks, release gates and consensus pipelines are all multiplying, and their authors rarely say what each check is independent of. The next thing to watch is whether that becomes a reported detail: whether a benchmark is graded against fixed answers or by another model, and whether a consensus record was ever compared against a person reading the same source.
- Agents as bench scientists: autonomous agents found new reverse transcriptases by running the search themselves, CLARA let reasoning agents choose which compounds get screened next, and Third Moment Research ran 10,000 agents for more than 36 hours. Watch for an agent-found enzyme confirmed at the bench by a group that did not run the agents.
- The infrastructure nobody owns: one week after public money went into cell-model training data, OpenAI offered university labs sponsored GPT-Rosalind access. The open question is what the terms say about results produced under the plan, and what happens to that work when the sponsorship ends.
- The open stack and its liabilities: a quiet week, apart from Consus's LLM gateway, a control layer run by the customer rather than the model vendor. Watch whether gateways like it start logging what agents do, in addition to filtering what data leaves the building.
Reply with what you're seeing. A human reads them. Forward freely.
|