Artificial intelligence models are brilliant at specific scientific tasks, but they do not yet work the way human scientists do, according to physicist Matthew Schwartz.
In a guest post published on Anthropic's research site on Thursday, he argues that researchers get the most from AI by giving it problems suited to its strengths.
The 'Impedance Mismatch'
Schwartz says the models are smart, but most of science is not like mathematics, where a problem can be fully stated and an answer checked absolutely. He borrows a physics term, "impedance mismatch", to describe two systems that each work fine but are poorly matched.
Here, the gap lies between what scientists want and what AI does well. Last December, he found Claude behaved like a strong graduate student at 20 times the speed, but he had to correct its writing and pull it back from dead ends.
What 'Claude-Shaped' Means
This summer, he changed approach and looked for what he calls "Claude-shaped" problems. These draw on Claude's breadth of knowledge, coding skills and command of mathematics, and on its ability to read papers and data at machine speed.
The work led him to build BootLoops, an open-source toolkit for exact calculations in quantitative science. In particle physics, it computed 30 complex integrals, 15 of them never calculated before.
ALSO READ: 'Big Short' Michael Burry Warns AI Bubble Could Burst Sooner Than Expected, Shifts To Puts
From Physics To Forests And Genomes
Since the same equations recur across disciplines, the toolkit travelled. Schwartz reports 36 manuscripts across 18 fields in three months. These include a model of tree life histories in forests, an analysis of 5.7 billion pairs of human genetic mutations that found evidence of gene conversion, and a database of word stress covering 6,072 languages.
An AI data editor also checked the replication packages of 4,452 economics papers.
Why Experts Still Matter
Schwartz cautions that Claude's results were often technically correct but scientifically unremarkable. Experts steered them.
An ecologist told him one finding would likely "be met with a shrug", then helped turn it into a better model. Schwartz admits he can judge Claude's claims in his own field, but tends to simply agree when it works in unfamiliar ones.
ALSO READ: Chinese Hackers Posed As US AI Experts To Target Policy Researchers, Steal Emails
Where The Tools Fall Short
He lists familiar failings: Claude declares victory too early, misjudges how long tasks take, grinds through long calculations instead of building faster tools, and cannot be trusted to judge what is interesting.
His advice is to check plots, question conclusions and supply the "taste" himself.
What It Means For Scientists
Schwartz says he sees no need to revisit the scientific method, since real progress still depends on data that must be gathered and checked. He is unsure how to train students, and worries about credit going to those who merely type.
Yet he is optimistic: freed from tedious calculation, humans can focus on direction and conceptual work.
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.