AI may be getting better at science, but that does not necessarily mean it is ready to work like a scientist.
Harvard physicist Matthew Schwartz argues that there is an "impedance mismatch" between AI and scientific research - a gap between what current Large Language Models (LLMs) are capable of doing and what researchers actually need from them.
In a guest post published by Anthropic, Schwartz explained that in physics, an "impedance mismatch" occurs when two systems each work well but are poorly matched.
Schwartz argues that a similar disconnect exists between AI and science. While LLMs are capable of handling a wide range of technical tasks, simply working with them as one would with a human collaborator does not necessarily bring out their strongest scientific capabilities.
ALSO READ | Why Is Accenture Betting On Palantir's Forward-Deployed Engineer Model?
AI's Impedance Mismatch With Science
The idea is not that AI systems are physically malfunctioning. Instead, Schwartz uses the physics concept as an analogy for the disconnect between AI's capabilities and the way scientific research is traditionally conducted.
Current LLMs can process huge amounts of information, write code, build models and perform complex calculations. But they still require humans to provide context, determine which questions are worth pursuing and assess whether an apparently correct result is scientifically meaningful.
Schwartz's own experience illustrates how quickly that balance is changing.
Two years ago, he says he would have advised aspiring scientists and engineers to learn Python. Today, he is less certain that traditional programming skills will remain as essential as they once were, given how quickly AI systems are becoming capable of handling coding and machine-learning tasks.
In one experiment, Claude reproduced results from one of Schwartz's research papers in around 20 minutes - work that had taken him weeks to code himself.
But faster execution does not necessarily translate into better scientific discovery.
Schwartz says current AI models can carry out calculations and produce technically correct results while still struggling to determine which questions are scientifically interesting. In one example, Claude generated a mathematically precise ecological model that experts initially considered scientifically unremarkable. It took a domain specialist to reframe the question and identify a potentially more meaningful line of inquiry.
This illustrates what Schwartz sees as the continuing importance of human scientific judgement.
Where AI Falls Short
According to Schwartz, current LLMs are not autonomous scientists. Models such as Claude and GPT can process large datasets and perform sophisticated computations, but they can still struggle with context, scientific relevance and determining when a result genuinely matters.
Schwartz highlights four principles for researchers working with AI:
- Verify everything visually: Researchers should directly inspect AI-generated outputs and charts rather than relying entirely on automated checks.
- Question the conclusions: AI-generated calculations and reasoning should be examined for theoretical and scientific validity.
- Provide scientific "taste": Researchers need to guide AI towards novel and meaningful questions rather than simply asking it to solve problems that are already well understood.
- Push beyond the first answer: Researchers should challenge AI systems to look beyond an initial solution and explore whether a more significant finding is possible.
The underlying lesson is that AI can accelerate the execution of scientific work, but humans remain responsible for deciding what is worth investigating and whether the results have genuine scientific value.
How BootLoops Tries To Bridge The Gap
To address this disconnect, Schwartz developed BootLoops, an open-source software harness designed to guide AI tools such as Claude through quantitative scientific research.
Rather than treating AI as an autonomous scientist, the approach combines the computational speed of AI with human direction and domain expertise.
The objective is to identify what Schwartz describes as "AI-shaped" opportunities - technical connections between disciplines that AI may be able to uncover but that still require scientists to determine whether they are meaningful.
Using the approach, Claude identified potential connections across fields including ecology, population genetics and other areas of science. Schwartz then worked with domain experts to determine which questions were worth pursuing.
The experiments highlight both the promise and the limitations of AI-assisted research. AI can search across disciplines, perform calculations and uncover connections that may otherwise take researchers considerably longer to explore. But identifying whether those connections represent genuinely useful science still requires human expertise.
ALSO READ | Musk To Co-Lead US Project On Future Warfare
The Human Role In AI-Powered Science
The rise of AI is therefore changing what it means to be technically skilled in scientific research.
Programming, data processing and complex calculations can increasingly be delegated to AI systems. But Schwartz's argument suggests that this makes scientific judgement, curiosity and the ability to recognise an important question even more valuable.
AI can explore large amounts of scientific territory at unprecedented speed. It can identify patterns, test ideas and build models. But it can still lose context, pursue technically correct but unimportant questions and treat an unproven assumption as a successful conclusion.
The challenge, then, may not be teaching AI to replace scientists. It may be finding better ways for humans and AI to work together, with machines handling increasingly complex technical tasks while researchers provide the context, direction and scientific judgement needed to turn those capabilities into meaningful discoveries.
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.