📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems have achieved near-complete automation of engineering tasks, with benchmarks indicating saturation. However, research activities are still partly dependent on human insight, though this gap may close faster than expected.
Recent AI benchmark data confirms that AI systems can now automate the majority of core engineering tasks involved in AI research, with some benchmarks reaching near-complete saturation. Meanwhile, the automation of AI research itself remains partial, though evidence suggests it could close the gap faster than previously thought. This development signals a significant shift in AI R&D capabilities, with potential implications for the pace and nature of future AI innovation.
Six key benchmarks measuring AI capabilities in core research skills show rapid progress toward saturation. For instance, the CORE-Bench, which tests research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025, with one author declaring it ‘solved.’ This indicates that AI can now handle reproducing research papers with reliability comparable to a competent post-doc, drastically reducing the cost and time of research replication. Similarly, the MLE-Bench, which assesses performance on Kaggle competitions, advanced from 16.9% in October 2024 to 64.4% in February 2026, reaching levels comparable to mid-tier human practitioners. These benchmarks are approaching their measurement limits, suggesting that the automation of engineering tasks is nearing full capability.
In contrast, the automation of research—such as generating novel hypotheses, designing experiments, and interpreting results—remains less certain. Clark’s analysis leaves open whether research itself is fundamentally a form of large-scale engineering, which could mean that ongoing research activities will also become automated at an accelerating pace. The current evidence indicates that the bottleneck may shift from engineering to higher-level creative and strategic tasks, which are still partly human-driven but increasingly within reach of AI systems.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
1000 AI Tools Directory 2026: The Ultimate Guide to AI Tools for Business, Productivity, Content Creation, Marketing, Coding, Design, Research and Automation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
AI Engineering: Building Applications with Foundation Models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
research replication AI tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
AI Workflow Tools for Researchers & Analysts: Automating Literature Reviews, Summaries, and Hypothesis Generation with ChatGPT, Claude, and Perplexity
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Automated Engineering and Residual Research
The rapid automation of engineering tasks in AI research suggests that the process of developing new AI models and infrastructure could soon be dominated by AI systems, reducing costs and accelerating innovation cycles. This shift could fundamentally alter how research teams operate, with fewer human resources needed for routine tasks and more focus on strategic and creative aspects. However, the partial automation of research activities raises questions about the future role of human researchers in hypothesis generation, experimental design, and interpretation, which remain less certain but are likely to evolve quickly.
Understanding these developments is critical for policymakers, research institutions, and industry leaders planning for the next decade. If the trend continues, the pace of AI advancement could accelerate significantly, potentially leading to breakthroughs in areas previously limited by human capacity. Conversely, the transition also raises concerns about the concentration of power and the need for oversight in AI-driven research processes.
Recent Progress in AI Benchmarks and Capabilities
Over the past 18 months, multiple independent benchmarks have shown consistent progress in AI capabilities relevant to research and engineering. The CORE-Bench, measuring research reproduction, reached near-complete automation by late 2025. The MLE-Bench, assessing performance on Kaggle competitions, also approached saturation in early 2026. These benchmarks are designed to evaluate AI’s ability to perform complex, multi-step tasks involved in research and engineering, and their rapid advancement indicates that AI systems are becoming increasingly capable of handling tasks that previously required human expertise.
These developments follow a pattern of steady improvement across various domains, including code generation, kernel design, and infrastructure optimization, as documented in recent research papers and industry reports. The progress suggests that the engineering aspect of AI R&D is nearing full automation, with the residual challenge being the automation of higher-level research activities that involve creativity, hypothesis formulation, and strategic decision-making.
“The pattern across multiple benchmarks indicates that AI is approaching saturation in core engineering tasks, with research automation still progressing but likely to accelerate.”
— Thorsten Meyer
Unresolved Questions About Research Automation
It remains unclear how quickly AI will fully automate the higher-level, creative aspects of research, such as hypothesis generation, experimental design, and interpretation. While engineering tasks are approaching saturation, the structural question of whether research itself is a form of large-scale engineering at scale is still open. The pace at which these activities will become automated depends on future breakthroughs in AI capabilities related to strategic thinking and scientific reasoning, which are still under active development.
Next Steps in Monitoring AI Research and Engineering Progress
In the coming months, researchers and industry observers will closely monitor the progress of emerging benchmarks and capabilities, especially in areas related to research creativity and strategic planning. Further developments may involve the deployment of AI systems in real-world research settings, testing their ability to generate novel insights independently. Additionally, policymakers and institutions will need to consider regulatory and ethical frameworks to manage increasingly autonomous AI research processes.
Expect continued publication of benchmark results, industry reports, and case studies illustrating the evolving role of AI in research and engineering. The next milestone likely involves demonstrating AI’s ability to autonomously propose, execute, and interpret complex scientific experiments, which could redefine the landscape of scientific discovery.
Key Questions
What does the automation of engineering tasks mean for AI research?
It indicates that many routine and complex engineering activities involved in developing AI models can now be handled by AI systems, reducing costs and speeding up development cycles.
Will AI eventually automate all aspects of research?
While engineering tasks are nearing full automation, higher-level research activities involving creativity and strategic thinking remain less certain but are progressing rapidly. The timeline for full automation of research is still uncertain.
What are the risks of highly autonomous AI research systems?
Potential risks include loss of human oversight, ethical concerns, and concentration of power. Proper regulatory frameworks will be essential as AI takes on more autonomous research roles.
How soon could fully autonomous AI research become a reality?
Experts believe it could happen within the next few years, but it depends on breakthroughs in AI reasoning and creativity, which are still under development.
What should research institutions do to prepare for these changes?
Institutions should invest in understanding AI capabilities, develop oversight frameworks, and prepare for shifts in research workflows as automation advances.
Source: ThorstenMeyerAI.com
