📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Recent evidence indicates AI systems are now capable of automating large parts of AI engineering, with research tasks remaining the residual challenge. This shift could transform AI R&D practices and institutional approaches.
Recent benchmarks and expert analyses confirm that AI systems are now capable of automating the core engineering tasks involved in AI development, with research activities remaining the residual challenge. This development signifies a potential shift in how AI R&D is conducted, impacting institutions, resource allocation, and innovation pace.
Multiple independent benchmarks, including CORE-Bench and MLE-Bench, show AI systems reaching near-complete automation of core engineering skills such as reproducing research and competing in Kaggle competitions. For example, CORE-Bench, which measures research reproduction, reached 95.5% reliability in December 2025, with the author of the benchmark declaring it ‘solved.’ Similarly, AI performance on Kaggle competitions has improved to a level where it can achieve bronze-medal performance on two-thirds of contests, matching mid-tier human practitioners.
These advances are not isolated; they follow a pattern of rapid progress across different skill domains, with models reaching saturation or measurement limits within 16 to 20 months. Experts like Thorsten Meyer interpret these trends as evidence that the bottleneck for AI R&D has shifted from engineering to research itself, which remains less automatable and more complex in its creative and investigative aspects.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI Tools for Finance and Accounting Professionals: Automate Tasks, Save Hours, Work Smarter
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
AI engineering development software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
automated AI research platforms
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
AI Kaggle competition tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications for AI R&D and Institutional Strategies
This shift indicates that AI systems can now handle substantial portions of engineering tasks, such as reproducing experiments and optimizing models, at a level comparable to or exceeding human capabilities. As a result, institutions may need to reconsider resource allocation, focusing more on innovative research activities that remain less automatable. This could accelerate AI development cycles and influence competitive dynamics across the industry.
Recent Advances in AI Engineering Capabilities
Over the past 18 months, multiple benchmarks have demonstrated rapid progress in AI’s ability to automate core engineering tasks. For example, the CORE-Bench, which tests research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025. Similarly, Kaggle competition performance has surged, with models achieving mid-tier human performance. These developments follow a pattern of saturation across different technical domains, suggesting a structural shift in AI R&D capabilities.
“The pattern across multiple benchmarks indicates that AI can now automate vast swaths of AI engineering, with research remaining the residual challenge.”
— Thorsten Meyer
Unresolved Questions About Research Automation
It remains unclear how much of AI research itself can be automated, given its inherently creative and investigative nature. While engineering tasks are approaching full automation, the structural question of whether research can be scaled similarly is still open. Experts suggest the residual may be less binding than initially thought, but definitive proof is lacking.
Next Steps in AI R&D and Benchmark Development
Researchers and institutions will likely focus on further benchmarking the limits of research automation, exploring whether AI can handle more creative or hypothesis-driven tasks. Additionally, the development of new benchmarks and evaluation standards will be crucial to understand the evolving capabilities of AI systems and to guide strategic planning in AI R&D.
Key Questions
What does automation of engineering mean for AI development?
It means that many core tasks involved in building, reproducing, and optimizing AI models can now be handled by AI systems, potentially reducing time and resource costs.
Will AI research itself become automated?
This remains uncertain. While engineering tasks are nearing full automation, the creative and investigative aspects of research are still less understood in terms of automation potential.
How might institutions adapt to these developments?
Institutions may shift focus toward more innovative, hypothesis-driven research activities, and reconsider resource allocation towards tasks less automatable.
Are there risks associated with this automation trend?
Potential risks include over-reliance on AI for research, challenges in ensuring quality and safety, and shifts in competitive advantage that could impact employment and innovation ecosystems.
Source: ThorstenMeyerAI.com
