🔍 Read the full analysis: The Hard Part Of AI May Be Checking What It Produces on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
AI systems can generate mathematical manuscripts, software changes and contract work faster than people can reliably assess them. The source material cites studies and company analyses indicating longer review waits and gaps in human review, but some figures come from vendors and need to be read with that limitation in mind.
AI-generated work is arriving faster than people can check it, according to figures and examples assembled in an analysis published by ThorstenMeyerAI.com. The pattern spans mathematics, software engineering and contract work: generation has become cheaper, but review still depends heavily on specialist human judgement, leaving organisations to decide what they can trust and use.
The analysis says OpenAI published 722 mathematical manuscripts produced from roughly 4,000 problems, with each result taking about three hours of compute on average. The manuscripts cover 372 problem families. Some proofs have been formally checked in Lean, while OpenAI cautioned that some results not formalized in that system could have issues. The source contrasts this output with the careful verification by five leading mathematicians of an earlier result from the same programme: a proposed counterexample to an old Erdős conjecture.
In software, the analysis cites figures from two commercial analytics providers. Faros AI reported that teams merged 98% more pull requests during high-AI-adoption periods, while review time rose 91%. LinearB, analyzing 8.1 million pull requests across 4,800 organizations, reported that AI-generated changes waited 4.6 times longer for review to begin and were accepted 32.7% of the time, compared with 84.4% for human-written changes. A peer-reviewed 2026 study cited by the source found that 61% of AI-agent pull requests had no human review before they were merged or closed.
The source also points to OpenAI’s partnership with contract-software company Ironclad. It says GPT-6 Astra was trained on real contracting workflows and met 55% of evaluation criteria on average across 11 tasks. That is described as a substantial improvement over an earlier model, but the source does not provide the earlier score. The remaining criteria matter because missed approval requirements or an incorrect jurisdiction clause could make a draft unsuitable for use.
The referee shortage: AI made doing cheap and checking expensive
OpenAI’s model produced a maths result in about three hours of compute. Verifying one earlier result took five of the world’s leading mathematicians. That ratio is the next decade of work: producing is cheap and abundant; trusting is slow, human and scarce.
Some Lean-checked; OpenAI warns unformalized ones “could have issues.” Verification abundance, adjudication scarcity.
Faros AI. LinearB (8.1M PRs): AI changes wait 4.6× longer, accepted 32.7% vs 84.4%.
Real progress. Someone still has to find the other 45% before the work can be used.
Machine output arrives without reasoning a reviewer can interrogate. It looks locally clean and gives no clue where it’s wrong.
A prover confirms the proof proves its statement; tests confirm what tests check. Neither confirms it’s what was needed.
Contracts are signed, designs stamped, papers defended. Responsibility is institutional — you can’t hold a model to it.
of AI-agent pull requests got no human review at all (EASE 2026). Zero-review merges up 31.3% (Faros).
of reviewers deliberately deprioritise AI changes (LinearB). Good machine work waits behind bad.
OpenAI chose which maths families were significant. When referees can’t keep up, the producer’s filter becomes the review.
Budget review hours next to model spend.
Provers, types, tests, policy engines.
Experts only where consequences are high.
Who profits from generation pays for checking.
Keep some production human for learners.
The first automation question was which jobs AI would do. The better one is which jobs AI makes more necessary: the ones that check, adjudicate and take responsibility. Expect a referee premium — senior engineers, auditors, specialist lawyers, reviewing scientists become the binding constraint on how much AI output anyone can use.Accountability — standing behind a result — may be the most durable form of human work there is.
Review Capacity Sets AI’s Practical Limits
If AI can produce more work than qualified people can assess, the practical limit on adoption may be review capacity rather than generation. A team may receive more code, research or contract drafts, but still have to pause before relying on them. That affects how quickly organizations can put AI output into production and how much staff time its use actually saves.
The source describes three possible responses when review falls behind: work may be approved without scrutiny, deliberately deprioritized because it was AI-generated, or filtered by the organization that produced it. Each carries a different risk. Skipped review can let defects through; suspicion can delay useful work; and reliance on a producer’s own selection process may leave readers without independent assessment.
The longer-term concern is who will become the next generation of reviewers. Senior engineers, lawyers and researchers typically develop judgement by doing the underlying work themselves. If entry-level staff mainly supervise machine-generated drafts instead of learning to produce and revise them, organizations could weaken the training pipeline for expert judgement just as demand for review grows. The source frames this as a risk, not an established outcome.
Products Worth Considering
LG UltraGear 32GX850A-B 32-Inch 4K UHD Glossy OLED Gaming Monitor
Logitech MX Mechanical Wireless Illuminated Keyboard Tactile (Graphite)
Logitech MX Keys S Wireless Keyboard, Graphite
As an affiliate, we earn on qualifying purchases.
Three Fields, One Review Bottleneck
The supplied analysis connects developments in mathematics, software and contracting through a distinction between producing an answer and determining whether it is correct, relevant and safe to use. In mathematics, formal tools can test whether a proof follows from its stated assumptions. In software, automated tests can check specified behavior. Neither process alone establishes that the theorem or test captures the actual question someone needed answered.
The article also identifies accountability as a reason human review remains significant. A person signs a contract, an engineer may certify a design, and researchers answer for published work. Those professional and institutional responsibilities do not transfer automatically to a model. The source calls the gap between abundant output and limited expert assessment “verification abundance, adjudication scarcity,” while noting that deciding what a result means can require more than checking its formal validity.
The evidence cited is not all of the same kind. The software figures include analyses by companies that sell code-review tools, a commercial interest readers should keep in mind. The source identifies the 2026 pull-request study as peer-reviewed, but gives no study title or methodology in the supplied material. The OpenAI and Ironclad figures describe a company partnership and an evaluation; they are not presented as an independent assessment of real-world contract outcomes.
What the Cited Metrics Cannot Show
The figures do not establish that AI use alone caused the reported changes in review time, acceptance rates or review practices. The supplied material does not give the methods, time windows or comparison baselines for every metric, and it does not identify the peer-reviewed study by title. Faros AI and LinearB sell tools in the code-review market, so their findings warrant attention alongside their commercial interests.
It is also unclear how representative the OpenAI mathematics output is of the broader field, how many manuscripts were independently assessed, or how many results were formally verified. For GPT-6 Astra, the source gives an average evaluation score but not the benchmark design, the earlier model’s score or evidence of performance in live legal work. The source’s prediction that reviewer shortages could deepen is an argument about future workforce effects, not a measured finding.
More Evidence on Review Outcomes
The next useful evidence would show whether increased AI adoption changes the rate of errors that reach users, how much expert time is spent checking generated work, and whether those outcomes differ by task or organization. For software, that means clearer reporting of review periods, definitions and comparison groups. For mathematics and contracts, it means independent examination of results and evaluation methods rather than output volume or a single average score alone.
Organizations adopting these systems will also need to decide who has authority to approve work and how junior staff gain the experience required for that responsibility. The supplied material does not name a specific upcoming study, benchmark or policy decision. For now, the confirmed picture is a growing supply of AI-generated work alongside evidence of review pressure; the scale and long-term consequences of that pressure remain unsettled.
Key Questions
What is the central development described in the article?
The analysis argues that AI can generate work faster than experts can verify it, with examples from mathematical manuscripts, software changes and contract tasks.
Did OpenAI publish 722 mathematically verified papers?
The source says OpenAI published 722 mathematical manuscripts drawn from about 4,000 problems. It says some results were formally checked in Lean and warns that some unformalized results could have issues; it does not say every manuscript was independently verified.
What do the software review numbers show?
The cited analyses report longer waits for review of AI-generated changes and, in one peer-reviewed study, no human review for 61% of AI-agent pull requests before they were merged or closed. The figures come from different sources and methods, so they should not be treated as directly comparable or proof that AI alone caused the outcomes.
Does the source prove there is already a shortage of human reviewers?
No. It presents metrics suggesting pressure on review workflows and argues that specialist assessment may become a constraint. The supplied material does not establish the size of any reviewer shortage across industries.
What remains unknown about AI-generated work?
The source does not establish how often AI-generated work contains consequential errors, how representative the cited examples are, or whether organizations can expand review capacity without weakening staff training. Those questions need further data and independent evaluation.
Source: ThorstenMeyerAI.com
