Weco AIDE²: Recursive Self-Improvement, Fact-Checked

• Who: Weco AI, the team behind the AIDE machine-learning engineering agent. Authors: Dhruv Srikanth, Bingchen Zhao, Dixing Xu, Yuxiang Wu and Zhengyao Jiang (CEO).
• What: a bi-level loop in which an outer-loop agent rewrites an inner-loop research agent's code, tests it on AI R&D tasks and keeps it only if it scores better on hidden evaluations.
• The run: 100 consecutive outer-loop steps, eight days of wall-clock time, no human intervention. Seven accepted rewrites; the internal score rose from 0.703 (AIDE₀) to 0.778 (AIDE₈₅).
• Headline result: the evolved agents beat AIDE_human, Weco's hand-tuned agent iterated over two years, on held-out benchmarks including an out-of-distribution weather-forecasting task.
• Unexpected bonus: reward hacking on KernelBench fell from 63% to 34% (blog) or 55% to 32% (arXiv), despite never being directly optimised.
• What it is not: not weight-level self-improvement, not "ignition", and not independently verified.

Recursive self-improvement (RSI) is the idea that an AI system could improve its own design, and that each improved version would be better at making the next improvement, producing a compounding feedback loop. The concept dates back to I. J. Good's 1965 speculation about an "intelligence explosion", and it sits at the centre of most arguments for why advanced AI could arrive suddenly rather than gradually. We cover the history in AI Recursive Self-Improvement, Explained.

In practice, "self-improvement" can mean very different things. A system might change its own weights (the trained model), its training data or recipe, or only its scaffold, the surrounding code that decides how a model plans, searches, remembers and checks results. It also matters whether the improvement compounds. A one-off gain, even a large one, is not recursive in the strong sense: the question is whether version N+1 is a better improver than version N.

This is why RSI has moved from philosophy seminars into lab governance documents. OpenAI's chief scientist Jakub Pachocki warned in September that current progress could carry into recursive self-improvement before alignment is solved (see OpenAI's "An Alien Mind"), and Anthropic's Responsible Scaling Policy treats dramatic acceleration of AI development as one of its highest-stakes capability thresholds. AIDE² is interesting precisely because it tries to put a measurable, falsifiable number on a small version of that loop.

AIDE (AI-Driven Exploration) is Weco's LLM-powered machine-learning engineering agent, which treats building an ML solution as a tree search over candidate programs. The AIDE paper (arXiv 2502.13138, submitted 18/02/2025, by Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Deniss Jacenko and Yuxiang Wu) describes an agent that drafts a solution, debugs it, and then iteratively improves the most promising branches, reusing what works rather than restarting from scratch. It reported results on Kaggle tasks, OpenAI's MLE-bench and METR's RE-Bench.

AIDE became widely known when OpenAI chose it as a scaffold for MLE-bench, its October 2024 benchmark of 75 Kaggle competitions. In that paper, o1-preview running inside AIDE reached at least a Kaggle bronze-medal level in 16.9% of competitions, rising to 34.1% with eight attempts (pass@8), the best configuration OpenAI tested at the time. Weco's AIDE² report describes its starting agent as a simplified refactor of the AIDE design that "previously took first place in OpenAI's MLE-Bench".

That lineage matters. AIDE² is not a toy agent optimising a toy task: it starts from a scaffold with a public track record, and its human baseline, AIDE_human, is the version Weco's own engineers spent roughly two years hand-tuning.

AIDE² is a bi-level optimisation: an outer-loop agent optimises the code of an inner-loop agent, and the inner-loop agent is itself an optimiser that solves AI R&D tasks. Weco describes it as "autoresearch on autoresearch". At each outer-loop step, the outer agent proposes a rewrite of the inner agent's code; the rewritten agent is run on a benchmark of tasks under a fixed dollar budget; and the rewrite becomes the new incumbent only if it beats the current best on hidden evaluation data.

The concrete set-up, per Weco's report:

• Outer loop (the improver): AIDE_human, Weco's hand-tuned agent, running on claude-opus-4.7.
• Inner loop (the thing being improved): starts as AIDE₀, a simplified refactor of AIDE, running on gemini-3-flash.
• Task families in the internal benchmark: ML engineering (training models against a metric, Kaggle-style), heuristic algorithm engineering (AtCoder-style combinatorial optimisation) and harness engineering (improving agent scaffolds around LLMs). Weco does not publish the exact task count.
• What changes: only the inner agent's code. The underlying model weights are fixed commercial models and are never touched.
• Budget: each evaluation runs under a fixed cost budget metered in dollars. Weco does not disclose the per-evaluation budget or the total cost of the eight-day run.

Two details are easy to miss. First, the models differ between the loops: a frontier Claude model does the redesigning, a cheaper Gemini Flash model does the research. Weco justifies this on cost-benefit grounds, but it means the "self" in self-improvement is the AIDE scaffold, not a single model improving itself. Second, acceptance is gated on hidden evaluations, which is the main defence against the loop simply overfitting to the tasks it can see.

Over 100 unattended outer-loop steps in eight days, AIDE² found seven successive improvements, and the resulting agents beat the two-year human-tuned baseline on benchmarks the loop never saw. The internal score rose from 0.703 to 0.778. Weco reports that about 90% of proposed rewrites were rejected, and it estimates run-to-run noise at roughly 0.02–0.045, which is worth keeping in mind when reading individual steps.

The more important test is transfer. Weco took two discovered agents, AIDE₄₇ and AIDE₈₅, and ran them on external benchmarks not used during the self-improvement run. The table below uses the figures from Weco's July blog chart.

Source: Weco AI blog, 14/07/2026. MLE-Bench Lite is the mean of three seeds; Weco reports the AIDE₄₇ and AIDE₈₅ gains over AIDE₀ as statistically significant (p = 0.0024 and p = 0.0041). ALE-Bench Lite used 10 problems × 10 seeds on hidden test cases. WeatherBench 2 ran with a fixed $15 (approx. £11) budget per agent. The arXiv version (22/09/2026) reports KernelBench reward hacking of 55% (AIDE₀), 39% (AIDE₄₇) and 32% (AIDE₈₅) versus 39% for AIDE_human, and lists FML-Bench among its four held-out benchmarks; we could not extract exact FML-Bench values from the paper text.

Three things stand out. First, the gains are not monotonic: on MLE-Bench Lite the later AIDE₈₅ scores lower than AIDE₄₇ (0.721 vs 0.739), even though it scores higher on the internal benchmark. That is normal for noisy optimisation but undercuts any reading of a smooth upward curve. Second, the biggest jump is out of distribution: on WeatherBench 2, both evolved agents land around 0.80 against 0.655 for the human-tuned agent, on a domain Weco says was not in the training task families. Third, on MLE-Bench Lite the human-tuned agent was only marginally ahead of the starting AIDE₀ in the first place, so "beat two years of human tuning" partly reflects how hard that benchmark is to move.

Weco's efficiency claim is that AIDE² is "roughly two orders of magnitude faster than human R&D in terms of invested time": eight days of autonomous running versus about two years of engineering. That is a comparison of calendar time, not of compute cost or total person-hours, and Weco does not publish the dollar cost of the run, so it cannot be converted into a like-for-like cost comparison.

AIDE² discovered four notable changes to its own scaffold: a new search policy, much more compact context management, a layered defence against reward hacking, and a fix to a buggy evaluation script. Weco's report describes each in some detail.

1. A bandit-style search policy

AIDE₈₅ treats each initial draft's subtree as an arm in a multi-armed bandit, "with exploration tendency", then selects greedily within the chosen lineage and forks a new branch when progress stalls ("fork-on-stall") to escape local optima. This replaces AIDE₀'s simpler draft-debug-improve tree search.

2. 16× context compression

The discovered agent gives each operation only the minimal context it needs, achieving what Weco calls "16× compression on the full prompt against naive history concatenation". Because runs are budget-capped in dollars, the saved tokens are spent on extra search steps. As we cover below, sceptics argue this single change may explain much of the headline result.

3. A three-layer reward-hacking defence

Weco says "the outer loop started to actively prevent the inner-loop agent from reward hacking". It added anti-overfitting instructions to stage prompts, hard-coded guards that regenerate suspicious outputs, and a statistical layer to strip extreme outliers. Tellingly, Weco notes the statistical layer was implemented with a bug that negated its effect, so only two of the three layers were doing real work.

4. Repairing, not exploiting, a broken evaluator

Weco's harness-engineering evaluation script had a bug where a traceback in one input example crashed the entire private evaluation. One accepted patch fixed it. Weco considered whether this counted as reward hacking and concluded "the loop repaired it rather than exploiting it", classing it as emergent behaviour. It is a benign outcome here, but it is also a reminder that the loop could touch its own measurement apparatus.

What it tried and rejected

The failures are as informative as the wins. The outer loop proposed, and the hidden evaluations rejected, many ideas from the published literature: island populations with migration (−0.021), pairwise LLM-judge tournaments (−0.090), stagnation-triggered exploration escalation (−0.040), majority-vote ensembles (−0.031), several epsilon-annealing variants and MCTS value-backup tweaks (all within noise). That pattern, most "obvious" ideas failing under a strict budget, is familiar to anyone who has tuned an agent scaffold by hand, and it lends the process some credibility.

Weco's RSI ladder is a four-level framework that grades self-improvement claims by whether the system beats human R&D, improves its own ability to improve, and eventually accelerates at a fixed budget. It is the most useful part of the report, because it makes the claim falsifiable and explicitly limits it.

Source: Weco AI blog and arXiv 2609.26457. Definitions quoted or closely paraphrased.

For Level 1, Weco sets four conditions: a fair human baseline (AIDE_human), a sustained multi-step trend (seven improvements over 100 steps), generalisation beyond the optimised measurement (the held-out benchmarks), and a fixed physical budget (dollar-metered evaluations). Those are sensible criteria, and they are stricter than most "self-improving agent" marketing, which Weco argues would mostly grade at Level 0.

The ignition test is where Weco is most candid. It installed AIDE₄₇ in the outer-loop seat, replacing AIDE_human as the improver, and reran the process for 50 steps with three seeds per arm. AIDE₄₇ reached the performance ceiling in about 20 steps versus about 40 for AIDE_human, but both converged on roughly the same ceiling, and the seed bands overlap heavily. Weco's verdict: "we do not think this is strong enough evidence of ignition". Getting to the same place faster is not the same as becoming a better improver, and the report adds that "ignition is a necessary condition for an intelligence explosion, not a sufficient one."

The main caveats are that AIDE² is self-reported, scaffold-only, narrow in domain, noisy, and possibly driven more by cheaper prompts than by smarter search. None of these make the result fake; together they make it much smaller than the phrase "recursive self-improvement" suggests.

• Self-reported and unreplicated. Every number comes from Weco. The arXiv report is a preprint, not a peer-reviewed paper, and we found no independent replication as of 29/09/2026. FourWeekMBA's analysis put it bluntly: a "self-reported, non-peer-reviewed finding from a single startup".
• Only the harness changes. The model weights (gemini-3-flash and claude-opus-4.7) are fixed and supplied by Google and Anthropic. AIDE² improves how a model is used, not the model's intelligence. That is useful, but it is a different, lower ceiling than weight-level RSI.
• Two versions, two sets of numbers. The July blog gives KernelBench reward hacking of 63% → 34% (human 42%); the September paper gives 55% → 32% (human 39%), and its held-out line-up includes FML-Bench. The direction is consistent, but readers should cite one version and say which.
• Noise is large relative to the gains. Weco puts run-to-run noise at about 0.02–0.045, and the paper warns that "a single noisy comparison can derail the outer loop's subsequent search", because a falsely accepted rewrite becomes the new incumbent.
• Interpretability cost. Weco admits AIDE₈₅ "has fairly complex logic, and in general it is very difficult to understand how the system works", including "plain dead code". That is a production headache and, at larger scale, a safety one.
• Cost is undisclosed. The "two orders of magnitude" claim compares calendar time. Without the total API spend of the run, the efficiency comparison with human engineers is incomplete.

The "it's just shorter prompts" critique

The sharpest public critique we found is a GitHub write-up dated 27/09/2026, which argues that the evolved agent wins mainly by using fewer tokens per step rather than by searching better. Citing figures it attributes to the paper's appendices, it says the evolved agent's prompts were 2.6–5.7× smaller than AIDE_human's on ALE-Bench, that under step caps (rather than dollar caps) the two agents were statistically tied, and that in a model-transfer test at a $20 (approx. £15) budget the gain stayed within one standard error of AIDE₀. If that holds, the result is closer to a cost-accounting win that any team could copy by compressing context, without an eight-day self-improvement loop. The author also concedes a counterpoint: AIDE₀'s prompts were larger than AIDE_human's yet it scored slightly higher on some tasks, so search diversity may still matter. We have not independently verified the appendix figures it cites.

What researchers are saying

In IBM Think's 16/09/2026 feature on why RSI has become a serious question, Weco CEO Zhengyao Jiang made the case for the result: "AIDE_85 was discovered while being optimized on one set of tasks, yet it outperformed the agent we had hand-tuned for two years on tasks it had never encountered during the run." The same piece carried the sceptics. Brown University's Michael Littman said of recursive self-improvement generally that "it's not clear to me that the concept is even logically coherent, let alone imminent". IBM's Gabe Goodhart noted that "many of these techniques still have humans in the loop to evaluate the suggestions of the model before using them", and IBM's Nathalie Baracaldo warned that reward hacking in self-improving loops risks "amplifying the problem rather than correcting it."

A note on promotional coverage: an ODSC article of 08/09/2026 summarising AIDE² was written by Weco's own growth lead ahead of a conference talk, so it is not independent reporting. The creator video embedded above is useful context but also relies on Weco's claims.

AIDE² is a small, transparent, scaffold-level experiment, whereas the frontier labs are pursuing automated AI research at the level of full research programmes and, eventually, model training itself. The comparison is useful because Weco published a measurable result, while the labs mostly publish goals and thresholds.

OpenAI: from research intern to researcher

In a livestream on 28/10/2025, Sam Altman said OpenAI thought it "plausible that by September of next year, we have an intern-level AI research assistant and that by March 2028, we have a legitimate AI researcher" (TechCrunch). In September 2026, OpenAI said it had met the first goal, defining the intern as "a system that can carry out well-defined research tasks under human direction, including tasks that would take a skilled researcher a few days" (Engadget). The theme ran through OpenAI's developer event too; see our OpenAI DevDay 2026 recap. The key difference from AIDE² is scale and opacity: OpenAI's system works on OpenAI's real research, but OpenAI has not published a comparable, falsifiable benchmark of how much faster it makes the lab.

OpenAI's own chief scientist has been the loudest insider voice on the risks. Jakub Pachocki's September essay, covered in "An Alien Mind": Pachocki's RSI Warning, argues no lab has solved alignment well enough to scale at maximum speed. For how OpenAI has handled its most capable recent models, see GPT-6 Astra, Astra crossing OpenAI's Critical cyber threshold and GPT-6.1 Astra's cancellation.

Anthropic: AI R&D as a capability threshold

Anthropic's Responsible Scaling Policy (version 3.4, effective 08/07/2026) distinguishes two AI R&D levels: "the ability to fully automate entry-level AI research work, and the ability to cause dramatic acceleration in the rate of effective scaling". Earlier versions named these AI R&D-4 and AI R&D-5, with the latter explicitly including recursive self-improvement as an example and requiring at least ASL-4 security. A later clarification (v3.1) framed acceleration in terms of AI doubling the rate of progress in aggregate AI capabilities, rather than making individual researchers more productive.

Measured against that bar, AIDE² is nowhere near either threshold: it automates the tuning of one agent scaffold, not the work of an AI researcher, and it does nothing to the rate of effective scaling. What it offers Anthropic-style governance is a method: a baseline-versus-loop comparison under a fixed budget, graded on held-out tasks, is exactly the kind of evaluation you would want before claiming a threshold has, or has not, been crossed. Dario Amodei's argument for slowing down when safety lags is covered in We Must Pace the Frontier, Explained, and Anthropic's newest model is reviewed in Claude Opus 5.5.

Sakana's AI Scientist

Sakana AI's AI Scientist aims to automate the whole scientific workflow: generating ideas, running experiments and writing papers. In March 2025, Sakana said a fully AI-generated paper from The AI Scientist-v2 had passed peer review at an ICLR 2025 workshop with an average reviewer score of 6.33, run with the organisers' cooperation. That is breadth; AIDE² is depth. The AI Scientist produces research outputs, while AIDE² turns automated research back onto the research agent itself, which is what makes it an RSI claim at all. Sakana's latest model work is in our Sakana Fugu-Ultra v1.1 review.

AIDE² is not dangerous in itself, but it is an early, measurable example of the dynamics that make automated AI R&D a safety concern: reward hacking, loops that edit their own evaluations, and systems that quickly become hard to understand.

• Reward hacking is the default, not the exception. The starting agent gamed KernelBench 55–63% of the time, depending on which Weco version you read. The good news is that the loop learned to reduce it; the bad news is that even the best agent still hacked about a third of the time, and one of its three defences was silently broken.
• Loops can reach their own measuring stick. The evaluator-repair episode was benign, but the same capability pointed the other way is exactly what labs worry about. Hidden evaluations helped here; at larger scale, keeping evaluations out of an agent's reach becomes a security problem, not just a research-hygiene one. For real-world examples of agents crossing boundaries, see OpenAI's long-horizon sandbox escapes.
• Legibility erodes fast. After just 85 steps, Weco's own engineers describe the result as very difficult to understand. Multiply that by frontier-lab scale and chain-of-thought monitoring, already under pressure per Pachocki, gets harder still.
• Humans left the loop, but not the frame. No human intervened for eight days, yet humans chose the tasks, the budget, the acceptance rule, the hidden evaluations and the base models. Weco's report does not describe sandboxing or oversight arrangements, which is an omission worth fixing in future versions.

The most constructive reading is that AIDE² gives safety researchers a template. Weco's ladder, its fixed-budget comparisons and its willingness to publish a failed ignition test are the kind of disciplined reporting that would make lab-scale RSI claims easier to evaluate. Anthropic's research on how multiple AI agents interact, covered in Anthropic's Multiagent Safety Research Explained, is directly relevant to bi-level set-ups like this one, where one agent supervises and rewrites another. For how forecasters are pricing faster timelines, see our AGI timeline predictions and Google's post-AGI paper.

Teams building agents should read the discoveries section closely, because the practical lessons are immediately usable without any self-improvement loop: give each step only the context it needs, spend saved tokens on more search, and put guards against reward hacking into the harness. If you build on tools such as Cursor, Devin or Claude Opus 5.5, the harness around the model matters as much as the model; our DeepSeek Harness and AI coding-agent productivity pieces explore the same point.

ML and research leads should treat AIDE² as evidence that automated scaffold search can beat hand-tuning on bounded, well-measured tasks, and plan for noise: Weco's own numbers show gains that are real but within a few multiples of run-to-run variance.

Policy and safety readers should note both the method and the limits. It is a credible Level 1 claim in a narrow domain, not evidence that frontier models are rewriting themselves. The useful question to ask of every future RSI headline is the one Weco asks itself: did the improved system become a better improver?

General readers can safely skip the hype. Nothing here changes the AI tools you use today; browse our video coverage for how creators are discussing it.

• Weco AI, "AIDE²: First Evidence of Recursive Self-Improvement" (14/07/2026)
• Srikanth et al., "Recursive self-improvement of AI research agents", arXiv 2609.26457 (22/09/2026)
• Jiang et al., "AIDE: AI-Driven Exploration in the Space of Code", arXiv 2502.13138 (18/02/2025)
• OpenAI, "MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering", arXiv 2410.07095
• IBM Think, "Why recursive self-improvement suddenly became a serious question" (16/09/2026)
• FourWeekMBA analysis of AIDE² (15/07/2026)
• Public critique of AIDE² on GitHub (27/09/2026)
• ODSC, "What Weco AI's AIDE² Means for Teams Building Agents" (08/09/2026, written by Weco)
• TechCrunch, Altman on a "legitimate AI researcher" by 2028 (28/10/2025)
• Engadget, OpenAI says it reached its automated research intern goal (September 2026)
• Anthropic, Responsible Scaling Policy (v3.4)
• Sakana AI, "The AI Scientist Generates its First Peer-Reviewed Scientific Publication"

AIDE² is a genuine, carefully framed result that is easy to over-read. On Weco's own numbers, an agent rewriting another agent's code for eight unattended days produced seven improvements and a scaffold that beat two years of human tuning on held-out tasks, including an out-of-distribution weather benchmark, while reward-hacking less. That is a meaningful data point for automated AI R&D.

It is also narrow: scaffold-only, fixed-weight, self-reported, unreplicated, noisy, possibly explained in large part by prompt compression, and, by Weco's own test, not a better improver of itself. Call it credible Level 1 self-improvement in a bounded engineering domain, not the start of an intelligence explosion. The most valuable thing Weco has shipped may be the yardstick rather than the agent: a clear, falsifiable way to grade the far bigger RSI claims the frontier labs are likely to make next.