Ornith 1.5 Review: Self-Improving Open Weights

Ornith 1.5 is the second release from DeepReinforce, a research startup founded by Dr Jiwei Li - a Stanford PhD previously behind the NLP startup Shannon.AI - that first appeared in June 2026 with Ornith 1.0. Where 1.0 introduced "self-scaffolding" (the model writing its own tool harnesses for a fixed set of human-authored tasks), 1.5 closes the loop entirely: task generation, scaffold construction and solution rollouts are all optimised together, so the model's own training curriculum evolves alongside its capability.

The result, on DeepReinforce's own numbers, is a flagship 397B model that trades blows with Claude Opus 4.8 on agentic coding benchmarks, a 35B MoE model that beats larger dense open models by wide margins on coding, and a 9B model small enough to run on a phone. None of this has been independently verified yet, and there is no published safety documentation at all - both facts this review treats as seriously as the benchmark table.

• Best for: developers who want a genuinely self-hostable, MIT-licensed coding and agentic model, at whichever of the three scales fits their hardware.
• Headline numbers: 86.1 on Terminal-Bench 2.1 and 56.0 on DeepSWE for the 397B flagship - both DeepReinforce-reported, both close to (and on the first, ahead of) Claude Opus 4.8.
• Defining trait: a self-improvement loop that generates its own tasks, scaffolds and RL rollouts, rather than training on a fixed human-curated curriculum.
• Main caveats: self-reported benchmarks with no independent reproduction yet, and no published safety card, CBRN evaluation or limitations section from a lab that is not one of the frontier safety-disclosure labs.

DeepReinforce built its early reputation on reinforcement-learning frameworks for CUDA kernel optimisation before pivoting to general-purpose agentic models. Ornith 1.0, released in June 2026 and built on top of Qwen3.5 and Gemma 4 with additional continued pretraining, introduced the "self-scaffolding" idea: rather than hand-writing the tool harnesses and orchestration logic an agent uses to attempt a coding task, the model learned to write its own scaffolds for a given problem. It shipped across the same three scales DeepReinforce has stuck with since - 9B dense, 35B MoE, and 397B MoE.

Ornith 1.5 keeps that same three-scale structure and the same base-model lineage, but extends the training method itself. Self-scaffolding solved half the problem - letting the model design its own approach to a task. Self-improvement, as DeepReinforce frames it, solves the other half: letting the model decide what task to attempt next, so the training curriculum is no longer bottlenecked by how much human-authored task data the lab can produce. On DeepReinforce's own comparison table, this shows up as a large jump over Ornith 1.0 on exactly the benchmarks you'd expect a better training curriculum to move - DeepSWE goes from 8 to 56 at the 397B scale, and Frontier-Bench v0.1 more than quadruples from 2.7 to 13.5.

Architecturally, DeepReinforce discloses relatively little beyond the parameter counts: a 397B Mixture-of-Experts flagship, a 35B MoE model (internally labelled 35B-A3B, meaning roughly 3 billion parameters activate per token), and a 9B dense model with a quantised Ornith-1.5-9B-Mobile variant built for on-device deployment. All three continue from the Ornith 1.0 checkpoints through further continued pretraining (CPT), mid-training and post-training, rather than being trained from scratch.

The more interesting disclosure is the training method itself. Each cycle runs in three stages, and DeepReinforce is unusually explicit about the maths behind it. Given an environment or codebase, the system first proposes a task pitched near the model's current capability frontier - not so easy it teaches nothing, not so hard it never succeeds. It then generates a scaffold: the tools, decomposition strategy and orchestration needed to attempt that task. Finally, conditioned on both, it produces solution rollouts, which are scored and fed back as a training signal - to the policy itself, but also back into what makes a good task and what makes a good scaffold, so all three stages improve together.

DeepReinforce publishes the reward function behind the task-generation stage: a product of three terms - validity (does the task and scaffold form a well-defined, verifiable problem; if not, reward is hard-gated to zero, regardless of how difficult the task looks), frontier difficulty (rollout success rate is measured directly and rewarded for landing near a target of roughly 20%, so tasks stay meaningfully hard without becoming unsolvable), and novelty (a penalty against generating near-duplicates of previously seen tasks). All three stages - question generation, harness generation and solution rollouts - are optimised jointly using GRPO (Group Relative Policy Optimisation). The diagram below, from DeepReinforce's own release post, illustrates the loop: a task branches into multiple candidate scaffolds, each scaffold produces several solution rollouts, the best-scoring ones generate the reward signal, and that reward flows back through a GRPO update into the next version of the model.

The validity gate is worth dwelling on, because it is the part doing the most work against reward hacking. A model that generates its own training tasks has an obvious failure mode: learning to propose tasks it already knows it can solve, or tasks whose scaffold happens to reward low-effort answers. By making validity a hard, multiplicative gate rather than a soft signal, DeepReinforce is explicitly trying to prevent the generator from being rewarded for gaming its own evaluation - though, as with any self-reported training methodology, the only real test of whether that gate holds up is independent, adversarial scrutiny that has not yet happened for this release.

Agentic coding

This is where DeepReinforce concentrates its evidence. Terminal-Bench 2.1, SWE-bench Verified, SWE-bench Pro and DeepSWE all show meaningful jumps over Ornith 1.0 at every scale, and the 397B model's Terminal-Bench 2.1 score (86.1) is the single number driving most of the "matches Claude Opus 4.8" framing in press coverage. The self-improvement loop is specifically tuned around coding and agentic environments - proposing tasks against a real codebase - so this concentration of strength is exactly what the training method would predict.

Reasoning

On Humanity's Last Exam and GPQA Diamond, Ornith-1.5-397B posts solid but not category-leading scores (44.6 without tools, 56.1 with tools, and 92.8 on GPQA Diamond) - ahead of GLM-5.2 and DeepSeek-V4-Flash-0731 on the with-tools HLE figure, but behind Claude Opus 4.8 on all three. Reasoning is a secondary strength here, not the headline.

Agentic tool use and search

On MCP-Atlas, WideSearch and BrowseComp - benchmarks that test multi-step tool orchestration and web research rather than pure coding - the 397B model is competitive with, and on WideSearch and BrowseComp specifically ahead of, Claude Opus 4.8. This is a plausible knock-on benefit of a training loop built around scaffold generation: a model trained to design its own tool harnesses should, in principle, generalise reasonably well to using tools it didn't design.

Edge and mobile deployment

The 9B model is the most immediately useful release for anyone without serious hardware. DeepReinforce reports it scoring 47.0 on Terminal-Bench 2.1 and 70.6 on SWE-bench Verified - beating the much larger, dense Gemma-4-31B (42.1 and 52.0 respectively) and holding up credibly against Qwen 3.6-35B-A3B on several metrics, despite being a fraction of the size. Paired with the quantised Ornith-1.5-9B-Mobile variant aimed at iPhone and Android, this is squarely positioned as the "runs on what you already own" option in the family.

The chart below is DeepReinforce's own comparison for the 397B flagship against DeepSeek-V4-Flash-0731 (284B), GLM-5.2 (753B), Claude Opus 4.8, and its own predecessor, Ornith-1.0 (397B). It is worth reading benchmark-by-benchmark rather than trusting the headline framing.

• Terminal-Bench 2.1 (Terminus-2 harness): Ornith-1.5-397B 86.1, edging out Claude Opus 4.8's 85.0, GLM-5.2's 81.0, and DeepSeek-V4-Flash-0731's 82.7.
• DeepSWE: Ornith-1.5-397B 56.0 - behind Claude Opus 4.8's 59.0, but well ahead of GLM-5.2 (46.2) and roughly on par with DeepSeek-V4-Flash-0731 (54.4). Notably, Kimi K3 leads all of them here at 67.5.
• Frontier-Bench v0.1 (DeepReinforce's hardest, newest suite): Ornith-1.5-397B scores 13.5 - a large jump over its own predecessor's 2.7, but still well behind Claude Opus 4.8's 21.1. This is the clearest evidence that "matches Opus 4.8" is true for some benchmarks and false for others.
• NL2Repo: 59.5 versus Claude Opus 4.8's 69.7 - another benchmark where Opus 4.8 leads clearly.
• WideSearch and BrowseComp (agentic web research): Ornith-1.5-397B leads Claude Opus 4.8 on both (80.8 vs 72.9, and 86.6 vs 84.3).

At the smaller scales, the pattern holds: Ornith-1.5-35B-A3B clearly beats its similarly-sized peer Qwen3.6-35B-A3B and the dense Gemma-4-31B and Meta Muse Glimmer-30B on most coding and agentic benchmarks (68.5 vs 43.4 and 51.7 on Terminal-Bench 2.1; 79.0 vs 52.0 and 76.0 on SWE-bench Verified), while Ornith-1.5-9B outperforms the larger Gemma-4-31B and holds up against Qwen 3.6-35B-A3B despite being roughly a quarter of the size.

Two honest caveats. First, every number above is DeepReinforce-reported, run on its own harness; no neutral aggregator (Artificial Analysis, LMArena, or similar) had published independently reproduced Ornith-1.5 scores at the time of writing. Second, at least one independent tester has already found a meaningful discrepancy at the smaller scale: a Hacker News commenter running their own DeepSWE evaluation reported Ornith-1.5-35B trailing Qwen3.8-27B by a wide margin (22.0 vs 42.2) - consistent with DeepReinforce's own published 35B DeepSWE figure of 22, but a useful reminder that this specific benchmark stresses real engineering work in a way that's harder to game than more leaderboard-friendly metrics, and that "beats comparable open models" is not true on every single benchmark even within DeepReinforce's own table.

Here the honest answer is short: there isn't one. DeepReinforce's release post and the Hugging Face model cards for all three Ornith-1.5 sizes contain no dedicated safety section, no CBRN (chemical, biological, radiological, nuclear) uplift evaluation, no refusal-behaviour analysis, and no equivalent of the Responsible Scaling Policy / AI Safety Level framework that Anthropic publishes alongside models like Claude Opus 4.8. This is not unusual for a lab of DeepReinforce's size - it mirrors the disclosure gap this site has also flagged for Alibaba's Qwen3.8-27B - but it is a genuine, material gap rather than a clean bill of health, and it should weigh directly against the appeal of a free, capable, MIT-licensed model.

No independent safety body - UK AISI, METR, SaferAI or similar - had published an Ornith-1.5-specific evaluation at the time of writing, and DeepReinforce does not appear on the list of Seoul AI Safety Commitment signatories. If you are considering Ornith 1.5 for anything beyond a personal or sandboxed coding assistant, the sensible default is to assume no safety testing has happened and apply your own review before granting it real-world tool access, file-system permissions, or network access.

Independent early testing has surfaced one specific, publicly reported quirk worth flagging directly: multiple users have reported that Ornith models will, in some contexts, identify themselves as Claude when asked what model they are. Independent commentary attributes this to training-data contamination - Ornith 1.5's post-training data likely includes transcripts or synthetic data generated by or referencing Claude - rather than to any form of model or weight theft, and there is no evidence it represents a security risk in itself. It is, however, a useful signal about how much of this model's behaviour has been shaped by data DeepReinforce did not author from scratch, and a reminder to verify model identity independently in any pipeline where that matters.

Beyond that specific quirk, the general agentic-deployment guidance for any capable open-weight model applies with extra force here given the absence of published safety testing: run it with scoped, minimal permissions, keep a human in the loop for irreversible actions, and log everything you can review. DeepReinforce's own self-improvement loop notably runs with network access disabled during RL training rollouts - a sensible containment choice for the training environment, but not a substitute for your own containment when you deploy the resulting model with real tool access.

Independent hands-on testing of the 35B model paints a more mixed picture than the benchmark table alone suggests - which is exactly the point of checking. Running Ornith-1.5-35B MoE at 4-bit quantisation on Apple Silicon via MLX, one independent tester found vision tasks (reading an invoice) worked reliably at around 98% accuracy, and that the model produced a working tower-defence game and a spreadsheet with live formulas from a single one-shot prompt. But a Three.js first-person-shooter prompt broke every single time on the same missing character - a small, specific failure that no aggregate benchmark score would ever surface.

The lesson generalises beyond this one test. Benchmark suites measure success or failure on a fixed task distribution; real usage exposes the long tail of small formatting quirks, edge cases in specific frameworks, and prompt sensitivities that never show up in an aggregate score. For a model whose entire training philosophy is built around generating its own task distribution, that gap between benchmark and real-world behaviour is worth watching particularly closely - a self-generated curriculum is only as good as the validity and novelty checks constraining it, and those checks are themselves untested by anyone outside DeepReinforce.

All three Ornith-1.5 sizes are published on Hugging Face under the MIT licence - no revenue threshold, no separate commercial agreement required, in contrast to some open-weight releases (such as Alibaba's largest Qwen3.8 checkpoint) that carry usage-based licensing triggers. "Free" here means the licence fee is zero; it does not mean the compute is free, and the compute requirement varies enormously across the three sizes.

• Ornith-1.5-9B / 9B-Mobile: the most accessible tier - runs on a single consumer GPU, a mid-range Apple Silicon Mac, or (via the quantised Mobile variant) directly on recent iPhone and Android hardware. GGUF and MLX builds are available for local inference tools like Ollama, LM Studio and llama.cpp.
• Ornith-1.5-35B-A3B: despite the 35B total parameter count, only around 3B activate per token, which is what let one independent tester run 4-bit quantisation comfortably on a Mac via MLX. GGUF, FP8 and NVFP4 builds are also published for GPU deployment.
• Ornith-1.5-397B: a genuinely large MoE model. FP8 and NVFP4 quantised builds reduce the footprint somewhat, but running this tier at any useful throughput realistically requires multi-GPU, datacentre-class infrastructure - the same category of hardware needed for comparable flagship open models like DeepSeek V4 Pro or GLM 5.3. At this scale, "free" mostly means avoiding a per-token API bill, not avoiding infrastructure spend - and for teams without that hardware already, a hosted third-party endpoint (once one exists) or a comparable API-based frontier model may work out cheaper in pounds (£) or dollars ($) than provisioning and running the 397B checkpoint yourself.

No official DeepReinforce-hosted API existed for any Ornith-1.5 size at the time of writing; self-hosting via Hugging Face, or waiting for a third-party inference host to list it, are the current options.

• Self-reported benchmarks, unverified: every figure in this article is DeepReinforce's own, run on its own harness; no neutral aggregator had reproduced the 397B's scores at the time of writing.
• "Matches Opus 4.8" is benchmark-specific, not blanket: true on Terminal-Bench 2.1, WideSearch and BrowseComp; false on DeepSWE, Frontier-Bench v0.1 and NL2Repo, where Claude Opus 4.8 leads clearly.
• No published safety documentation: no system card, no CBRN evaluation, no independent safety assessment - a real gap for a model capable enough to be given agentic tool access.
• 397B is not lightweight: despite the free licence, running the flagship at useful speed needs datacentre-class multi-GPU hardware.
• Independent testing has already found gaps: a Three.js rendering failure and a DeepSWE shortfall against Qwen3.8-27B at the 35B scale, both from hands-on testers rather than DeepReinforce's own table.
• Self-generated training data is an unaudited process: the validity, difficulty and novelty checks that gate the self-improvement loop are DeepReinforce's own design, with no independent audit yet of how well they actually resist reward hacking at scale.
• No official hosted API at any size - self-hosting is currently the only route to using these models.

Against fellow open-weight flagships from the same August 2026 window - DeepSeek V4 Pro and GLM 5.3 - Ornith-1.5-397B is competitive on coding benchmarks and MIT-licensed, which is a cleaner licence than some rivals carry, but it is the newest and least independently scrutinised of the three. Against Qwen3.8-27B, the comparison cuts both ways: Ornith's 9B model is smaller and more mobile-friendly, but at least one independent DeepSWE test found the mid-sized 35B trailing Qwen3.8-27B by a wide margin, a useful counterweight to DeepReinforce's own more favourable framing. Against the dense Meta Muse Glimmer-30B, Ornith-1.5-35B leads clearly on DeepReinforce's own coding and agentic benchmarks, consistent with the general pattern that well-trained MoE models tend to out-punch similarly-sized dense ones on these tasks.

Against Claude Opus 4.8, the honest summary is: on a handful of specific agentic and search benchmarks, Ornith-1.5-397B edges ahead; on several coding and reasoning benchmarks, and on the entire dimension of published safety testing, Opus 4.8 leads clearly. The free MIT licence and self-hostability are Ornith's real advantages over any closed, API-only frontier model - not an unqualified capability win.

Worth trying now if you want a genuinely free, MIT-licensed, self-hostable coding and agentic model, and your hardware matches one of the three tiers - a phone or consumer GPU for the 9B, a single high-end GPU or well-specced Mac for the 35B MoE, or a multi-GPU setup for the 397B flagship. It is a strong pick for personal projects, research, and sandboxed agentic experimentation where you control the blast radius.

Worth waiting on if you need independently verified benchmark numbers before committing production workloads, or a published safety evaluation before granting the model real-world tool, file-system or network access - neither exists yet for Ornith 1.5. For those cases, a model with a published system card and a formal safety-disclosure framework, such as Claude Opus 4.8, remains the more accountable choice even at a higher price.

Ornith 1.5 is a genuinely interesting piece of engineering - a training loop where the model itself decides what to practise next, gated by explicit validity, difficulty and novelty checks designed to keep that self-generated curriculum honest. On DeepReinforce's own numbers it delivers real capability, particularly in agentic coding, at a free MIT licence that beats the terms attached to several larger rivals.

None of that changes two facts that matter just as much as the benchmark table: no outside lab has yet reproduced DeepReinforce's numbers, and no safety evaluation has been published for a model family capable enough to be handed real agentic tool access. Download it, benchmark it against your own workload rather than DeepReinforce's, and apply your own review before trusting it with anything that matters - the same standard this site applies to every open-weight release without a published safety framework.