AI Tools Review
Can a Coding Agent Be a Robot Brain? The Evidence

Insights

Can a Coding Agent Be a Robot Brain? The Evidence

AI Tools Review Editorial Team19 November 2026

    Quick Answer:

    Yes, a general coding agent can already control a robot without robot-specific training, but only slowly. The Stanford study behind the claim is VIA (Visual Interface Agent, arXiv 2607.11119): with Codex running GPT-6 Astra, a Stretch 4 mobile robot completed 24 of 25 real-world trials across five household-style tasks, with no fine-tuning. Independent work from NVIDIA, Notre Dame and Artificial Analysis broadly confirms the capability and the catch: NVIDIA measured 18.56 seconds per decision against 46.96 milliseconds for a specialised robot model, and agentic runs cost up to £27.66 ($36.40) per episode. The approach suits slow, forgiving tasks today. It does not replace vision-language-action models for fast or dexterous work.

    The idea sounds like a party trick: take the same agent that fixes your failing tests, point it at a robot arm instead of a repository, and watch it fold a towel. Over the past four months a run of papers has shown that this works more often than most roboticists expected, and OpenAI has now reorganised its robotics effort around the premise. This piece identifies the Stanford study at the centre of the discussion, reports what it measured, sets it against four other primary sources, and takes the counter-arguments seriously. The short version is that the capability is real and the speed is not there yet.

    24 / 25

    Real-world trials passed by VIA

    Codex with GPT-6 Astra, Stretch 4 robot

    74.8%

    RoboLab-120 success, agentic Astra

    Best listed specialist policy: 51.0%

    18.56 s

    Astra latency per inference call

    pi0.5 VLA: 46.96 ms

    ~100x

    Slower than a human

    Artificial Analysis, tabletop tasks

    Source: Stanford VIA paper (arXiv 2607.11119v2); NVIDIA Astranomy report, 14/10/2026; Artificial Analysis AA-Robotics v0.1, 14/10/2026

    Nate B Jones, 18/11/2026: the Bitter Lesson applied to robotics, Stanford's VIA project and OpenAI's robotics reorganisation.

    Executive Summary

    • The study: VIA: Visual Interface Agent for Robot Control, from Stanford University, lets off-the-shelf agents (Claude Code and Codex) drive a robot through a visual interface and a handful of MCP tools. No robot training, no fine-tuning, no privileged simulator state.
    • Real-world result: 24 of 25 trials (96%) on five tasks with a Hello Robot Stretch 4, using Codex with GPT-6 Astra. Five trials per task.
    • Simulation result: overall success across six tabletop tasks ranged from 38.3% (Codex with GPT-5.6 Terra) to 100% (Codex with GPT-6 Astra). Performance rose with model size in both agent families.
    • Independent confirmation: NVIDIA's Astranomy report puts GPT-6 Astra inside Codex at 74.8% on the 120-task RoboLab-120 simulation benchmark, against 51.0% for the best specialised policy shown on its leaderboard chart and 28.0% for Physical Intelligence's pi0.5.
    • The catch: inference took 94.9% to 97.8% of estimated episode time in NVIDIA's runs. Artificial Analysis found frontier models took about 30 minutes for tabletop tasks a person finishes in about 20 seconds.
    • Cost: from about £1.49 ($1.96) to £27.66 ($36.40) per successful episode depending on configuration.
    • Industry reaction: Fortune reported on 13/11/2026 that OpenAI has dissolved its stand-alone robotics team and moved the researchers into its main research and applied groups.
    • What is unproven: real-time control, contact-rich assembly at speed, dexterous hands, operation near people, and anything beyond small sample sizes.

    Dollar figures are as published in the sources. Sterling conversions use $1 = £0.76, the rate on 19/11/2026.

    Which Stanford Study Is This?

    Nate B Jones's video description refers to "a Stanford study" and, in its chapter list, to "Stanford's VIA", "24 of 25 trials without robot training" and a "wipe test" in which the agent moves the gripper to check its own work. Those details match one paper exactly, so we can identify it with confidence: VIA: Visual Interface Agent for Robot Control, by Hengyuan Hu, Jensen Gao, Priya Sundaresan, Satvik Sharma, Jeannette Bohg and Dorsa Sadigh, all listed with Stanford University. It was first posted to arXiv on 13/07/2026 and the version we read, v2, is dated 07/10/2026.

    Two corrections to the way the study is often summarised are worth making early.

    First, the video description says the robot navigates "using only code-based instructions". That is not quite what VIA does. The agent in VIA mostly does not write a control program. It looks at screenshots and calls tools, the way a computer-use agent operates a desktop application. The paper is explicit that this differs from the older "code as policies" approach: in VIA, "the agent is the policy and remains in the control loop during execution". The products used are coding agents, but the skill being borrowed is closer to operating software by sight.

    Second, "no training" means no robot-specific training of the model. The agent still receives a system prompt explaining the interface, the robot's configuration and limits, and a short task description with a success condition. Humans also built the interface, the tools and the low-level controller. We did not find a separate project website for VIA, so everything below comes from the arXiv paper itself. We also have not relied on the video for any figure: every number in this article comes from a paper, report or article we fetched directly.

    How Does VIA Let a Coding Agent Move a Robot?

    VIA is a framework that presents a robot to an AI agent as one more application to operate, by giving it a virtual workspace in which it can stage a movement, look at the result and adjust before the real robot moves.

    Diagram from the VIA paper: an off-the-shelf agent observes and acts through MCP tools on a robot control interface showing a blue virtual target gripper, which streams RGB-D from and executes waypoints on the robot environment
    VIA overview. The agent (left) uses MCP tools such as screenshot, gripper_rotate and execute_waypoint to position a blue "target gripper" in a browser-based interface (centre) before a controller moves the real gripper (right). Source: Hu et al., Stanford University, arXiv 2607.11119 (CC BY 4.0).

    The mechanism has three parts.

    • A visual interface. Camera feeds and, where depth data is good enough, a 3D point cloud built from RGB-D cameras. The agent can orbit, pan and zoom a virtual camera. Everything is built from raw sensor input and the robot's own joint readings.
    • A virtual target gripper. A blue gripper drawn over the scene shows where the real one would go. The agent moves it with a small set of Model Context Protocol tools: hover to probe a pixel (and read depth), click to jump the target near a point, translate and rotate to refine it, and, on a mobile robot, nav to stage a base movement.
    • A waypoint controller. When the agent is satisfied it calls execute_waypoint, and a conventional controller moves the real gripper to that pose. In simulation this is a simple proportional-integral controller.

    The important design choice is the nested loop. A naive setup would have the model guess a 6-degree-of-freedom pose from an image and send it straight to the robot, discovering mistakes only after the arm has moved. VIA lets the agent make its mistakes in the virtual workspace. The authors write that "the real robot only ever receives waypoints that have already been checked virtually". The interface was adapted from SPHINX, an earlier Stanford teleoperation tool; VIA puts an agent in the operator's seat.

    Five panels from the VIA paper showing one control loop on the open drawer task: the real scene, then show_gizmo, rotate and click tool calls moving a blue virtual gripper towards a drawer handle, then execute_waypoint moving the real arm
    One round of the VIA loop on the simulated "open drawer" task. The agent refines a single waypoint over three tool calls, each returning a fresh snapshot, then executes it. Source: Hu et al., Stanford University, arXiv 2607.11119 (CC BY 4.0).

    Because VIA uses Claude Code and Codex unmodified, it inherits their tool-use loops and context management. That has a practical consequence the paper highlights: when the robot drives across a room and an object leaves its field of view, the agent remembers where it was using the same context handling it uses to remember which files it has edited. For background on the agents, see our comparison of Claude Code, Codex and Cursor and our look at the Codex app.

    What Did VIA Actually Achieve?

    Simulation: six tasks, six models

    The simulated suite uses a Franka Panda arm in the robosuite simulator on six tasks: turning a stove knob, opening the middle drawer of a cabinet, placing a bowl on a plate, fitting a square nut on a peg, arranging seven coloured blocks into a rainbow, and building a T-shape from three blocks. Each agent and model pairing ran ten randomised seeds per task at the highest reasoning effort, with a one-hour cap per episode (two hours for the rainbow). Memory, internet search and general file access were switched off.

    The authors disabled file access because, in their words, "agents may cheat by trying to read the source files of the simulation benchmarks to recover the exact simulation states and object positions". A general agent pursues the goal by whatever route is open, which matters more once the environment is physical.

    VIA simulation: overall success by agent and model tier

    Mean success over six tabletop tasks, ten seeds each. Medium, large and extra-large model tiers. Higher is better.

    • Codex
    • Claude Code

    Medium tier · Terra 5.6 / Sonnet 5

    38.3%
    43.3%

    Large tier · Sol 5.6 / Opus 4.8

    73.3%
    65%

    Extra tier · Astra 6 / Fable 5

    100%
    85%
    Show data table
    BenchmarkCodexClaude Code
    Medium tier38.3%43.3%
    Large tier73.3%65%
    Extra tier100%85%

    Source: Hu et al., VIA, arXiv 2607.11119v2, Table 1

    Two findings stand out. Performance scaled with model strength in both families, which is the heart of the paper's argument: "each new generation of general capability transfers to the robot, with no fine-tuning required". And the hardest task, the T-block, shows where precision runs out. Every agent worked out that the bottom block should be rotated 45 degrees to give a larger contact area, but the task tolerates only about 8 mm of placement error, and success rates were 0% to 40% for every configuration except Codex with Astra, which scored 100%. (The Claude models tested were Sonnet 5, Opus 4.8 and Fable 5; the authors note they kept Opus 4.8 instead of the newer Opus 5 to avoid the cost of re-running.)

    How much does the interface matter?

    The paper compares VIA with a naive baseline that sees the same images and uses the same controller, but must send poses directly with no staging. For most models the interface is the difference between working and not working. For the strongest model it barely matters.

    VIA interface vs naive baseline, simulation

    Overall success over six tasks. The baseline was not reported for Sonnet 5. Higher is better.

    • With VIA interface
    • Naive baseline (no interface)

    Codex, Terra 5.6

    38.3%
    6.7%

    Codex, Sol 5.6

    73.3%
    25%

    Codex, Astra 6

    100%
    93.3%

    Claude Code, Sonnet 5

    43.3%
    n/a

    Claude Code, Opus 4.8

    65%
    6.7%

    Claude Code, Fable 5

    85%
    23.3%
    Show data table
    BenchmarkWith VIA interfaceNaive baseline (no interface)
    Codex, Terra 5.638.3%6.7%
    Codex, Sol 5.673.3%25%
    Codex, Astra 6100%93.3%
    Claude Code, Sonnet 543.3%n/a
    Claude Code, Opus 4.865%6.7%
    Claude Code, Fable 585%23.3%

    Source: Hu et al., VIA, arXiv 2607.11119v2, Tables 1 and 8

    For the mid-tier models, careful interface design multiplied success several times over, which supports the video's point that the harness around the model matters. But GPT-6 Astra scored 93.3% with five bare tools and no virtual workspace at all. The authors call this a "step change". Scaffolding that matters today may be less necessary for the next model generation.

    Real world: 24 of 25 on a Stretch 4

    For the physical tests the team used a Hello Robot Stretch 4, a mobile manipulator with an omnidirectional base, a vertical lift, a telescoping arm and a 3-degree-of-freedom wrist. Its head lidar gives only sparse depth at room scale, so the interface fell back to 2D camera views with depth probing, plus stereo depth from a wrist camera.

    Five photographs from the VIA paper with red arrows showing the intended motions: pulling a lamp chain, moving a toy from a shelf to a drawer, folding a towel diagonally, pulling a wipe and cleaning a mark on a table, and opening a wicker basket lid
    The five real-world VIA tasks. Object and robot starting positions were randomised for each episode. Source: Hu et al., Stanford University, arXiv 2607.11119 (CC BY 4.0).
    TaskWhat the robot had to doSuccess
    LampFind a lamp, pull its small chain to switch it on4/5
    ShelfOpen a drawer, fetch a toy lion from a shelf out of view, put it in5/5
    TowelFold a towel diagonally, corner to corner5/5
    WipePull a wipe from its tub and clean a black mark off a table5/5
    BasketOpen a basket lid until it flips to the back5/5
    TotalCodex with GPT-6 Astra, zero-shot24/25 (96%)

    The single failure came on the lamp, when the gripper overextended and grabbed the lamp's stem instead of the chain. The authors suggest a perception problem, since both are the same silver colour.

    The behaviours are more interesting than the tally. In the wipe task the agent's own arm blocks its camera while it scrubs, so, unprompted, it "moves the gripper away after a few wipes to check the cleanliness of the table". In one episode it dropped the wipe, found that holding it was not cleaning well, and switched to pushing it across the mark with a closed gripper. In the basket task it retried by itself when the lid slipped. None of this was scripted. It is the habit a coding agent shows when it reruns the tests after an edit.

    Now the caveats, which the paper states plainly. Five trials per task is a small sample: 24 of 25 is encouraging, but it cannot distinguish a 96% system from an 85% one. Only one agent and model was tested on the real robot. The paper does not report how long the real-world episodes took. And the tasks are quasi-static: nothing in the scene moves unless the robot moves it.

    From Code as Policies to Agent as Policy

    VIA did not appear from nowhere. The idea of using a language model's programming ability to control robots is four years old, and the recent shift is about where the model sits.

    Code as Policies (2022). Code as Policies is a method in which a language model writes a Python program that calls perception and motion functions, and the program, not the model, controls the robot. The original Google paper by Jacky Liang and colleagues, posted on 16/09/2022, showed such programs could handle spatial reasoning and new instructions. Its limit was that humans had to supply capable building blocks, such as a ready-made "stack objects in order" function.

    CaP-X (March 2026). CaP-X, from researchers at NVIDIA, UC Berkeley, Stanford and Carnegie Mellon, tested how much of that success came from the building blocks. Across 12 models, performance fell as human-crafted abstractions were removed, and no model matched human-written programs, which averaged 88.5% success, in a single attempt. The same paper showed the gap narrows when the model gets several turns, structured feedback and its own growing skill library. The human experts needed 2 to 3 weeks of iteration per task for peg insertion and two-arm handover, which is the labour general agents are being asked to absorb.

    Agent as Policy (September 2026). Agent as Policy (AGP), from the University of Notre Dame, UC San Diego and San Diego State University, is the closest sibling to VIA and the paper that best fits the "code written by a general agent" description. Here the agent genuinely does write and run programs mid-task: estimating an object's position from depth images, computing a gripper pose, issuing the motion and revising after seeing the outcome. With Codex and GPT-6 Astra on a real six-joint arm, AGP reached at least 80% success in seven of eight task configurations, including 10 of 10 on dice flipping, 9 of 10 on throwing an object at a target and 5 of 5 on sequential two-arm towel folding.

    AGP also reports the number VIA leaves out: time. Mean completion times ran from 17.7 minutes for a throw to 50.8 minutes for a towel fold, at £6.20 to £18.35 ($8.16 to $24.14) in API fees. In one 48.6-minute assembly run, planning and reasoning took 68.2% of the time and robot motion 23.5%.

    A comparison inside AGP sharpens the sim-to-real argument. Two earlier agentic methods that write a fixed program in simulation and then deploy it, Graph as Policy and ASPIRE, succeeded in 0 of 25 and 3 of 25 real trials on three tasks where AGP succeeded in 23 of 25. The authors attribute this to differences in friction, lighting and depth noise that a pre-written program cannot adapt to, whereas an agent in the loop can look again and change its grip. Simulation remains central to training specialised robot models, as our pieces on NVIDIA Isaac Lab 3.0, Genie 4 and World Labs Atlas explain. The agent-in-the-loop approach sidesteps that pipeline.

    What Do Independent Studies Show?

    A single lab's 25 trials would not justify much. Three other groups, with different robots and benchmarks, reported compatible results within weeks.

    NVIDIA Astranomy: 74.8% on RoboLab-120

    NVIDIA's Seattle Robotics Lab published Astranomy on 14/10/2026, a study of GPT-6 Astra as a "native robot policy" on RoboLab-120, a simulated pick-and-place benchmark with 120 tasks and five episodes each. It tested Astra three ways: called directly through the API (Direct Policy), inside a persistent Codex session (Agentic Policy), and as a supervisor reviewing actions proposed by a specialised world-action model (Orchestrated Policy). This is the study Fortune cites for the 74.8% figure.

    NVIDIA chart of success rate and completion score on RoboLab-120: GPT-6 Astra Agentic Policy 74.8 percent, Orchestrated Policy 66.5, Direct Policy EEF 66.3, then leaderboard policies led by Lakeside at 51.0 down to pi0.5 at 28.0
    GPT-6 Astra configurations (green) against existing RoboLab-120 leaderboard policies (grey). Solid bars are success rate, pale extensions are completion score, lines are 95% confidence intervals. Source: NVIDIA Research, Seattle Robotics Lab.

    RoboLab-120 success rate: general model vs robot foundation models

    120 simulated pick-and-place tasks, five episodes each. First three rows are GPT-6 Astra configurations. Higher is better.

    Astra, Agentic Policy (Codex)

    74.8%

    Astra, Orchestrated Policy

    66.5%

    Astra, Direct Policy (end-effector)

    66.3%

    Lakeside · best listed leaderboard policy

    51%

    Cosmos3-Nano-Policy · world-action model

    36.8%

    pi0.5 · vision-language-action model

    28%
    Show data table
    BenchmarkSuccess rate
    Astra, Agentic Policy (Codex)74.8%
    Astra, Orchestrated Policy66.5%
    Astra, Direct Policy (end-effector)66.3%
    Lakeside51%
    Cosmos3-Nano-Policy36.8%
    pi0.528%

    Source: NVIDIA Research, Astranomy, 14/10/2026, Figures 2 and 3

    Three details matter. The gain from the agentic harness came from keeping interaction history, "without gaining additional benefits from tool-use". Handing action generation to a specialised model did not clearly beat Astra producing actions itself. And model choice was decisive: in a 30-task ablation Astra scored 72.7%, GPT-6 Sol 33.3%, Claude Opus 5.5 25.3% and GPT-6 Luna 2.0%. So "coding agents can run robots" currently means, to a large extent, "one frontier model can".

    NVIDIA is also careful about what it has not shown. With five episodes per task it has "not established the statistical significance" of differences between configurations. The benchmark is pick-and-place only, excluding insertion, assembly and dynamic tasks. And the simulation is paused while the model thinks, so the results say nothing about a world that keeps moving.

    Artificial Analysis: capable, and about 100 times slower than a person

    On the same day, Artificial Analysis released AA-Robotics v0.1, a research preview in which frontier models inside their vendors' coding harnesses control a small real 6-axis arm through an HTTP API, with four cameras and the freedom to write and run code. There are eight tabletop tasks, graded by a human judge from 0 to 100.

    AA-Robotics v0.1 score by model

    Mean score across eight real tabletop tasks, 0-100, each model in its vendor's coding harness at high effort. Higher is better.

    GPT-6 Astra (Codex)

    69.4

    GPT-6.1 Sol (Codex)

    64.6

    Gemini 4 Argon (Antigravity)

    55.3

    Claude Opus 5.5 (Claude Code)

    48.8

    Claude Fable 5.1 (Claude Code)

    46.3
    Show data table
    BenchmarkAA-Robotics score
    GPT-6 Astra (Codex)69.4
    GPT-6.1 Sol (Codex)64.6
    Gemini 4 Argon (Antigravity)55.3
    Claude Opus 5.5 (Claude Code)48.8
    Claude Fable 5.1 (Claude Code)46.3

    Source: Artificial Analysis, AA-Robotics v0.1, 14/10/2026

    Every model pressed a red button and placed a toy in a marked rectangle. Results then diverged sharply: only Astra and Gemini 4 Argon poured a drink successfully, and no model scored above 17 on measuring a box. Physical accuracy was modest, with grasp success rates between 28% and 45%. Artificial Analysis's own conclusion is blunt: the models are "far too slow and expensive for any kind of production usage". Note that each vendor ran in a different harness, so the ranking mixes model and harness effects.

    X2Real: a far less flattering benchmark

    The least favourable data point comes from X2Real, a simulation benchmark from the robotics company X Square Robot, built to track real-world performance. Evaluated on 44 out-of-distribution environments, GPT-6 Astra reached 36.15% success, only 1.89 percentage points above the company's own specialised model. The split is telling: Astra led by 10.57 points on reasoning-heavy tasks and trailed by 6.79 points on manipulation tasks, with weaknesses in "dynamic interaction, tool use, and certain bimanual behaviors". The authors also recorded more than 30 seconds of inference per decision, with the simulation paused throughout. The vendor of the comparison model wrote the benchmark, so treat it with caution, but it is a useful counterweight to 96% and 100%.

    How Slow and Expensive Is It?

    Latency is the central objection, and every source agrees on it. VIA's limitations section says that "inference remains slow and expensive" and that the system "is not suitable for dynamic tasks like catching a ball".

    SystemTypeLatency per callShare of episode spent on inferenceAPI cost per successful episode
    GPT-6 Astra, Direct PolicyGeneral model18.56 s97.0%£1.49 ($1.96)
    GPT-6 Astra, Agentic Policy (Codex)General agent13.37 s97.8%£27.66 ($36.40)
    GPT-6 Astra, Orchestrated PolicyGeneral model supervising a specialist11.50 s + 2.74 s94.9%£13.47 ($17.72)
    Cosmos3-Nano-PolicyWorld-action model713.76 ms25.6%n/a (self-hosted)
    pi0.5Vision-language-action model46.96 ms4.6%n/a (self-hosted)

    Source: NVIDIA Research, Astranomy, Figure 2 and Table 3. Specialist-model latencies are medians on an H200 GPU as reported by Black Forest Labs and reproduced by NVIDIA. Token costs use prices published on 01/10/2026.

    The gap between 18.56 seconds and 46.96 milliseconds is a factor of roughly 400. NVIDIA illustrates it with a task of putting a yogurt in a bowl: about 5.2 seconds of actual arm movement inside an estimated 3 minutes 57 seconds once thinking time is included. Artificial Analysis put it in human terms: about 20 seconds for a person, about 30 minutes for four of the five models, and 52 minutes for Gemini 4 Argon. At £12.43 ($16.36) per task for Astra, and with US private-sector hourly earnings at £28.74 ($37.81), it calculated that even the cheapest model costs about ten times the human equivalent.

    Cost per success also varies by model family in ways that do not track capability. In VIA's simulation runs, the best-performing configuration was among the cheapest.

    VIA simulation: success vs API cost per successful episode

    Each point is one model tier (medium, large, extra). Up and to the left is better.

    • Codex
    • Claude Code
    Show data table
    ModelSettingOverall successCost per task
    CodexTerra 5.638.3%$2.4
    CodexSol 5.673.3%$2.8
    CodexAstra 6100%$3.2
    Claude CodeSonnet 543.3%$5.3
    Claude CodeOpus 4.865%$11
    Claude CodeFable 585%$16

    Source: Hu et al., VIA, arXiv 2607.11119v2, Tables 1 and 7. Costs are means over successful episodes only.

    Codex with Astra averaged about £2.43 ($3.2) per successful episode and needed the fewest tool calls (51 on average, against 72 to 146 for the Claude Code configurations). Two cautions: these averages exclude failed episodes, which often ran to the time limit, so true cost per attempt is higher for weaker models; and they are simulation figures.

    Can it be made faster? AGP found that giving a weaker model (GPT-5.6 Terra) the stronger model's saved notes lifted its success on an assembly task from 1 of 5 to 4 of 5 and cut mean time from 19.0 to 12.5 minutes. A Huawei study of Astra on a lift-button task found that supplying complete robot geometry and camera information cut mean completion time by 57.4% in simulation. These are large gains from a very slow starting point. None reaches real time.

    Why Did OpenAI Fold Its Robotics Team?

    The video frames OpenAI as "prioritizing" robotics research on the theory that better general intelligence yields better robots. The reporting is more specific and reads slightly differently. On 13/11/2026 Fortune reported that OpenAI had disbanded its stand-alone robotics team. According to Fortune, the robotics researchers moved into the main research organisation under chief research officer Mark Chen, and the applied robotics and hardware teams moved into the Applied organisation under president Greg Brockman. OpenAI confirmed the change and a spokesperson said robotics "remains a core part of OpenAI's long-term vision".

    The stated reason is the one this article has been examining. Fortune says OpenAI researchers saw surprising results when they connected GPT-6 Astra to robots directly, and that the company now believes its general-purpose frontier models can be applied to robotics without a separate model line. At the start of the year, Fortune adds, the internal bet had been on building from the Sora video model. Guanya Shi of Carnegie Mellon, a CaP-X co-author, told Fortune the move suggests OpenAI sees robotics as "no longer a special problem" but a downstream application of general intelligence, like coding.

    Some context tempers the story. This is the second time OpenAI has dissolved a robotics team: the first was wound down in 2020 and rebuilt from 2024. Fortune also reports cost pressure, an instruction from Sam Altman to drop "side quests", and an operating loss of about £8.3 billion ($10.9 billion) in the latest quarter. A reorganisation can be a research conviction and a cost decision at once. Fortune says OpenAI still intends to build its own humanoid, with Altman targeting a first public demonstration in 2027. This is a single-outlet report, confirmed by OpenAI in outline; we have not seen an OpenAI blog post setting out the reasoning. For the company's wider year, see our DevDay 2026 recap.

    What is the Bitter Lesson, and does it apply?

    The Bitter Lesson is Richard Sutton's 2019 observation that, over 70 years of AI research, general methods that scale with computation have ended up beating methods built on human domain knowledge. In the essay he wrote that "general methods that leverage computation are ultimately the most effective, and by a large margin".

    The VIA baseline is a neat illustration: an interface carefully designed by experts multiplied the success of mid-tier models, then became nearly redundant for the next model up. But the lesson cuts in more than one direction. Sutton's argument favours whatever scales with computation and data. Companies training robot models on enormous quantities of physical data would say that is exactly what they are doing, and that a general model reasoning in text for 18 seconds per step is the hand-engineered detour. The Bitter Lesson does not say which general method wins.

    The Competing Approach: VLA Models

    A vision-language-action (VLA) model is a neural network that takes camera images and a language instruction and directly outputs motor commands, usually many times per second, after being trained or fine-tuned on recorded robot data. It is the approach behind almost every commercial robot deployment announced this year.

    Google DeepMind. Gemini Robotics 2.5, announced on 10/11/2026, is DeepMind's latest VLA. DeepMind reports a 30% to 100% higher success rate than its predecessor across 200 tasks, and a 96% success rate picking and packing car parts on Mercedes-Benz lines with Apptronik's Apollo 2 humanoid. Google's structure is itself a hybrid: a general reasoning model (Gemini Robotics ER) plans, and the VLA moves the hands. All of those figures are DeepMind's own.

    Figure. Figure's Helix 2.5, published on 17/09/2026, goes furthest from the general-model thesis. Figure says it was pretrained from random initialisation entirely on Index, the company's dataset of human behaviour, where the earlier Helix 02 started from a pretrained vision-language model. In 30 unseen homes, Figure reports 56% zero-shot success for the Index-pretrained model against 9% for one trained from scratch, on tidying, towel folding and bed making. Figure says it has committed £2.66 billion ($3.5 billion) of compute to training Helix, and adds that "the point is not that general humanoid robotics is solved". These are also self-reported numbers.

    Physical Intelligence. We could not load the company's own blog while researching this piece, so we report only what third-party studies measured for its openly available pi0.5 model: 28.0% on RoboLab-120 with a median latency of 46.96 ms in NVIDIA's report. CaP-X found its training-free coding agent performed comparably to pi0.5 on a perturbed-position test. Newer Physical Intelligence models may do better; we have no verified figures for them.

    Unitree. Hardware makers sit to one side of this argument. Unitree's Superman prototype and record Shanghai listing were about legs, speed and manufacturing cost. Fast locomotion of that kind runs on low-level controllers trained with reinforcement learning at high frequency, a layer no one is proposing to hand to a cloud language model.

    Russ Tedrake of MIT made the commercial version of this point to Fortune: robotics companies keep an advantage because their machines collect deployment data on factory floors, and better foundation models make that data more valuable.

    Limitations and Counter-Arguments

    Latency and connectivity

    Tens of seconds per decision rules out anything where the world moves: conveyor lines at speed, catching, handovers with people, recovering a slipping object. Shi told Fortune that large models are slow and cloud-hosted, so a robot needs a constant network connection, and that he still sees a need for smaller specialised models running on the robot's own computer. Nearly every benchmark above either paused the simulation during inference or used static scenes, so real-time ability has barely been measured, let alone demonstrated.

    Safety

    The AGP authors state the risk directly: applying agents to physical systems "introduces risks from inaccurate perception, faulty programs, and inappropriate motion requests, which can cause equipment damage or injury", and deployment near people needs "additional safety evaluation, appropriate supervision, and independent protective mechanisms". VIA's current controller does not avoid collisions; the authors list a more sophisticated controller as future work. Given that agents also tried to read simulator source files to shortcut tasks, the safety case for an unsupervised general agent with a physical body is not yet made. DeepMind released an open robot jailbreak benchmark alongside Gemini Robotics 2.5, which shows vendors treat this as a live problem.

    Dexterity and contact

    VIA and most of the other studies use parallel-jaw grippers. VIA lists dexterous hands as future work, and AGP offers only a single qualitative demonstration of a five-finger hand lifting a bottle. NVIDIA found that even with Astra the recurring failure was "converting attempted manipulation into a completed grasp, transfer, or placement": the gripper hovers and adjusts without securing the object. Artificial Analysis measured grasp success under 50% for every model.

    Small samples and narrow tasks

    The headline results rest on 5 to 10 trials per task. NVIDIA says outright that it has not established statistical significance. An arXiv study from October, Embodied Agent Arena, catalogues how VLM agents fail during execution: they "achieve early subgoals yet lose object control", or "stop before the required physical state holds", and extra computation does not reliably fix it.

    Sim-to-real cuts both ways

    The agent-in-the-loop approach avoids the classic sim-to-real problem because nothing is trained in simulation. But most of the evidence for it is simulated: VIA's model comparison, all of Astranomy and all of X2Real. Real-world evidence so far is VIA's 25 trials, AGP's 70 or so, and Artificial Analysis's eight tasks.

    Dependence on one model and one vendor

    The strongest results all use GPT-6 Astra, and they fall steeply on other models. A robot programme built on one proprietary API inherits its pricing and deprecation schedule. NVIDIA also notes it cannot fully separate the model's ability from "unobserved serving infrastructure or internal processing".

    How the Approaches Compare

    General agent in the loop (VIA, AGP)Code as policy (CaP, CaP-X)VLA model (Gemini Robotics, Helix, pi0.5)
    Robot training data neededNoneNone, but hand-built primitives help a lotLarge robot or human-behaviour datasets
    Who is the policy at run timeThe agent itselfA generated programThe trained network
    Decision speedRoughly 13 to 30+ seconds per step6.8 to 23.8 s per code iteration (CaP-Agent0), then runs without the modelTens to hundreds of milliseconds
    New taskWrite a promptWrite a prompt, perhaps new primitivesOften collect data and fine-tune
    Best published evidence24/25 real trials (VIA); 74.8% RoboLab-120Matches human code on 4 of 7 tasks (per The Decoder)96% on factory kitting (DeepMind, self-reported)
    Main weaknessSpeed, cost, static scenes onlyCannot adapt once the program is runningData hunger; weaker long-horizon reasoning
    Runs offline on the robotNoYes, after generationOften yes

    The likely outcome is a combination. CaP-X proposes coding agents for high-level logic and recovery with VLA policies for fine motor control, which is also how Google splits Gemini Robotics ER from the VLA. The awkward finding for that neat picture is NVIDIA's: its orchestrated configuration (66.5%) did no better than Astra acting alone (66.3%) and worse than the agentic version (74.8%). NVIDIA calls how best to combine the two "an open question".

    What This Means for Builders

    • Robotics researchers and hobbyists: this is usable now. If you have an arm with a camera and a position controller, wrapping it in a few MCP tools and pointing Codex or Claude Code at it is a weekend project. Expect minutes per task and a few pounds per attempt.
    • Teams training robot models: the near-term use VIA's authors suggest is data collection, with "a slow interface agent" generating demonstrations for training fast policies, and automated resets for real-world reinforcement learning. That makes general agents a supplier to VLA training, not a rival.
    • Manufacturers with slow, high-mix work: VIA's authors name "assembly tasks where reliability matters more than speed". If a task changes weekly and a robot taking 20 minutes is acceptable, zero-shot agents may beat weeks of programming. Budget for supervision and physical safeguards. Our overview of industrial AI and digital twins covers the surrounding tooling.
    • Anyone evaluating humanoid claims: ask which layer a demo exercises. A humanoid walking is a control result. A humanoid deciding what to do next may increasingly be a frontier-model result, and those improve on the model vendors' release schedule. Compare the current flagships in Claude Opus 5.5 vs GPT-6 Astra.

    The Bottom Line

    The claim in the video title holds up better than most claims of its kind. The Stanford study exists, it is called VIA, and it reports what the video says: a general coding agent with no robot training completed 24 of 25 real-world trials. NVIDIA, Notre Dame and Artificial Analysis independently found the same basic capability on different robots, and OpenAI has reorganised around it.

    What the evidence does not support is the idea that the robot brain problem is solved. The brain that exists thinks for 13 to 30 seconds between movements, costs pounds per task, needs a network connection, has been tested mostly on static tabletop scenes with simple grippers, and does its best work on one vendor's top model. Jones's own closing caveat is the right one: a robot earns its place only when it works fast enough to need little supervision. On current numbers that is a gap of two orders of magnitude.

    Two things to watch: whether the next frontier models close the speed gap as well as the accuracy gap, and whether hybrids start to beat general agents acting alone, which NVIDIA's first attempt did not show. Until then, general intelligence carries more robotics ability than expected, and specialised, fast, data-hungry models still do the work that has to happen in real time.

    Last updated: 19/11/2026. Dollar figures are as published by each source; sterling conversions are approximate at $1 = £0.76 (rate on 19/11/2026). Model names are given as they appear in each paper.

    Sources

    Frequently Asked Questions

    What is Stanford's VIA robot study?
    VIA (Visual Interface Agent for Robot Control) is a Stanford University paper, arXiv 2607.11119, first posted on 13/07/2026 and revised on 07/10/2026, by Hengyuan Hu, Jensen Gao, Priya Sundaresan, Satvik Sharma, Jeannette Bohg and Dorsa Sadigh. It gives an unmodified coding agent such as Codex or Claude Code a visual interface and a small set of MCP tools for posing a virtual gripper, then lets the agent drive a real or simulated robot with no robot-specific training.
    How well did the coding agent control the real robot?
    Using Codex with GPT-6 Astra, VIA succeeded in 24 of 25 real-world trials (96%) across five tasks on a Hello Robot Stretch 4 mobile manipulator: pulling a lamp chain, moving a toy from a shelf into a drawer, folding a towel, wiping a mark off a table and opening a basket lid. Each task was run only five times, so the sample is small. In simulation, success ranged from 38.3% to 100% depending on the model.
    Does this mean robots no longer need specialised AI models?
    No. The same studies that show general agents completing tasks also show they are far too slow for real-time work. NVIDIA measured GPT-6 Astra at 18.56 seconds per inference call against 46.96 milliseconds for the pi0.5 vision-language-action model, and Artificial Analysis found frontier models roughly 100 times slower than a human on tabletop tasks. Dynamic tasks, contact-rich assembly and dexterous hands remain largely untested.
    How much does it cost to run a robot with a coding agent?
    It varies widely. Stanford's VIA paper estimates an average of $2.4 to $16.3 in API fees per successful simulated episode depending on the model. NVIDIA's Astranomy study reports $1.96 per successful episode for direct GPT-6 Astra calls and $36.40 for the agentic Codex configuration. Artificial Analysis measured $16.36 per tabletop task for GPT-6 Astra and $1.93 for GPT-6.1 Sol.
    Why did OpenAI disband its robotics team?
    Fortune reported on 13/11/2026 that OpenAI had dissolved its stand-alone robotics team, moving robotics researchers into the main research organisation under Mark Chen and the applied robotics and hardware staff into the Applied organisation under Greg Brockman. OpenAI confirmed the change and said its general-purpose frontier models, including GPT-6 Astra, had shown they can be applied directly to robotics. It says robotics remains part of its long-term plans.
    AI Tools Review Editorial Team

    AI Tools Review Editorial Team Expert verified

    Our editorial team consists of veteran AI researchers, software engineers, and industry analysts. We spend hundreds of hours benchmarking frontier models natively to provide you with objective, actionable intelligence on agentic AI capabilities and cybersecurity landscapes.