NVIDIA Nemotron 3.5 Lightning: Specs & Benchmarks

Nemotron 3.5 Lightning is NVIDIA's successor to Nemotron 3 Nano, the small, fast, open-weights model NVIDIA has positioned as a specialist worker for long-running, always-on AI agents rather than a generalist chatbot competing with the likes of Claude, Gemini or GPT. The pitch is narrow and deliberate: agent workflows spend most of their compute on repetitive, well-defined sub-tasks - parsing a document, calling a tool, classifying an intent, drafting a routine response - and only occasionally need the heaviest, most expensive reasoning a frontier model provides. Lightning is built to handle the former cheaply and quickly, in bulk.

NVIDIA paired the model release with NeMo Switchyard, an open-source routing library that automates exactly this kind of triage: it inspects each step in an agent's workflow and sends it to the most appropriate model available, whether that is Lightning itself, a larger open model, or a proprietary frontier model like Claude Opus 4.8, based on configurable priorities around quality, cost and speed. NVIDIA's own published example shows this combination holding task completion roughly level with using a frontier model for everything, while cutting cost to close to a third.

• Best for: high-volume, latency-sensitive steps inside multi-agent pipelines - tool calls, classification, routine drafting, and long-running always-on agents.
• Headline numbers: 31.6B total / 3.6B active parameters, up to 4x faster output, 30% faster task completion on PinchBench, roughly 670 tokens/second on an NVFP4 endpoint.
• Defining trait: hybrid Mamba-2 + Mixture-of-Experts + Attention architecture, released as fully open weights under NVIDIA's OpenMDW-1.1 licence.
• Main caveat: it is not the smartest open model available - NVIDIA's own larger Nemotron 3 Super and third-party models like Qwen3.6 35B score meaningfully higher on general intelligence benchmarks.

Nemotron 3.5 Lightning carries forward the architecture NVIDIA introduced with Nemotron 3 Nano: a hybrid design that interleaves Mamba-2 state-space layers with Mixture-of-Experts feed-forward blocks, plus a smaller number of standard Attention layers layered on top. Independent analysis from Artificial Analysis puts the total parameter count at 31.6 billion, with only 3.6 billion active for any given token - figures NVIDIA's own Hugging Face model card rounds to the more marketing-friendly "30B/3B". Whichever figure you use, the practical effect is the same: Lightning does the compute work of a roughly 3-4B dense model while drawing on the broader knowledge of a 30B-class network, which is what keeps both memory footprint and inference cost low.

The Mamba-2 layers matter specifically for long context. NVIDIA lists a headline context window of 1 million tokens for multi-GPU deployments (a single H100 is memory-bound to roughly 256,000 tokens), and state-space layers scale far more gracefully with sequence length than pure Attention does - a meaningful factor for an "always-on agent" model that may be expected to hold a long-running conversation or tool-call history in context for hours at a time.

On the training side, NVIDIA describes the process as "harness-optimised" - tuned specifically against agent frameworks and tool-calling workflows rather than general chat - and says pretraining included multi-token prediction (MTP), which also underpins the model's inference-time speed trick. Nemotron 3.5 Lightning ships with two small draft models, codenamed DSpark and DFlash, that perform speculative decoding: the draft model proposes several tokens ahead, and the main model verifies them in a single pass, which is a meaningful part of how NVIDIA gets to its "4x faster output" claim without changing the active parameter count. NVIDIA also references a purpose-built "Nemotron-RL Agentic Terminal Pivot" dataset used in post-training, aimed at reinforcement-learning-style agentic and terminal-based tasks rather than static question-answering.

Weights are released in both BF16 and NVFP4 (NVIDIA's 4-bit floating-point format for Blackwell-generation hardware) checkpoints, with the model compatible across Blackwell, Hopper and Ampere GPU generations. The full model, post-training recipes and training data are released under NVIDIA's own OpenMDW-1.1 licence, which permits commercial use, fine-tuning and redistribution without requiring NVIDIA's sign-off - a genuinely open release, not a restricted research licence.

It is worth being precise about what "customisable" means here, because NVIDIA leans on it heavily in its own framing. Because the training recipes and data are published alongside the weights, not just the model itself, teams can reproduce NVIDIA's own post-training process on their own domain data rather than starting from a frozen checkpoint and hoping generic fine-tuning tools work well enough. NVIDIA's launch material and the SiliconANGLE case studies below point at this as the more differentiated part of the release compared with a typical open-weights drop: the recipe, not just the resulting model file, is the product.

Agentic throughput and speed

This is the category Nemotron 3.5 Lightning is built around. NVIDIA's own developer blog reports the model reaching 86% accuracy on PinchBench (its internal agentic-task benchmark) while completing a batch of 10,000 tasks 30% faster than Qwen3.6 35B at broadly comparable accuracy - the published Hugging Face model card for the BF16 checkpoint lists the more precise figure of 85.37%. Independent pre-release testing by Artificial Analysis, run against a DeepInfra endpoint serving the final NVFP4 weights, measured a median output speed of nearly 670 tokens per second, which the firm noted was considerably faster than comparable open models are typically served in production today.

Software engineering and terminal use

On SWE-bench Verified, the BF16 checkpoint scores 51.56% - a genuinely useful score for a model this small and this fast, though well short of frontier coding specialists. The bigger jump is on Terminal-Bench v2.1, where Artificial Analysis records Lightning at 24%, up sharply from Nemotron 3 Nano's 7% - evidence that NVIDIA's harness-optimised, agent-framework-specific training genuinely improved the model's ability to operate command-line tools rather than just answer questions about them.

General knowledge and reasoning

Lightning is respectable but unremarkable here, which is by design. It scores 81.94% on MMLU Pro, 75.44% on GPQA Diamond and 71.88% on IFBench (loose) per its own model card. On the composite Artificial Analysis Intelligence Index (which blends nine evaluations including GDPval-AA v2, Terminal-Bench v2.1, GPQA Diamond and Humanity's Last Exam), Lightning scores 24 - a genuine improvement of +9 points over Nemotron 3 Nano's 15, and tied with the much-discussed open model gpt-oss-120b, but well below larger open models in the same comparison chart, including Qwen3.8 Max (58), GLM-5.2 (53) and even NVIDIA's own larger sibling, Nemotron 3 Super (26).

Agentic task quality (not just speed)

NVIDIA's specific agentic benchmark, GDPval-AA v2, is where Lightning shows its largest relative gain: Artificial Analysis records an Elo of 824, a jump of +334 Elo over Nemotron 3 Nano that is large enough to move Lightning past both gpt-oss-120b and NVIDIA's own larger Nemotron 3 Super on that specific measure - a useful reminder that "smaller and faster" did not mean "worse at every agentic task" here, even where it clearly does mean lower general intelligence.

Tool calling and structured output

Although NVIDIA has not published a dedicated function-calling benchmark score for Lightning in the sources reviewed for this article, the model's entire design brief - harness-optimised post-training, an agentic RL dataset built specifically around terminal and tool-use tasks, and speculative decoding tuned for the short, structured completions typical of JSON tool calls - points squarely at reliable, low-latency tool invocation as a first-class use case rather than an afterthought. That is consistent with the strong Terminal-Bench v2.1 and GDPval-AA v2 results above, both of which depend heavily on a model correctly and efficiently calling external tools rather than reasoning in free text alone.

NVIDIA's own developer blog frames Lightning's positioning with a simple two-axis chart: PinchBench accuracy against GPU-hours needed to complete 10,000 tasks. It is worth looking at directly rather than taking the "30% faster" headline at face value.

The honest read of that chart: Nemotron 3.5 Lightning is not the most accurate of the three models shown - Qwen3.6-35B sits slightly above it on the y-axis. What Lightning wins outright is time: it clears the same batch of tasks using meaningfully fewer GPU-hours than either comparator, which is the entire point of an "efficiency frontier" model built for high-volume agent work rather than maximum single-shot capability.

Zooming out to general intelligence, the independent Artificial Analysis Intelligence Index tells a more sobering story about where Lightning sits in the wider field:

• Artificial Analysis Intelligence Index: 24 - up +9 from Nemotron 3 Nano's 15, tied with gpt-oss-120b, but roughly 7x faster per task (about 0.5 minutes versus gpt-oss-120b's 3.4 minutes).
• GDPval-AA v2: 824 Elo, +334 over Nemotron 3 Nano, ahead of gpt-oss-120b and NVIDIA's own Nemotron 3 Super on this specific agentic measure.
• Terminal-Bench v2.1: 24%, up from 7% for Nemotron 3 Nano.
• SWE-bench Verified: 51.56% (BF16 checkpoint).
• PinchBench: 85.37% (model card) to 86% (NVIDIA's own blog claim), completing 10,000 tasks roughly 30% faster than Qwen3.6 35B at similar accuracy.
• MMLU Pro / GPQA Diamond / IFBench (loose): 81.94% / 75.44% / 71.88%.

Taken together, the pattern is consistent and NVIDIA is not hiding it: Lightning made large, specific gains on agentic and terminal benchmarks relative to its own predecessor, and wins decisively on speed-per-task against similarly sized open models. But on broad, general-purpose intelligence, it remains near the bottom of the current open-model field, below larger models several times its total parameter count and even below NVIDIA's own larger Nemotron 3 Super. That is the trade-off the model is built around, not a hidden weakness.

As of 11 August 2026, Nemotron 3.5 Lightning is available immediately as open weights on Hugging Face and ModelScope, as a hosted pay-per-token model on OpenRouter, and as an NVIDIA-managed NIM microservice on build.nvidia.com. NVIDIA also lists a wider ecosystem of NVIDIA Cloud Partners and inference providers hosting the model, including DeepInfra, Fireworks, FriendliAI, CoreWeave, GMI Cloud, Nebius and Crusoe.

Because it is genuinely lightweight for its capability class, Lightning is also explicitly positioned for local and edge deployment: NVIDIA lists support for NVIDIA Jetson devices, consumer cards like the GeForce RTX 5090, and its own DGX Spark desktop system, alongside the usual local-inference tooling - LM Studio, llama.cpp, Ollama and Unsloth - with dozens of community-quantised GGUF variants already circulating at launch. CNBC's coverage specifically framed this as a "lightweight" model that "can run on a single GPU on a laptop or desktop," a notable departure from the multi-GPU cluster requirements of most 2026-era frontier-adjacent models.

NeMo Switchyard ships as a separate, open-source repository rather than a hosted product. It is designed to sit in front of an existing agent framework and route each step to whichever configured model - open, proprietary, or NVIDIA's own - best fits that step, based on developer-set priorities for quality, cost, speed or specialised expertise, without requiring the calling application to be rewritten. NVIDIA named a first wave of partners integrating or trialling it at launch, including Boomi, Cadence Design Systems, Classmethod, Cognition AI, Kong, LangChain, Nous Research and Siemens. Kari Briski, NVIDIA's VP of Generative AI, is quoted by SiliconANGLE describing the core pitch simply: "Lightning is remarkably easy to customise."

Read that chart carefully rather than just the headline: the two NeMo Switchyard configurations complete a broadly similar, in one case very slightly higher, share of tasks compared with Opus 4.8 running alone, while landing at roughly £51-£67 (about $65-$85) instead of Opus 4.8's roughly £138 (about $175) for the same task set - NVIDIA's own basis for its "nearly a third of the cost" claim. That is a single published example rather than a guarantee for every workload, and NVIDIA's own reported range of partner results (21% to 74% cost reduction, with accuracy trade-offs at the higher end) makes clear the saving is workload-dependent, not fixed.

SiliconANGLE also reported early customisation case studies from launch partners: CodeRabbit reportedly trained a router-agent variant of Lightning for around £67 (about $85) in roughly two hours using NVIDIA's standard "auto" post-training recipe, while other partners reported completing fine-tuning runs on a single H100 GPU or overnight with minimal setup - concrete evidence for NVIDIA's broader claim that Lightning is cheap and fast to specialise for a narrow enterprise task, not just to run.

NVIDIA has not published a single, unified price for Nemotron 3.5 Lightning, because there is no single way to run it - and that is the point of an open-weights release. The clearest reference point is OpenRouter, which lists standard hosted pricing of roughly £0.06 (about $0.08) per million input tokens and £0.16 (about $0.20) per million output tokens, alongside a separate free-tier listing for lower-volume or evaluation use. Given the model's small active-parameter count, this sits well below typical mid-size open-model API pricing.

NVIDIA's own hosted catalogue at build.nvidia.com is free to try for prototyping through the NVIDIA Developer Program, rate-limited to roughly 40 requests per minute. Because the weights themselves are open under the OpenMDW-1.1 licence, self-hosting for evaluation, research or even production use costs nothing beyond your own compute - the model was specifically designed to run on a single consumer or datacentre GPU, which is a materially different cost proposition to models that require multi-GPU clusters just to load.

The one place actual published NVIDIA pricing exists is for production-grade NIM microservice deployment with enterprise support, which requires an NVIDIA AI Enterprise licence. NVIDIA's public NIM FAQ prices this from roughly £3,540 (about $4,500) per GPU per year, or approximately £0.79 (about $1) per GPU-hour when consumed through a cloud provider (plus the underlying cloud instance cost), with a free 90-day trial available when deploying under an AI Enterprise licence. That licence buys production and long-term-support branches, access to select NVIDIA-maintained NIM microservices, vGPU compute entitlements, Omniverse libraries and the Run:ai orchestration platform - it is a support and infrastructure product, not a per-token model fee.

The honest bottom line on cost: if you just want to run Nemotron 3.5 Lightning, it is free (open weights, your own hardware) or very cheap (fractions of a penny per thousand tokens via OpenRouter or a cloud partner). If you want NVIDIA's enterprise support wrapper around it in production, that is a separate, GPU-metered infrastructure licence rather than a model price - budget for it as such rather than assuming a simple per-token rate applies everywhere.

The CodeRabbit case study above is a genuinely useful reference point for total cost of adoption rather than raw inference pricing: roughly £67 (about $85) and two hours of NVIDIA's standard "auto" recipe to get a specialised router agent trained is a small fraction of what a comparable fine-tuning project on a larger, closed frontier model would typically cost, both in direct spend and in engineering time. For teams evaluating whether to specialise Lightning versus simply prompting a larger model well, that gap in customisation cost is arguably as important as the per-token inference price.

• Not a frontier-intelligence model: a score of 24 on the Artificial Analysis Intelligence Index sits well below larger open models and even NVIDIA's own Nemotron 3 Super (26) - do not deploy it for tasks that need deep, general reasoning.
• PinchBench accuracy trails Qwen3.6 35B: NVIDIA's own positioning chart shows Lightning winning decisively on speed, not on raw accuracy, against the models it compares itself to directly.
• Context window is deployment-dependent: the advertised 1-million-token window needs multi-GPU serving; a single H100 is memory-bound to roughly 256,000 tokens, which is still generous but a meaningfully different number.
• No unified official pricing: because it is open-weights, published $-per-token figures come from third-party hosts (OpenRouter, cloud partners) rather than a single NVIDIA price list, so costs vary by provider and can change independently of NVIDIA's own announcements.
• NeMo Switchyard is new infrastructure, not a drop-in guarantee: its cost-saving numbers (21%-74% in partner results) come from a small set of early launch partners and depend heavily on how well an agent workflow's steps are actually separable by difficulty - poorly structured workflows will see smaller gains.
• This article covers a same-week launch: independent, large-scale, adversarial testing of both Lightning and Switchyard beyond NVIDIA's own benchmarks and early partner case studies was limited at time of writing; treat headline percentages as vendor-reported until broader third-party verification accumulates.

Nemotron 3.5 Lightning is not really competing head-on with the larger open-weights releases getting attention this same week - GLM 5.3, Qwen 3.8 Max and DeepSeek V4 Pro are all substantially larger models chasing frontier-adjacent general intelligence, and all comfortably outscore Lightning on the Artificial Analysis Intelligence Index. Lightning occupies a different niche entirely: the small, cheap, fast worker model that a router - like NeMo Switchyard, or a hand-rolled equivalent - calls for the bulk of an agent pipeline's steps, while reserving a larger model for the handful of steps that actually need it.

That makes the more useful comparison an architectural one rather than a leaderboard one. Readers evaluating DeepSeek Harness or other open agentic coding tools should note that Lightning is a plausible cheap "worker" model to slot underneath such a harness for routine steps, with a heavier model reserved for the genuinely hard parts of a task - which is precisely the pattern NVIDIA's own NeMo Switchyard example demonstrates by pairing Lightning with Claude Opus 4.8.

Within NVIDIA's own Nemotron family, Lightning is a clear step up from Nemotron 3 Nano on every agentic and terminal metric NVIDIA published, while remaining deliberately smaller and less capable than Nemotron 3 Super. Judge it on cost-per-task-at-scale and latency, not on how it ranks against models many times its active parameter count on a general intelligence chart - that comparison was never the point of the release.

Worth using now if you are building or running multi-step agent pipelines with a high volume of repetitive, well-defined sub-tasks - tool calls, document parsing, classification, routine drafting - and want to cut the cost and latency of the steps that do not need frontier-level reasoning. It is also a strong candidate if you want a genuinely open, single-GPU-runnable model for local or edge agent deployment, or if you are specifically interested in pairing it with NeMo Switchyard to automate the routing decision rather than hand-coding it.

Worth skipping if you need one model to handle both the easy and hard steps of a workflow well, or if your use case is dominated by tasks that genuinely require broad general knowledge or deep reasoning - Lightning's own benchmark numbers put it behind larger open models like Qwen3.6 35B and NVIDIA's own Nemotron 3 Super on exactly those measures. In that case, a larger model used directly, or Lightning paired with a stronger model via a router, will serve you better than Lightning alone.

Nemotron 3.5 Lightning is a narrowly and honestly scoped release: NVIDIA is not claiming it beats the frontier, or even the rest of the current open-weights field, on general intelligence - its own comparison charts show it clearly behind larger models. What it does claim, and backs with real benchmark numbers from its own team and independent verification from Artificial Analysis, is that it is meaningfully faster and cheaper than comparably sized open models at the specific, repetitive work that dominates real agent pipelines, while remaining fully open and small enough to self-host on a single GPU.

The more interesting product story here may actually be NeMo Switchyard: a small, fast, cheap model is only as useful as the system deciding when to call it instead of something bigger, and NVIDIA shipping both pieces together - plus early real partner adoption from names like Cognition AI, LangChain and Siemens - suggests NVIDIA is betting on "intelligent routing across a mix of models" becoming as standard a piece of agent infrastructure as the models themselves. Worth adopting for the specific workload it targets; not a reason to skip evaluating larger open models for anything that needs genuine reasoning depth.