Pokee-Isaac 28B Review: A Real 10M-Token Local Model

• Released 5 August 2026 by Pokee AI, a 28-billion-parameter, non-decoder-only agentic foundation model.
• 10-million-token context window, with 93.3% RULER accuracy at that length, the only model in the report's comparison panel to hold a non-zero score past 1M tokens.
• Runs on a single GPU. On one NVIDIA B200, it prefills at up to 137,200 tokens/second and decodes at roughly 335 tokens/second, flat regardless of how much context is resident.
• Leads its comparison panel on BFCL v4 function calling (70.94) and tau3-bench interactive tasks (0.662 average), and posts the safest DTAP security profile of six models tested.
• Second on Terminal-Bench 2.1 (65.1%) and third on MCP-Atlas (74.59%), behind GPT-5.6 Luna in both cases, so it is not a clean sweep.
• $0.15 / $1.00 per million input/output tokens, provisional, and licensed for VPC, on-premises or on-device deployment rather than shipped as open weights.
• Text-only for now. No image, audio or video input; coding was not a training priority for this first release; AMD hardware support is still in progress.

Pokee AI frames Isaac around two problems it says the rest of the industry has left unsolved. The first is data sovereignty: long-context, long-horizon agentic capability today is delivered almost exclusively from the cloud, because the compute cost of serving it any other way has been prohibitive. That disqualifies regulated industries, public-sector institutions and on-device applications outright, not on price but on the simple fact that their data is not permitted to leave a defined boundary. The second is affordability: under standard token-based pricing, cost scales steeply with context length, so long-horizon workflows that genuinely need a large window become financially unviable at scale even when a provider will sell you the tokens.

Isaac's pitch is that a 28B model, small enough to serve on a single GPU or a client workstation, can hold effectively all of that 10M-token window without degrading, which converts a variable, cloud-metered cost into a fixed one tied to hardware you already provisioned. Whether that pitch survives contact with independent testing is the open question this article does not have the standing to close, but it is worth being precise about what the report claims and what it actually measured, which is the rest of this piece.

Pokee AI describes Isaac as a non-decoder-only foundation model, a deliberate architectural departure from the standard decoder-only transformer that most contemporary LLMs use, engineered specifically to keep serving cost flat as context grows. The company has not published full architectural details in the way it has published evaluation methodology, but confirms that some of Isaac's weights are fine-tuned from Alibaba's Qwen3.6-27B, released under the Apache 2.0 licence, placing Isaac in the increasingly common category of models that build on an open base rather than training entirely from scratch.

The efficiency numbers back up the architectural claim in a way that is at least internally consistent: prefill throughput on a single NVIDIA B200 GPU actually rises with context length, from roughly 42,000 tokens/second at 1M tokens to 137,200 tokens/second at 10M, while decode throughput holds essentially flat at around 335 tokens/second across the entire range. A ten-fold increase in context costs the model roughly three times the time-to-first-token, not ten times or more, which is the specific behaviour that makes a 10M window usable in practice rather than merely addressable on paper.

Pokee AI evaluated long-context retrieval on two benchmarks against five baselines: GPT-5.6 Luna, Gemini 3.5 Flash Lite and Claude Haiku 4.5 (the cost-optimised tiers of the three major cloud providers), plus Nemotron 3 Super 120B and Qwen 3.5 122B (leading self-hostable open-weight models, both carrying more than four times Isaac's parameter count). Frontier flagships like GPT-5.6 Sol, Claude Opus 5 and Gemini 3.1 Pro were deliberately excluded as addressing a different, much more expensive deployment envelope.

On RULER, a synthetic benchmark that holds task difficulty constant while scaling context length, Isaac stays above 93.3% accuracy at every tested point from 256K up to the full 10M-token window. GPT-5.6 Luna and Gemini 3.5 Flash Lite track closely with Isaac up to 512K, then hit context-overflow errors from 1M onward and score 0.0% from 2M. Claude Haiku 4.5 and Qwen 3.5 122B score 0.0% across the entire sweep, a consequence of their native context windows (200K and 262K respectively) being too small to accept the test inputs at all. Nemotron 3 Super 120B, evaluated using NVIDIA's own self-reported figures up to 1M (a distinction the report is careful to label), degrades to 0.0% from 2M onward under Pokee AI's own testing.

The second test, MRCR v2 (Multi-Round Coreference Resolution, introduced by Google DeepMind), is a harder variant of needle-in-a-haystack that distributes multiple targets through a long synthetic conversation rather than hiding just one, explicitly penalising partial recall. Here Isaac again leads at every tested length, peaking at 0.743 at 512K and holding 0.500 at 1M. Gemini 3.5 Flash Lite is the closest competitor, but the gap widens as context grows, from a 0.133-point margin at 256K to 0.295 points at 1M, where Gemini has already dropped to 0.205. GPT-5.6 Luna, despite scoring 95.0% on RULER at 256K, collapses to 0.050 on MRCR v2 at 1M, a reminder that single-needle retrieval and multi-needle disambiguation are genuinely different capabilities that one score cannot substitute for the other.

BFCL v4: function calling

On the Berkeley Function-Calling Leaderboard v4, maintained by UC Berkeley's Gorilla group, Isaac scores 70.94 overall, ahead of GPT-5.6 Luna (70.61) and Claude Haiku 4.5 (67.52). The report itself is careful to characterise the 0.33-point margin over Luna as parity rather than a decisive win, framing the real claim as Isaac matching the strongest cost-optimised cloud baseline while remaining deployable on one GPU. Isaac's standout category is the live_parallel subtest (93.8, versus Luna's 56.2), while it trails on some multi-turn recovery categories like multi_turn_miss_param (44.5, versus Luna's 62.5).

tau3-bench: multi-turn interactive tasks

Sierra's tau3-bench simulates customer-service interactions across four domains against an LLM-simulated user rather than a fixed, pre-specified request. Isaac leads the panel with a 0.662 average across retail (0.789), airline (0.760), telecom (0.912) and the notoriously difficult banking domain (0.186, where the strongest reported result at release across the whole field was 25.5%). Its telecom score is particularly strong; that domain is diagnostic rather than transactional, requiring the agent to talk a simulated user with varying technical competence through physical fixes like reseating a SIM card.

MCP-Atlas: multi-server tool orchestration

MCP-Atlas is the one benchmark in the report that runs exclusively against real, independently maintained production MCP servers (GitHub, Slack, Google Workspace, Notion, Oxylabs and more, 36 in total) rather than mocks authored by the benchmark itself. Here Isaac places third at 74.59% coverage, behind GPT-5.6 Luna (77.90%) and Gemini 3.5 Flash Lite (76.67%). The efficiency detail worth noting: Isaac reaches within 2.1 points of Gemini's coverage while taking 9.10 tool turns per task against Gemini's 14.99, roughly 60% of the trajectory length for comparable results.

Terminal-Bench 2.1: agentic execution in a shell

Terminal-Bench places an agent at a Linux command line with no enumerated action space, just whatever the machine can run. Isaac places second at 65.1% (56 of 86 text-compatible tasks), four tasks behind GPT-5.6 Luna's 69.8%, but well clear of every open-weight baseline, including Qwen 3.5 122B and Nemotron 3 Super 120B despite both carrying far more parameters. This is the one benchmark in the report where a cloud baseline unambiguously finishes ahead of Isaac, and Pokee AI reports it as measured rather than reframing the loss.

Pokee AI evaluated Isaac's security behaviour on DTAP (DecodingTrust-Agent Platform), a red-teaming benchmark that places an agent in simulated real-world environments and measures whether injected attacks succeed, across 12 Linux-Docker domains and 6,195 judged tasks. Three rates matter: Direct ASR (attack success rate when a harmful request appears directly in the prompt), Indirect ASR (when the attack arrives through tool output or the environment itself), and BSR (benign task success rate, i.e. utility, where higher is better).

Isaac posts the lowest combined attack-success rate of the six models tested, at 35.6%, against Claude Haiku 4.5's 37.9%, GPT-5.6 Luna's 50.1%, and Gemini 3.5 Flash Lite's notably weaker 66.3%. Its Direct and Indirect ASR scores differ by only 0.8 points (36.0 vs 35.2), the tightest balance of any model tested; the report notes that models with lighter refusal training show the opposite pattern, with Nemotron 3 Super 120B, Gemini 3.5 Flash Lite and Qwen 3.5 122B all running 13 to 38 points higher on direct attacks than indirect ones. The trade-off is utility: Isaac's benign success rate of 82.5% places third of six, 2.6 points behind GPT-5.6 Luna and 0.8 points behind Gemini 3.5 Flash Lite, a fairly thin margin for what the report frames as a meaningfully safer profile.

List pricing of $0.15 per million input tokens and $1.00 per million output tokens is explicitly marked provisional in the report and subject to confirmation at launch. Against the two baselines that can actually be bought at long-context lengths, Isaac undercuts GPT-5.6 Luna by $0.25 on input and $0.80 on output, and Gemini 3.5 Flash Lite by $0.15 and $1.50, while sustaining accuracy across a window an order of magnitude larger than either. The other three baselines are not cheaper alternatives so much as unavailable at these context lengths altogether: Claude Haiku 4.5 caps at 200K and Qwen 3.5 122B at 262K, and while Nemotron 3 Super 120B has a 1M native window, no public endpoint Pokee AI could find serves it beyond 262K.

The bigger structural claim, and the one that is harder to independently verify from a price sheet alone, is that per-token pricing understates the difference for the deployments Isaac targets. Licensed for operation inside a VPC, on-premises, or on-device, its cost becomes a fixed function of provisioned hardware rather than a variable one that scales with tokens consumed, and no request crosses an external network boundary at all.

Portability is positioned as a first-class feature rather than an afterthought. Isaac reportedly runs fully on-device on the Intel Arc Pro B70 discrete GPU and on Intel Core Ultra Series 3 ("Panther Lake") client SoCs, with both prefill and decode executing entirely on the chip and no discrete accelerator required in the latter case. It has also been validated on Qualcomm's Snapdragon X2 Elite NPU-class mobile silicon. As one indicator of on-device quantised quality, Pokee AI reports Isaac scoring 0.9567 on the Pinchbench 116-task SuperClaw suite, ahead of the much larger GLM-5 (744B MoE, cloud-hosted, 0.929) and Qwen3-Coder-Next (80B MoE, local, 0.866).

Adaptation to a third accelerator family, AMD, is explicitly described as still in progress, and Pokee AI says per-chip throughput measurements for its supported platforms will follow in a subsequent revision once the respective hardware vendors sign off on their release. Treat the current portability story as real but partial: two of three major non-Nvidia accelerator families are covered today, the third is not yet.

Pokee AI is unusually direct about what this first release does not do, framing all three gaps below as explicit targets for the next version rather than a standing design choice.

• Text-only. No image, audio or video input. This is also why the report evaluates only 86 of Terminal-Bench 2.1's 89 tasks (excluding the three requiring non-text modalities) and only the text track of tau3-bench.
• Coding not yet optimised. Pokee AI states coding was not a training priority for this first model and the report includes no dedicated code-authoring benchmark; Terminal-Bench 2.1 is the nearest proxy, but it measures shell execution rather than pure code generation, so Isaac's placement there should not be read as either evidence of coding strength or its absence.
• Partial hardware adaptation. AMD accelerator support is still in progress; only Intel and Qualcomm silicon are validated today.
• Not open-weight. No public model weights, so nothing here can currently be independently reproduced by a third party on their own hardware; every figure in this article is Pokee AI's own internal measurement.
• Single-vendor evaluation harness. All comparative numbers, including the baseline models' scores, come from Pokee AI running its own harness. The report is candid that benchmark results are sensitive to harness, prompt template and sampling parameters, and explicitly notes discrepancies with some third-party-reported numbers for the same baseline models, which cuts both ways: it is a mark of transparency, but it also means no outside party has yet reproduced Isaac's own scores.

Against the two cost-optimised cloud baselines it beats most often, GPT-5.6 Luna and Gemini 3.5 Flash Lite, Isaac's edge is concentrated in exactly the place its architecture targets: retrieval fidelity past the 1-2M-token mark, where both cloud models simply stop returning usable output. Where Luna pulls ahead, Terminal-Bench 2.1 and MCP-Atlas, the margin is real but not huge (four tasks out of 86; roughly three points of coverage), and Isaac often gets there in meaningfully fewer tool-call turns.

Against the self-hostable open-weight alternatives in its own report, Nemotron 3 Super 120B and Qwen 3.5 122B, both of which carry more than four times Isaac's parameter count, the comparison is closer to a rout on long context specifically: both baselines drop to 0.0% on RULER well before 10M tokens, while Isaac holds 93.3%. For teams already evaluating small, efficient, agent-oriented models more broadly, Isaac sits alongside recent efficiency-tier releases like Ling 3.0 Tiny and LLaDA2.2-Flash, though Isaac is chasing a different specific capability (context length under agentic load) rather than general small-model efficiency.

Worth evaluating now: teams in regulated, sovereign or otherwise data-boundary-constrained environments (finance, public sector, healthcare) who need genuinely long-context agentic capability but cannot send data to a third-party cloud API; anyone running long-horizon workflows that repeatedly hit context-overflow limits on 1M-token cloud models today; and teams that want function-calling and multi-turn interactive performance competitive with cost-optimised cloud tiers, at a lower headline price and inside their own infrastructure.

Better to wait: anyone needing multimodal input, coding-specialised performance, or AMD on-device support, none of which this first release provides; and anyone who requires independently reproduced benchmark numbers before committing budget, since every figure currently in circulation is Pokee AI's own internal measurement.

Pokee-Isaac 28B's core claim, that a 28-billion-parameter model can hold retrieval fidelity across a genuine 10-million-token window on a single GPU, is exactly the kind of thing worth being sceptical of by default. What earns it a closer look is that Pokee AI's own technical report backs the claim with a fixed, disclosed evaluation harness, names the specific points where competitors fail (context overflow, native window limits) rather than picking favourable comparisons, and is equally explicit about where Isaac itself loses (Terminal-Bench 2.1, MCP-Atlas) and what it cannot yet do (vision, audio, dedicated coding, AMD hardware).

None of that substitutes for independent verification, and as of publication no third-party evaluator had reproduced these numbers. But for organisations that have been priced or policy-blocked out of long-context agentic AI entirely because the only capable models live behind a cloud API, a credibly-evaluated local alternative, even one still awaiting outside confirmation, is a genuinely new option rather than an incremental one.