Liquid AI RIPL: Sub-50 ms Inference Engine Explained

• What it is: RIPL, short for Real-time Inference Pipeline, is an engine for serving Liquid Foundation Models (LFM2.5) on NVIDIA GPUs inside live request paths.
• Speed claim: selected workloads return a full response in under 50 ms, with stable tail latency under continuous peak traffic.
• Benchmark claim: p95 latency is 5.6x to 28.4x lower than Qwen3.5 served on vLLM, on a single NVIDIA B200, run by Liquid.
• Models: LFM2.5 models from 230M to 2.6B parameters, plus multimodal LFMs and Liquid's d1 decision models.
• Customer: Shopify uses LFMs for search enhancement, with production inference reported under 20 ms, per Liquid.
• Availability: enterprise-only, by request, on H100, H200, B200 and RTX PRO 6000. No public price, download or open-source release.

The important word is 'path'. Liquid describes RIPL as an engine for models inside a live request path, meaning the chain of services that runs while a user waits: a search box, an ad auction, a card payment. In those systems a language model is not a chatbot. It is one component with a strict time budget, often a few tens of milliseconds, and if it is late the whole request is late.

That framing explains the design goals. Throughput, the number of tokens per second a GPU can produce in aggregate, is what most engines optimise. RIPL optimises latency under load, and in particular the tail: the slowest few per cent of requests. A system that averages 20 ms but occasionally takes 400 ms will break a 50 ms service-level objective, so tail latency, not the average, is the number that matters here.

Liquid says the supported models range from 230 million to 2.6 billion parameters. These are tiny by frontier standards, where models such as Kimi K3.1 Pro run to trillions of parameters. The bet is that many production tasks do not need a large model; they need a small model that is fast, predictable and cheap enough to call on every request.

Read the first row carefully. The 28.4x figure compares a 1.2-billion-parameter Liquid model on RIPL against a smaller 0.8-billion-parameter Qwen model on vLLM. So the number blends two effects: the engine and the model architecture. Liquid addresses this with a separate same-model test, running LFM2.5-350M on both engines, in which RIPL still comes out ahead, but the size of that gap is not in the figures we could access.

The test conditions also matter. Liquid used open-loop Poisson arrivals, which means requests arrive at random times regardless of whether the server has caught up. That is a realistic and demanding way to test, because queues build when the server is slow and tail latency explodes, which is exactly where a general-purpose engine tends to struggle at small batch sizes. It is also a setting that rewards an engine designed for it.

Liquid also reports quality figures for its small models: LFM2-2.6B scores 79.56% on IFEval and 82.41% on GSM8K. These are model benchmarks, not engine benchmarks, and they show the models are usable for instruction-following and simple maths but they do not say anything about frontier-level reasoning.

The chart plots latency in milliseconds against load for LFM2.5-350M on a B200. A dashed line marks a 50 ms budget. Two vLLM curves rise as load increases: the Qwen3.5-0.8B line climbs so fast that Liquid notes it goes off the top of the chart, while a second grey line crosses the 50 ms budget at moderate load and reaches roughly 170 ms. The purple RIPL line stays close to 20 ms across the range shown.

Two things to note. The image we embed is cropped, so the bottom of the chart and its legend are not visible, and we cannot tell you from the picture alone which grey line is which model or engine. The caption and the 'off chart' label identify the top line as Qwen3.5-0.8B on vLLM. Second, the chart is vendor-produced for a workload Liquid picked, which is normal for a launch but means it shows the best case, not the typical one.

• Ad retrieval: LFM2.5-350M, 32 input and 32 output tokens, expanding a query into advertiser keywords.
• E-commerce reranking: 1,000 input and 1 output token, under 8 ms end to end at up to 500 requests per second, per Liquid.
• Query intent classification: 128 input and 1 output token.
• Payment authorisation: LFM2.5-1.2B on a B200, 512 input and 16 output tokens.
• Credit decisioning: 8,000 input and 64 output tokens.
• Image alt text: LFM2.5-VL-450M on a 512 by 512 JPEG, 64 input and 16 output tokens.
• Decision models: Liquid's d1 models, which we covered in our d1 article.

Notice how many of these are classification or scoring tasks with one or a few output tokens. For those, the cost is dominated by processing the input, the so-called prefill, rather than generating text. An engine tuned for that profile can be dramatically faster than one balancing long conversations from many users.

This is also where RIPL overlaps with retrieval infrastructure. A reranker, like the one in our Cohere Rerank 5 article, scores candidate documents so a search system returns the right ones. Liquid is pitching small generative models into the same slot, with the extra flexibility of generating text when needed.

Treat it as a case study rather than an audit. The figures come from Liquid's blog, the comparison alternatives are not named, and 'under 20 ms' is a latency for one workload. It does, however, show the commercial logic: a retailer has millions of searches an hour, and shaving tens of milliseconds and a lot of GPU spend per request compounds quickly.

The reported 50% parameter saving matters as much as the speed. A smaller model needs fewer GPUs for the same traffic, and the saving appears directly in the infrastructure bill. This echoes the argument in our enterprise AI pricing analysis: the cost that matters is the cost of the system that does the job, not the price of a token.

That design is a poor match for a 50 ms budget on a tiny model. Batching helps throughput but adds waiting time, scheduling overhead is small for big jobs but large relative to a 20 ms response, and tail latency suffers when bursts arrive. A purpose-built engine can skip much of that machinery. This is analysis rather than something Liquid has detailed in the coverage we could access, but it is consistent with the gap Liquid reports.

It also means the comparison is not a verdict on vLLM. A fair reading is that RIPL is much faster for one narrow class of workload, and that for general serving of chatty, long-output models vLLM remains the sensible default. Teams should also compare against other low-latency stacks and against vendor-tuned engines on NVIDIA hardware, which Liquid did not report.

Consider a product search. A shopper types a query and expects results almost instantly. The page already needs time for the network, the search index, ranking, personalisation and rendering. If a language model is added to improve the query, it might be allotted 20 or 30 milliseconds out of a total budget of a few hundred. A cloud API call to a large model, which commonly takes hundreds of milliseconds to seconds, cannot fit. A small model on a local GPU can, but only if the serving software does not add its own delays.

This is why small language models are attracting attention from infrastructure teams rather than only from chatbot builders. The task is not to be impressive. It is to be correct enough, fast enough and cheap enough to run on every single request, millions of times a day. Cost compounds the same way as latency: a model that costs a fraction of a penny per call is acceptable at ten thousand calls and ruinous at ten billion.

The same logic applies to fraud checks and payment authorisation, where a decision must be made while the customer waits at a till or checkout. Missing the deadline means falling back to a rule-based system or declining the transaction, so a faster model is not a convenience but a requirement for using AI at all.

RIPL's pitch therefore is not that small models are smarter, but that they become deployable in places where nothing else could go. If the figures hold, the engine turns a 150 to 450 millisecond problem into a sub-50 millisecond one for the right model sizes, and that can be the difference between a prototype and a production feature.

A batching engine collects several requests and processes them together, because GPUs are far more efficient on a batch than on one item. That raises throughput but forces early arrivals to wait for later ones. For a long chat response the wait is invisible, because the answer takes seconds anyway. For a 20 ms task it is the dominant cost.

Tail latency is usually quoted as p95 or p99: the time within which 95% or 99% of requests complete. Liquid's headline figures are p95. A p95 of 155.8 ms means one in twenty requests took longer than that, and the gap between the median and the p95 shows how stable the system is. Under random arrivals, queues form in bursts, and engines without tight control of scheduling see their p95 balloon while the median still looks fine. Liquid's claim of stable tail latency under continuous peak traffic is therefore the more important half of the announcement than the raw speed.

For anyone reading vendor numbers, the practical rule is to ask for the distribution, not the average: median, p95 and p99, at several load levels, with your own request shapes. The numbers in Liquid's chart do that for a single workload, which is a good sign, but they are not a substitute for testing.

• Define the budget. Write down the end-to-end time you can spend and the percentile that matters. If you cannot state it, you do not yet have a latency problem that RIPL solves.
• Test quality first. A fast model that is wrong is useless. Run LFM2.5 models of two sizes on a labelled sample of your prompts, and compare against your current model on accuracy, not just speed.
• Ask for a same-model comparison. To isolate the engine, run the same model on RIPL and on a tuned open engine on the same GPU. That is the only fair engine test.
• Replay real traffic. Use recorded arrival times, including bursts, rather than a steady rate. Measure p50, p95 and p99.
• Cost the whole system. Licence, GPUs, engineering time and support. A smaller model on fewer GPUs may or may not outweigh a closed-engine fee.
• Check lock-in. A closed engine ties you to Liquid's models and to NVIDIA hardware. Decide whether you can switch back to an open stack if terms change.

Teams without a strict latency budget should probably wait. For them, the sensible default remains an open engine with a model sized to the task, and the broader pricing picture in our API pricing comparison shows how cheap capable models have already become.

Both movements point the same way: AI is becoming infrastructure, where the question is cost and delay per call rather than raw capability. When that happens, the engine matters as much as the model, and specialised serving software becomes a product in its own right.

There is a caution. Small models are easier to run but are weaker at open-ended reasoning, and they inherit the risks of any automated decision system, including bias and drift. For credit decisioning or fraud, where Liquid lists example workloads, regulators and customers will expect explanations and audit trails that a speed claim does not provide. Speed is necessary for these use cases but far from sufficient.

Watch for three signals: independent benchmarks that reproduce the numbers, a public price or an open release, and a second named production customer beyond Shopify. Until then, RIPL is a promising claim from a credible research company rather than an established standard.

• Vendor benchmarks only. Every figure is Liquid's, run on Liquid's chosen workloads. No independent reproduction exists yet.
• Models and engine are mixed. The 28.4x headline compares LFM2.5-1.2B on RIPL with Qwen3.5-0.8B on vLLM. A same-model test on LFM2.5-350M is reported, but its ratio is not in the coverage we could read.
• Closed and by request. There is no open-source release, no self-service download and no published pricing.
• NVIDIA only. Supported GPUs are H100, H200, B200 and RTX PRO 6000.
• Small models only. The range is 230M to 2.6B parameters, so it will not replace a frontier model for open-ended reasoning.
• The '20X' headline is not Liquid's. The published range is 5.6x to 28.4x, and creators rounding it into one number lose the context.

If you are evaluating it, ask Liquid for a same-model comparison on your own prompts, measure p95 and p99 under your real arrival pattern, and cost the full system including support and licensing. The headline ratio is where a conversation starts, not where it ends.

RIPL is a credible and interesting idea: a purpose-built engine for tiny models that have to answer inside a user's request. The published numbers, 5.6x to 28.4x lower p95 latency and sub-50 ms responses, are large enough to matter if they hold on your workload.

But they are Liquid's numbers, on Liquid's workloads, partly comparing different models, for a product you can only get by asking. Treat RIPL as a shortlist candidate for latency-bound, high-volume, short-output tasks, ask for a same-model test on your own data, and ignore the 20X headline.

• MarkTechPost: Liquid AI launches RIPL, a real-time inference engine for sub-50 ms LFMs
• AlphaSignal: Liquid AI RIPL engine beats vLLM by 28x on real-time inference
• Liquid AI: RIPL low-latency calculator
• Tilnote: Liquid AI RIPL analysis (Korean)