Three days after open-sourcing the model, DeepSeek open-sourced the machine that serves it: a production inference engine under Apache 2.0, serving economics no other lab publishes — 155 trillion tokens a day from one cluster, US$38 revenue per card-hour — and a hardware list that names NVIDIA and Huawei in the same breath.
The short version.
DeepInfer, the framework behind DeepSeek’s production traffic, is now open source under Apache 2.0 — serving exactly one model today, DeepSeek-V4.1-Flash-1017 (weights: MIT, three days ago).
The same engineering post published numbers no other lab discloses: one cluster serving 155 trillion tokens a day, US$38 revenue per card-hour (company-reported, valley-price basis).
The engine is open; the cluster machinery — KV cache, prefill/decode disaggregation, load balancing — is described but not shipped. Our read: the economics line, not the code, is what competitors should study.
Editor’s note. Three days ago we covered V4.1-Flash-1017, the weights (issue #1). This morning DeepSeek shipped the other half of the story: DeepInfer, the in-house framework that actually serves that model to production traffic. The announcement came as a four-post thread on X; the substance lives in a same-day first-party engineering post on Zhihu and in the repository itself. Both are linked at the bottom; every fact below traces to them.
Propagation, for the record — heat data only, never facts: roughly twelve hours after the morning announcement, the launch post showed about 18K likes, 2K reposts and 1.2M views, slower than the model launch three days ago (2.2M views in five hours). Which fits the material: an inference framework is news for a narrower audience than a frontier model — and that narrower audience is who this piece is written for.
What happened, verified
Everything in this section comes from DeepSeek’s own announcement, the DeepInfer repository README, and DeepSeek’s engineering post on Zhihu — first-party, published October 20, 2026.
1. The release. DeepSeek announced DeepInfer, the inference framework powering its large-scale production services, now fully open source under Apache License 2.0 on GitHub. The stated intent, in DeepSeek’s own words: share the production stack “to help the community lower compute costs, reduce energy use, and make AI more affordable.” One model is supported in this release: DeepSeek-V4.1-Flash-1017, the checkpoint released three days ago. The acknowledgments section is unusually candid about lineage: DeepInfer was “originally built primarily on vLLM,” with additional inspiration from SGLang and TensorRT-LLM.
2. What’s in the box. An engine plus a kernel stack. The full feature surface, traced to the README:
CapabilityWhat it isOptimized kernelsFlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, custom TileLang kernelsParallelismAttention DP / balanced CP; MoE expert parallelism with expert-placement load balancingDSpark speculative decodingDraft tokens with confidence-scheduled verificationSWA bounded replayRecompute a bounded suffix instead of a full window rebuildAsynchronous scheduling”Zero-overhead batch scheduler”Batch-invariant inference, deterministic samplingIdentical results across batch compositions (NVIDIA only)Structured outputsConstrained to regex, JSON schema or EBNF grammar
Prerequisites are strict — Linux, GCC ≥ 13.0, Python 3.12, CUDA 13 (NVIDIA) or CANN 9.2 (Ascend) — and the published benchmarks were measured on 4 SM100 GPUs for prefill and 64 for decode, each within one scale-up domain.
3. The numbers nobody else publishes. The Zhihu post states that a single cluster serves 155 trillion tokens per day, and that revenue is US$38 per card per hour — with the post’s own parenthetical qualifier, 按谷价 (”at valley-price” periods). The system’s three stated design goals: run on multiple chip vendors; prefill on ordinary 8-GPU machines, decode on 64–128-card supernodes; retain KVCache for 72+ hours to make prefix reuse pay. Supporting detail from the same post: the KVCache pool lives on a 3FS storage cluster with roughly 100 PiB of SSD, agent-traffic cache TTL of 3 days and chatbot TTL of 7 days, peak aggregate read bandwidth of 10 TiB/s — and an explicit accounting of the cost of cheaper retention (a 3-day TTL costs chatbot traffic 0.5–1% of hit rate).
4. The quieter companion release. Alongside the engine, DeepSeek open-sourced deepseek-recipe, a production-grade token codec that converts between mainstream API protocols (Chat Completions, Responses, Messages) and model token sequences. Its handling logic is stated to align exactly with the official API, so third-party deployments reproduce the official API experience — including the security-relevant details: special-token strings typed by users are encoded as ordinary text, preventing token-injection control of model behavior. The core is Rust, with Python bindings.
5. What is not in the release. The README is explicit: a complete inference service still requires a KV cache system, VE/prefill/decode disaggregation, and load balancing across data-parallel replicas — none of which ship here. The Zhihu post explains why: those components couple deeply to internal infrastructure. It then describes exactly what stays inside: bucketed prefill routing driven by a fitted quadratic cost model; decode balancing across request count and KVCache length; expert-parallel rebalancing via redundant experts (measured imbalance 1.03–1.09); prefill/decode auto-scaling with a 1-minute grace period on instance drain. The engine is open. The cluster choreography is described, not shipped.
The model was the teaser. The economics are the story.
The rest of this post is our analysis, clearly separated from the facts above.
First: DeepSeek is open-sourcing its serving path layer by layer — and publishing the cost basis underneath it. Weights went MIT on October 17; the engine and the token codec went Apache 2.0 today. Anyone with the hardware can now run the exact production engine, and anyone modeling serving costs has a published anchor: US$38 revenue per card-hour. No Western lab publishes that number. It recontextualizes the pricing we covered in issue #1 — the RMB 5 search API and the Flash price point are being set by an operator willing to show its per-card arithmetic. Caveats, stated plainly: company-reported, single-cluster, and the “valley-price” qualifier is the post’s own; we don’t know the averaging window, so treat it as a disclosed anchor, not an audited average.
Second: the release is deliberately engine-shaped, not cluster-shaped. What’s absent — KV cache, disaggregation, cross-replica load balancing — is precisely what turns engines into a 100-PiB, 10-TiB/s operation. The engineering post’s most interesting disclosure is how much of that machinery is described rather than shipped: the SWA bounded replay trick (recompute only the last 128 tokens instead of a full layer-by-layer window rebuild, cutting decoder prefill work nearly in half), the decision not to adopt the V4 tech report’s “Zero SWA Caching” because its recomputation cost was too high in production, DSpark’s variable-length verification (+6% throughput over fixed-length draft verification). Third parties get the hard part free and the harder part as documentation. That gap is the moat that remains when everything else is open.
Third: the hardware line is the quiet headline. “NVIDIA SM100 GPUs” and “Huawei Ascend 950 NPUs” appear as co-equal targets in one README, and multi-vendor chips are design goal #1 of the production system. Whatever your export-control priors, the serving engine of China’s most-watched AI lab now treats NVIDIA and Huawei as interchangeable build targets, and the open release carries both. One honest gap: the post’s production numbers do not break out which hardware they were measured on, so Ascend’s production parity remains an open question — the repo listing it is evidence of intent, not of measured equality.
Can you use it outside China?
The code: yes. GitHub, Apache 2.0, build from source. No geographic restriction is stated in the repository.
The hardware: partly, at a price. SM100 is current top-bin NVIDIA silicon — globally available, expensive. Ascend 950 is Huawei’s latest NPU; our read is that practical availability outside China remains limited.
The scope: know what you’re getting. This is an offline-inference engine plus kernels, not a turnkey serving platform — the README itself lists the missing pieces. It is not yet a drop-in replacement for your vLLM or SGLang deployment.
What to do with it now
Benchmark it. Build DeepInfer, run the offline example against V4.1-Flash-1017, and compare against your current vLLM/SGLang stack on identical GPUs. The README’s 4-GPU-prefill / 64-GPU-decode benchmark shape is the reference curve.
Adopt deepseek-recipe if you self-host DeepSeek models behind an API facade — it is the piece that makes third-party deployments behavior-identical to the official API, including the anti-injection tokenizer behavior.
File the US$38 number, with its caveats. The next time a DeepSeek price cut lands, the question to ask is whether the disclosed per-card arithmetic makes it inevitable rather than aggressive.
Watch the commit log. The engineering post says future models’ inference code and performance work will keep landing in DeepInfer — the repo is now a leading indicator of unannounced models.
Track what’s still closed. If KV cache, disaggregation and load balancing ever open, self-hosting DeepSeek at full API parity becomes an engineering project instead of a research program.
What would change our read
Independent operators cannot reproduce meaningful throughput or latency gains over vLLM/SGLang on identical hardware.
The US$38 per-card-hour figure proves unrepresentative — a cherry-picked cluster or window — and future disclosures stop being consistent with it.
The Ascend path lags materially in practice, or production numbers turn out to be NVIDIA-only.
The repository goes stale: future model code fails to land in DeepInfer as promised.
Sources and method
Announcement and propagation data: DeepSeek on X, October 20, 2026 — the launch thread (4 posts); post metrics observed ~12 hours after publication and used only as heat evidence.
Release facts (license, features, hardware, exclusions): DeepInfer README, GitHub — first-party repository documentation. Companion repo: deepseek-recipe.
Architecture and production numbers (Chinese, first-party): DeepSeek V4.1 Flash 大规模推理部署实践, Zhihu, October 20, 2026 — the throughput, revenue, KVCache and load-balancing facts trace here. Its inline figures are images we do not republish; every number we cite from it appears in its text.
Facts and judgments are separated by section above; social metrics are named as such. Corrections welcome — reply to this email and we’ll log the change.
AIGC.NEWS — China AI, from the inside. Verified from primary Chinese sources.