DeepSeek open-sources production inference engine a day after US removal order

Summary

• DeepSeek released DeepInfer, its production inference framework, under Apache 2.0 on October 20, 2026.
• The engine runs DeepSeek-V4.1-Flash on Nvidia SM100 chips and Huawei Ascend 950 accelerators.
• Release landed one day after Homeland Security ordered federal civilian agencies to strip DeepSeek products.

The latest

DeepSeek has published the full source code of DeepInfer, the inference framework its repository documentation describes as powering the company's large-scale production services, under an Apache 2.0 license. The release covers offline inference for a single model, DeepSeek-V4.1-Flash, and runs on both Nvidia and Huawei silicon. The company said on X it is sharing its production stack to cut compute costs and energy use.

Details

• The license: DeepInfer is released under Apache 2.0, a permissive license carrying no fee and no restriction on commercial deployment, at github.com/deepseek-ai/DeepInfer. A companion library, deepseek-recipe, is published separately under MIT and converts Chat Completions, Responses and Messages requests into DeepSeek's own format.
• Model support: The framework supports DeepSeek-V4.1-Flash only, pulling the deepseek-ai/DeepSeek-V4.1-Flash-1017 checkpoint from Hugging Face on first run unless a local path is given. DeepSeek said inference code for upcoming models and further optimizations will be committed to the same repository.
• The hardware: Two accelerator families are supported: Nvidia SM100 processors and Huawei Ascend 950 units. Requirements are Linux, GCC 13.0 or higher, Python 3.12, and either CUDA 13 or CANN 9.2 — making the Huawei path a route around US chip supply for operators outside the Nvidia ecosystem.
• What's inside: The stack bundles optimized kernels and a runtime built on FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels. It adds attention data parallelism, balanced context parallelism, and MoE expert parallelism with EPLB load balancing across experts.
• Speed features: DSpark speculative decoding proposes up to 5 tokens per step with confidence-scheduled verification. Limited SWA re-run reduces prefill work in the decoder, and asynchronous scheduling is on by default. Fixed-batch inference and deterministic sampling are Nvidia-only. Generation can be constrained through regex, JSON schema and EBNF grammars.
• The gaps: This release covers offline inference. Not included: a KV cache memory system, full prefill-decode disaggregation, and load balancing across prefill and decode parallel replicas. The documentation states most of the engine functionality required for disaggregation is already implemented.
• The benchmarks: The documentation points to Pareto curves for prefill and decode without publishing figures in text. Measurements were taken on 4 SM100 processors for prefill and 64 for decode, each within a single scale-out domain, with examples/sweep/run_sweep.sh provided for reproduction.
• Early traction: By the morning of October 22 the repository had drawn 17,900 stars and 1,800 forks, against just 4 commits on the main branch and no numbered releases — adoption interest running well ahead of the project's public development history.
• The Washington order: One day before the release, the Department of Homeland Security issued directive 2026-21447 instructing federal civilian agencies to remove DeepSeek products and every third-party service built on them.

Background

DeepSeek has repeatedly released core infrastructure rather than models alone, and several DeepInfer components — FlashMLA, DeepGEMM, DeepEP — were already published separately. DeepInfer packages them into the serving stack the company says it runs in production.

Between the lines

Shipping Huawei Ascend 950 support alongside Nvidia means operators barred from US accelerators can run the same production path DeepSeek uses. Arriving a day after the Homeland Security removal order, the Apache 2.0 terms put the stack beyond any single government's procurement reach: agencies can delete the service, but the code and its deterministic-sampling and structured-output features remain freely redeployable.

What's next

Watch whether the KV cache system and full prefill-decode disaggregation land in the same repository, whether support widens beyond V4.1-Flash, and whether other federal or allied agencies extend the Homeland Security directive.

Source: DeepSeek (GitHub), DeepSeek on X, US Department of Homeland Security