DeepInfer DeepInfer is the high-performance inference framework that powers DeepSeek’s large-scale production services. This open-source release provides offline inference with the core features needed for serving at scale. Supported models: DeepSeek-V4.1-Flash Supported features: Optimized kernels, model and sampler runtime Parallelism: Attention DP / balanced CP, MoE EP + EPLB DSpark and confidence-scheduled verification SWA bounded replay Asynchronous scheduling (zero-overhead batch scheduler) Batch-invariant inference Deterministic sampling Structured outputs Feature details and configuration Optimized kernels: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels. DSpark and confidence-scheduled verification: accelerate decoding with draft tokens and…
DeepSeek just open-sourced its inference engine
Advertisement