Legion

Legion is an LLM inference engine for mixture-of-experts models on NVIDIA unified-memory systems. It is Rust first: the host side is Rust, kernels are PTX where we can, and C/C++ only where there is no other choice (ggml provides the quantized kernels underneath).

• Models. First-class: Qwen3.8-Flash-Next (Qwen4 architecture), GLM5.3-Flash and DeepSeek4.1-Flash. Also supported: Gemma 4, Qwen3.8-27B and smaller MoEs such as Nemotron3.5-30B-A3B, Qwen3.6-35B-A3B and Gemma4-26B-A4B.
• Hardware. Built for unified memory: very good support for the IBM AC922 (V100 over NVLink 2.0 + ATS) and the DGX Spark (GB10). Models bigger than VRAM keep their experts in host memory, read in place or cached in VRAM by heat. Minimum GPU: sm_70 (Volta). -…