1bit engine: local LLM inference on AMD Strix Halo (Ryzen AI MAX+ 395)

One binary to rule them all: the 1bit engine is a model-agnostic local LLM inference engine for the AMD Ryzen AI MAX+ 395 (Strix Halo) on Linux. It serves any GGUF on the Radeon 8060S iGPU (gfx1151) through HRX with our own HIP kernels, and models on the XDNA 2 NPU through our own NPU runtime, as a backend of AMD’s Lemonade Server and its OpenAI-compatible API. Boot a USB stick, and the machine serves local LLMs on its GPU and NPU. That is the whole setup.

Get started

1. Download 1bit-os-<version>.img.zst from the releases page.
2. Write it to a USB stick (8 GB or more).
3. Boot the machine from the stick. Secure Boot can stay on; the image is signed.
4. Lemonade comes up with its catalog and web UI. Try it from the live session; to keep it, run 1bit-install <disk> --yes there and the 1bit appliance is on the disk (os/v2/README.md).

Already running Lemonade on Ubuntu? The lemonade-onebit…