AI Engine RunTime, lite: language models and image generation on the NPU of AMD Ryzen AI processors
// momentum
// why it's here
- Created 4 weeks ago and already shipping
- Built almost entirely by one person
- Health 86/100 with only 15 stars
// how to help
// readme
Artlite runs LLMs and Stable Diffusion on the XDNA 2 NPU (an AIE2P AI Engine array) of AMD’s Strix Point, Krackan
Point and Strix Halo processors. Its runtime is a small C++ library that talks to the amdxdna driver directly, in
place of XRT; on Windows 11 the same library runs through XRT.
Its kernels are compiled at run time, as with NVRTC and HIPRTC, from sources generated for the task: each operator renders C++ specialized for its shapes, data formats and place on the array, plus an IRON design of the array’s data movement. oneDNN generates such kernels (brgemm, fused graph partitions) as machine code with xbyak; Artlite generates C++ and lets Peano, LLVM for AI Engine, optimize it, steered by hints chosen per loop. Everything is tuned for this one device rather than for portability.