artlite
Newborn Newborn Python PY
↑ 46 ?
artlite artlite-project/artlite

AI Engine RunTime, lite: language models and image generation on the NPU of AMD Ryzen AI processors

HEALTH ? 86 / Healthy
// solo builder// hidden gem amdamdxdnagemmaggufllamallm-inferencemlir-aienpuqwenryzen-ai ?

// momentum

tracking since W47

// why it's here

  • Created 4 weeks ago and already shipping
  • Built almost entirely by one person
  • Health 86/100 with only 15 stars

// how to help

// readme

Artlite runs LLMs and Stable Diffusion on the XDNA 2 NPU (an AIE2P AI Engine array) of AMD’s Strix Point, Krackan Point and Strix Halo processors. Its runtime is a small C++ library that talks to the amdxdna driver directly, in place of XRT; on Windows 11 the same library runs through XRT.

Its kernels are compiled at run time, as with NVRTC and HIPRTC, from sources generated for the task: each operator renders C++ specialized for its shapes, data formats and place on the array, plus an IRON design of the array’s data movement. oneDNN generates such kernels (brgemm, fused graph partitions) as machine code with xbyak; Artlite generates C++ and lets Peano, LLVM for AI Engine, optimize it, steered by hints chosen per loop. Everything is tuned for this one device rather than for portability.

Read the full README on GitHub ↗