Newborn 
C A dependency-free C inference engine for Liquid AI's LFM2 / LFM2.5 models. Terminal chat, an OpenAI-compatible server, a web agent and a terminal agent, all in one executable.
https://benmaster82.github.io/Pithos/ ↗// readme
Terminal chat, an OpenAI-compatible server, a web agent and a terminal agent, all in one executable.
| About 1.9× faster than llama.cpp | 17–22 tok/s against 11 tok/s with CUDA, on the same laptop and model (head-to-head) |
| Runs a 13.5 GB model on 16 GB of RAM | experts stream from disk, the hot ones live in a 4 GB GPU |
| Agents included | Pandora in the browser, and a terminal agent that runs commands, fixes code and checks its own work |
| Runs on an Xbox One | an OpenAI-compatible server for your home network, with LFM3-1.7B on the console’s GPU: 29.5 tok/s generation, ~140 tok/s prompt (Xbox One) |
Pithos is an inference engine written from scratch in C11 with no dependencies: no external
libraries at build time, only the operating system API (Win32 or POSIX). It runs GGUF models of
Liquid AI’s LFM2 / LFM2-MoE / LFM3 architectures and is tuned for LFM2.5-24B-A2B-Q4_K_M.gguf
(24 billion parameters, about 2 billion active per token) on an ordinary laptop. The small
LFM3-1.7B runs too: about 55 tokens per second with the GPU, 20 on the CPU alone.…