High-Throughput Batch Inference
// momentum
// why it's here
- Active: last commit today
- Built almost entirely by one person
- Health 90/100 with only 22 stars
// how to help
// readme
BatchGen
High-throughput batch inference for large MoE models
Selected results ยท Support matrix ยท Deployment guides ยท Batch API ยท Contributing ยท Paper