Tiny-vLLM

high performance LLM inference engine in C++ and CUDA

Tiny-vLLM was launched on Hacker News as "Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA". It received 205 points and active community acclaim.