Hacker News Logo

Offline

dayweek

Real-time LLM Inference on Standard GPUs: 3k tokens/s per request

104 points|blog.kog.ai|
NicoConstant|5hrs