Offline
day
week
Real-time LLM Inference on Standard GPUs: 3k tokens/s per request
104 points
|
blog.kog.ai
|
NicoConstant
|
5hrs