My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug.
20× Less waiting per turn 1024 Gemma’s sliding window, in tokens 12/12 Eval tasks passed on a 550-token prompt 🧰The […]
My local AI was pausing 7 seconds before every reply. It turned out to be one cache bug. Read Post »





