Ollama – v0.30.0-rc16
Ollama v0.30.0-rc16 🛠️
If you’re running local LLMs, you know Ollama is the go-to for getting models like Llama 3 and DeepSeek-R1 up and running with zero friction. This latest release candidate is a focused update aimed at squeezing more efficiency out of your hardware!
What’s new:
- Batch Size Tuning: The big headline here is the ability to tune batch sizes. This is a huge win for anyone trying to optimize inference speed and squeeze every bit of performance out of their GPU or CPU setup. 🚀
Fine-tuning these parameters can make a massive difference in throughput, especially when you’re experimenting with larger models on limited VRAM! Perfect for those of us pushing our local rigs to the limit.