Inferencer v2.5.0

Inferencer with Faster Model Streaming

In version 2.5.0, we added support for N-gram, Engram and Expert streaming for faster streaming performance on memory constrained devices.

N-gram and Engram Streaming

DeepSeek V4.1 Flash and Qwen3.8 Flash Next now support N-gram and Engram streaming, allowing the “s” suffix versions of the models to be run with up to 280 GB in memory savings at no loss in quality or performance.

Expert Streaming

Expert streaming has also been added to “se” edition models, allowing the full DeepSeek model to be run on memory constrained devices such as the M1 Max 64GB at a much faster ~2 t/s and M5 Max 128GB at ~7 t/s compared to the previous ~1 t/m layer streaming technique.

Muse Spark and MiniMax M3.1

Support for leading open-weight models MiniMax M3.1 Flash and Muse Spark has also been added.

More Improvements

In this release, we’ve also:

• Added Kolibri 1 support
• Improved multi-node clustering stability and caching
• Improved image caching reliability
• Added multi-processing support for MTP
• Added relative path support and WebGL error reporting for the HTML viewer
• Fixed additional bugs and improved performance

Thanks again for your support, and as a reminder, all AI processing is done offline, directly on your devices. No telemetry, no background "update" checks.

P.S. If you find Inferencer useful, please consider leaving a review on the App Store. It would be much appreciated.