Inferencer v2.0.1 with Multiprocessing

In version 2.0.1, we added a multiprocessing engine for running multiple models at the same time, expanded multimodal model support, and brought major performance improvements to DeepSeek V4.

Multiprocessing Engine

This release introduces a multiprocessing engine that allows Inferencer to run multiple models at the same time. This opens up more flexible local AI workflows, such as keeping a fast model ready for routine requests while a larger model remains available for more complex tasks, or running multiple models side by side for comparison and agent-style routing.

More Multimodal Models

We also added support for several new models, including Nemotron 3 Ultra, Nemotron Omni, Step 3.7 Flash Vision, Granite Vision, and Qwen3-Omni. These additions expand Inferencer’s support for multimodal and omni-style models, making it easier to work with models that can handle more than just plain text.

DeepSeek V4 Performance

DeepSeek V4 received a significant performance update in this release. We added MTP speculative decoding and batching support, which should improve throughput and make longer generations more efficient. We also fixed verbose thinking behaviour, helping DeepSeek V4 produce cleaner and more predictable output.

Server API and Multimodal Fixes

We made improvements to Server API tool calls, helping make agentic workflows more reliable when interacting with Inferencer over the server interface. We also fixed multimodal inference end token and cache issues, improving stability for models that process images, audio, or other non-text inputs.

More Improvements

In this release, we’ve also:

• Fixed distributed compute detection
• Improved persistent prompt caching for faster repeated interactions
• Improved performance for batching cancellations
• Improved Markdown rendering
• Added more bug fixes and performance improvements

As always, if you have any features or suggestions you’re more than welcome to add them to our public roadmap.

Thanks again for your support, and as a reminder, all AI processing is done offline, directly on your devices. No telemetry, no background "update" checks.

P.S. If you find Inferencer useful, please consider leaving a review on the App Store. It would be much appreciated.