v3.7.0
Welcome to LocalAI 3.7.0 :wave:
This release introduces Agentic MCP support with full WebUI integration, a brand-new neutts TTS backend, fuzzy model search, long-form TTS chunking for chatterbox, and a complete WebUI overhaul.
We’ve also fixed critical bugs, improved stability, and enhanced compatibility with OpenAI’s APIs.
📌 TL;DR – What’s New in LocalAI 3.7.0
| Feature | Summary |
|---|---|
| 🤖 Agentic MCP Support (WebUI-enabled) | Build AI agents that use real tools (web search, code exec). Fully-OpenAI compatible and integrated into the WebUI. |
| 🎙️ neutts TTS Backend (Neuphonic-powered) | Generate natural, high-quality speech with low-latency audio — ideal for voice assistants. |
| 🖼️ WebUI enhancements | Faster, cleaner UI with real-time updates and full YAML model control. |
| 💬 Long-Text TTS Chunking (Chatterbox) | Generate natural-sounding long-form audio by intelligently splitting text and preserving context. |
| 🧩 Advanced Agent Controls | Fine-tune agent behavior with new options for retries, reasoning, and re-evaluation. |
| 📸 New Video Creation Endpoint | We now support the OpenAI-compatible /v1/videos endpoint for text-to-video generation. |
| :snake: Enhanced Whisper compatibility | Whisper.cpp is now supported on various CPU variants (AVX, AVX2, etc.) to prevent illegal instruction crashes. |
| 🔍 Fuzzy Gallery Search | Find models in the gallery even with typos (e.g., gema finds gemma). |
| 📦 Easier Model & Backend Management | Import, edit, and delete models directly via clean YAML in the WebUI. |
| ▶️ Realtime Example | Check out the new realtime voice assistant example (multilingual). |
| ⚠️ Security, Stability & API Compliance | Fixed critical crashes, deadlocks, session events, OpenAI compliance, and JSON schema panics. |
| :brain: Qwen 3 VL | Support for Qwen 3 VL with llama.cpp/gguf models |
🔥 What’s New in Detail
🤖 Agentic MCP Support – Build Intelligent, Tool-Using AI Agents
We're proud to announce full Agentic MCP support a feature for building AI agents that can reason, plan, and execute actions using external tools like web search, code execution, and data retrieval. You can use standard chat/completions endpoint, but powered by an agent in the background.
Full documentation is available here
✅ Now in WebUI: A dedicated toggle appears in the chat interface when a model supports MCP. Just click to enable agent mode.
✨ Key Features:
- New Endpoint:
POST /mcp/v1/chat/completions(OpenAI-compatible). - Flexible Tool Configuration:
mcp: stdio: | { "mcpServers": { "duckduckgo": { "command": "docker", "args": ["run", "-i", "--rm", "ghcr.io/mudler/mcps/duckduckgo:master"] } } } - Advanced Agent Control via
agentconfig:agent: max_attempts: 3 max_iterations: 5 enable_reasoning: true enable_re_evaluation: truemax_attempts: Retry failed tool calls up to N times.max_iterations: Limit how many times the agent can loop through reasoning.enable_reasoning: Allow step-by-step thought processes (e.g., chain-of-thought).enable_re_evaluation: Re-analyze decisions when tool results are ambiguous.
You can find some plug-n-play MCPs here: https://github.com/mudler/MCPs Under the hood, MCP functionality is powered by https://github.com/mudler/cogito
🖼️ WebUI enhancements
WebUI had a major overhaul:
- The chat view now has an MCP toggle in the chat for models that have
mcpsettings enabled in the model config file. - The Editor mask of the model has now been simplified to show/edit the YAML settings of the model
- More reactive, dropped HTMX in favor of Alpine.js and vanilla javascript
- Various fixes including deletion of models
🎙️ Introducing neutts TTS Backend – Natural Speech, Low Latency
Say hello to neutts a new, lightweight TTS backend powered by Neuphonic, delivering high-quality, natural-sounding speech with minimal overhead.
🎛️ Setup Example
name: neutts-english
backend: neutts
parameters:
model: neuphonic/neutts-air
tts:
audio_path: "./output.wav"
streaming: true
options:
# text transcription of the provided audio file
- ref_text: "So I'm live on radio..."
known_usecases:
- tts
:snake: Whisper.cpp enhancements
whisper.cpp CPU variants are now available for:
avxavx2avx512fallback(no optimized instructions available)
These variants are optimized for specific instruction sets and reduce crashes on older or non-AVX CPUs.
🔍 Smarter Gallery Search: Fuzzy & Case-Insensitive Matching
Searching for gemma now finds gemma-3, gemma2, etc. — even with typos like gemaa or gema.
🧩 Improved Tool & Schema Handling – No More Crashes
We’ve fixed multiple edge cases that caused crashes or silent failures in tool usage.
✅ Fixes:
- Nullable JSON Schemas:
"type": ["string", "null"]now works without panics. - Empty Parameters: Tools with missing or empty
parametersnow handled gracefully. - Strict Mode Enforcement: When
strict_mode: true, the model must pick a tool — no more skipping. - Multi-Type Arrays: Safe handling of
["string", "null"]in function definitions.
🔄 Interaction with Grammar Triggers:
strict_modeand grammar rules work together — if a tool is required and the function definition is invalid, the server returns a clear JSON error instead of crashing.
📸 New Video Creation Endpoint: OpenAI-Compatible
LocalAI now supports OpenAI’s /v1/videos endpoint for generating videos from text prompts.
📌 Usage Example:
curl http://localhost:8080/v1/videos \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-..." \
-d '{
"model": "sora",
"prompt": "A cat walking through a forest at sunset",
"size": "1024x576",
}'
:brain: Qwen 3 VL in llama.cpp
Support has been added for Qwen 3 VL in llama.cpp. We have updated llama.cpp to latest! As a reminder, Qwen 3 VL and multimodal models are also compatible with our vLLM and MLX backends. Qwen 3 VL models are already available in the model gallery:
qwen3-vl-30b-a3b-instructqwen3-vl-30b-a3b-thinkingqwen3-vl-4b-instructqwen3-vl-32b-instructqwen3-vl-4b-thinkingqwen3-vl-2b-thinkingqwen3-vl-2b-instruct
Note: upgrading the llama.cpp backend is necessary if you already have a LocalAI installation.
🚀 (CI) Gallery Updater Agent: Auto-Detect & Suggest New Models
We’ve added an autonomous CI agent that scans Hugging Face daily for new models and opens PRs to update the gallery.
✨ How It Works:
- Scans HF for new, trending models
- Extracts base model, quantization, and metadata.
- Uses cogito (our agentic framework) to assign the model to the correct family and to obtain the model informations.
- Opens a PR with:
- Suggested
name,family, andusecases - Link to HF model
- YAML snippet for import
- Suggested
🔧 Critical Bug Fixes & Stability Improvements
| Issue | Fix | Impact |
|------|-----|--------|
| 📌 WebUI Crash on Model Load | Fixed can't evaluate field Name in type string error | Models now render even without config files |
| 🔁 Deadlock in Model Load/Idle Checks | Guarded against race conditions during model loading | Improved performance under load |
| 📞 Realtime API Compliance | Added session.created event; removed redundant conversation.created | Works with VoxInput, OpenAI clients, and more |
| 📥 MCP Response Formatting | Output wrapped in message field | Matches OpenAI spec — better client compatibility |
| 🛑 JSON Error Responses | Now return clean JSON instead of HTML | Scripts and libraries no longer break on auth failures |
| 🔄 Session Registration | Fixed initial MCP calls failing due to cache issues | Reliable first-time use |
| 🎧 kokoro TTS | Returns full audio, not partial | Better for long-form TTS |
🚀 The Complete Local Stack for Privacy-First AI
LocalAI |
The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required. |
LocalAGI |
A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI. |
LocalRecall |
A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI. |
❤️ Thank You!
A huge THANK YOU to our growing community! With over 35,000 stars, LocalAI is a true FOSS movement — built by people, for people, with no corporate backing.
If you love privacy-first AI and open source, please:
- ✅ Star the repo
- 💬 Contribute code, docs, or feedback
- 📣 Share with others
Your support keeps this stack alive and evolving!
✅ Full Changelog
📋 Click to expand full changelog
What's Changed
Bug fixes :bug:
- fix(chatterbox): chunk long text by @mudler in https://github.com/mudler/LocalAI/pull/6407
- fix(grammars): handle empty parameters on object types by @mudler in https://github.com/mudler/LocalAI/pull/6409
- fix(mcp): register sessions by @mudler in https://github.com/mudler/LocalAI/pull/6429
- fix(llama.cpp): correctly set grammar triggers by @mudler in https://github.com/mudler/LocalAI/pull/6432
- fix(mcp): make responses compliant to OpenAI APIs by @mudler in https://github.com/mudler/LocalAI/pull/6436
- fix(ui): models without config don't have a .Name field by @mudler in https://github.com/mudler/LocalAI/pull/6438
- fix(realtime): Add transcription session created event, match OpenAI behavior by @richiejp in https://github.com/mudler/LocalAI/pull/6445
- fix: guard from potential deadlock with requests in flight by @mudler in https://github.com/mudler/LocalAI/pull/6484
- fix: handle multi-type arrays in JSON schema to prevent panic by @robert-cronin in https://github.com/mudler/LocalAI/pull/6495
- fix: properly terminate llama.cpp kv_overrides array with empty key + updated doc by @blob42 in https://github.com/mudler/LocalAI/pull/6672
- fix: llama dockerfile make package by @blob42 in https://github.com/mudler/LocalAI/pull/6694
- feat: return complete audio for kokoro by @lukasdotcom in https://github.com/mudler/LocalAI/pull/6842
Exciting New Features 🎉
- feat: Add Agentic MCP support with a new chat/completion endpoint by @mudler in https://github.com/mudler/LocalAI/pull/6381
- fix: add strict mode check for no action function by @mudler in https://github.com/mudler/LocalAI/pull/6294
- feat: add agent options to model config by @mudler in https://github.com/mudler/LocalAI/pull/6383
- feat(ui): add button to enable Agentic MCP by @mudler in https://github.com/mudler/LocalAI/pull/6400
- feat(api): support both /v1 and not on openai routes by @mudler in https://github.com/mudler/LocalAI/pull/6403
- feat(ui): display in index when a model supports MCP by @mudler in https://github.com/mudler/LocalAI/pull/6406
- feat(neutts): add backend by @mudler in https://github.com/mudler/LocalAI/pull/6404
- feat(ui): use Alpine.js and drop HTMX by @mudler in https://github.com/mudler/LocalAI/pull/6418
- chore: change color palette such as is closer to the logo by @mudler in https://github.com/mudler/LocalAI/pull/6423
- chore(ui): simplify editing and importing models via YAML by @mudler in https://github.com/mudler/LocalAI/pull/6424
- chore(api): return json errors by @mudler in https://github.com/mudler/LocalAI/pull/6428
- chore(ui): display models and backends in tables by @mudler in https://github.com/mudler/LocalAI/pull/6430
- feat(ci): add gallery updater agent by @mudler in https://github.com/mudler/LocalAI/pull/6467
- feat(gallery): add fuzzy search by @mudler in https://github.com/mudler/LocalAI/pull/6481
- chore(gallery search): fuzzy with case insentivie by @mudler in https://github.com/mudler/LocalAI/pull/6490
- feat(ui): add system backend metadata and deletion in index by @mudler in https://github.com/mudler/LocalAI/pull/6546
- feat(api): OpenAI video create enpoint integration by @gmaOCR in https://github.com/mudler/LocalAI/pull/6777
- feat: add CPU variants for whisper.cpp by @mudler in https://github.com/mudler/LocalAI/pull/6855
- feat: do also text match by @mudler in https://github.com/mudler/LocalAI/pull/6891
🧠 Models
- chore(model gallery): add lemon07r_vellummini-0.1-qwen3-14b by @mudler in https://github.com/mudler/LocalAI/pull/6386
- chore(model gallery): add liquidai_lfm2-350m-extract by @mudler in https://github.com/mudler/LocalAI/pull/6387
- chore(model gallery): add liquidai_lfm2-1.2b-extract by @mudler in https://github.com/mudler/LocalAI/pull/6388
- chore(model gallery): add liquidai_lfm2-1.2b-rag by @mudler in https://github.com/mudler/LocalAI/pull/6389
- chore(model gallery): add liquidai_lfm2-1.2b-tool by @mudler in https://github.com/mudler/LocalAI/pull/6390
- chore(model gallery): add liquidai_lfm2-350m-math by @mudler in https://github.com/mudler/LocalAI/pull/6391
- chore(model gallery): add liquidai_lfm2-8b-a1b by @mudler in https://github.com/mudler/LocalAI/pull/6414
- chore(model gallery): add gliese-4b-oss-0410-i1 by @mudler in https://github.com/mudler/LocalAI/pull/6415
- chore(model gallery): add qwen3-deckard-large-almost-human-6b-i1 by @mudler in https://github.com/mudler/LocalAI/pull/6416
- chore(model gallery): add ai21labs_ai21-jamba-reasoning-3b by @mudler in https://github.com/mudler/LocalAI/pull/6417
- chore(ui): skip duplicated entries in search list by @mudler in https://github.com/mudler/LocalAI/pull/6425
- chore(model gallery): add yanolja_yanoljanext-rosetta-12b-2510 by @mudler in https://github.com/mudler/LocalAI/pull/6442
- chore(model gallery): add agentflow_agentflow-planner-7b by @mudler in https://github.com/mudler/LocalAI/pull/6443
- chore(model gallery): add gustavecortal_beck by @mudler in https://github.com/mudler/LocalAI/pull/6444
- chore(model gallery): add qwen3-4b-ra-sft by @mudler in https://github.com/mudler/LocalAI/pull/6458
- chore(model gallery): add demyagent-4b-i1 by @mudler in https://github.com/mudler/LocalAI/pull/6459
- chore(model gallery): add boomerang-qwen3-2.3b by @mudler in https://github.com/mudler/LocalAI/pull/6460
- chore(model gallery): add boomerang-qwen3-4.9b by @mudler in https://github.com/mudler/LocalAI/pull/6461
- gallery: :robot: add new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/6478
- gallery: :robot: add new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/6480
- chore(model gallery): :robot: add new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/6501
- chore(model gallery): add mira-v1.7-27b-i1 by @mudler in https://github.com/mudler/LocalAI/pull/6503
- chore(model gallery): :robot: add new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/6504
- chore(model gallery): :robot: add new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/6507
📖 Documentation and examples
- chore(docs): add MCP example by @mudler in https://github.com/mudler/LocalAI/pull/6405
- chore(docs): enhancements and clarifications by @mudler in https://github.com/mudler/LocalAI/pull/6433
👒 Dependencies
- chore(deps): bump actions/stale from 10.0.0 to 10.1.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6392
- chore(deps): bump github.com/rs/zerolog from 1.33.0 to 1.34.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6274
- chore(deps): bump github.com/nikolalohinski/gonja/v2 from 2.3.2 to 2.4.1 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6394
- chore(deps): bump github.com/docker/go-connections from 0.5.0 to 0.6.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6393
- chore: update cogito and simplify MCP logics by @mudler in https://github.com/mudler/LocalAI/pull/6413
- chore(deps): bump github.com/docker/docker from 28.3.3+incompatible to 28.5.0+incompatible by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6399
- chore(deps): bump github.com/multiformats/go-multiaddr from 0.16.0 to 0.16.1 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6277
- chore(deps): bump github.com/quic-go/quic-go from 0.54.0 to 0.54.1 in the go_modules group across 1 directory by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6431
- chore(deps): bump github/codeql-action from 3 to 4 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6451
- chore(deps): bump github.com/containerd/containerd from 1.7.27 to 1.7.28 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6448
- chore(deps): bump github.com/schollz/progressbar/v3 from 3.14.4 to 3.18.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6446
- chore(deps): bump dario.cat/mergo from 1.0.1 to 1.0.2 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6447
- chore(deps): bump github.com/ebitengine/purego from 0.8.4 to 0.9.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6450
- chore(deps): bump google.golang.org/grpc from 1.67.1 to 1.76.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6449
- feat(mcp): add planning and reevaluation by @mudler in https://github.com/mudler/LocalAI/pull/6541
- chore(deps): bump github.com/prometheus/client_golang from 1.23.0 to 1.23.2 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6600
- chore(deps): bump github.com/tmc/langchaingo from 0.1.13 to 0.1.14 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6604
- chore(deps): bump securego/gosec from 2.22.9 to 2.22.10 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6599
- chore(deps): bump github.com/gpustack/gguf-parser-go from 0.17.0 to 0.22.1 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6602
- chore(deps): bump github.com/onsi/ginkgo/v2 from 2.25.3 to 2.26.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6601
- chore(deps): bump github.com/gofrs/flock from 0.12.1 to 0.13.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/6598
Other Changes
- docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/6378
- chore: :arrow_up: Update ggml-org/llama.cpp to
128d522c04286e019666bd6ee4d18e3fbf8772e2by @localai-bot in https://github.com/mudler/LocalAI/pull/6379 - chore: :arrow_up: Update ggml-org/llama.cpp to
86df2c9ae4f2f1ee63d2558a9dc797b98524639bby @localai-bot in https://github.com/mudler/LocalAI/pull/6382 - feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/6384
- chore: :arrow_up: Update ggml-org/llama.cpp to
ca71fb9b368e3db96e028f80c4c9df6b6b370eddby @localai-bot in https://github.com/mudler/LocalAI/pull/6385 - chore: :arrow_up: Update ggml-org/llama.cpp to
3df2244df40c67dfd6ad548b40ccc507a066af2bby @localai-bot in https://github.com/mudler/LocalAI/pull/6401 - chore: :arrow_up: Update ggml-org/whisper.cpp to
c8223a8548ad64435266e551385fc51aca9ee8abby @localai-bot in https://github.com/mudler/LocalAI/pull/6402 - chore: :arrow_up: Update ggml-org/llama.cpp to
aeaf8a36f06b5810f5ae4bbefe26edb33925cf5eby @localai-bot in https://github.com/mudler/LocalAI/pull/6408 - chore: :arrow_up: Update ggml-org/llama.cpp to
9d0882840e6c3fb62965d03af0e22880ea90e012by @localai-bot in https://github.com/mudler/LocalAI/pull/6410 - chore: :arrow_up: Update ggml-org/whisper.cpp to
8877dfc11a9322ce1990958494cf2e41c54657ebby @localai-bot in https://github.com/mudler/LocalAI/pull/6411 - chore: :arrow_up: Update ggml-org/whisper.cpp to
98930fded1c06e601a38903607af262f04893880by @localai-bot in https://github.com/mudler/LocalAI/pull/6420 - chore(deps): bump llama.cpp to '1deee0f8d494981c32597dca8b5f8696d399b0f2' by @mudler in https://github.com/mudler/LocalAI/pull/6421
- chore: :arrow_up: Update ggml-org/whisper.cpp to
85871a946971955c635f56bca24ea2a37fed6324by @localai-bot in https://github.com/mudler/LocalAI/pull/6435 - chore: :arrow_up: Update ggml-org/llama.cpp to
e60f01d941bc5b7fae62dd57fee4cec76ec0ea6eby @localai-bot in https://github.com/mudler/LocalAI/pull/6434 - chore: :arrow_up: Update ggml-org/llama.cpp to
11f0af5504252e453d57406a935480c909e3ff37by @localai-bot in https://github.com/mudler/LocalAI/pull/6437 - chore: :arrow_up: Update ggml-org/whisper.cpp to
a91dd3be72f70dd1b3cb6e252f35fa17b93f596cby @localai-bot in https://github.com/mudler/LocalAI/pull/6439 - chore: :arrow_up: Update ggml-org/llama.cpp to
a31cf36ad946a13b3a646bf0dadf2a481e89f944by @localai-bot in https://github.com/mudler/LocalAI/pull/6440
New Contributors
- @robert-cronin made their first contribution in https://github.com/mudler/LocalAI/pull/6495
- @gmaOCR made their first contribution in https://github.com/mudler/LocalAI/pull/6777
- @lukasdotcom made their first contribution in https://github.com/mudler/LocalAI/pull/6842
Full Changelog: https://github.com/mudler/LocalAI/compare/v3.6.0...v3.7.0