:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more. Features: Generate Text, MCP, Audio, Video, Images, Voice Cloning, Distributed, P2P and decentralized inference
LocalAI 3.10.0 is big on agent capabilities, multi-modal support, and cross-platform reliability.
We've added native Anthropic API support, launched a new Video Generation UI, introduced Open Responses API compatibility, and enhanced performance with a unified GPU backend system.
For a full tour, see below!
📌 TL;DR
| Feature | Summary |
|--------|--------|
| Anthropic API Support | Fully compatible /v1/messages endpoint for seamless drop-in replacement of Claude. |
| Open Responses API | Native support for stateful agents with tool calling, streaming, background mode, and multi-turn conversations, passing all . |
| | New video gen UI + LTX-2 support for text-to-video and image-to-video. |
| | GPU libraries (CUDA, ROCm, Vulkan) packaged inside backend containers — on Nvidia, AMD, and ARM64 (Experimental). |
| | Full support for streaming tool calls and XML-formatted tool outputs. |
| | Only see backends your system can run (e.g., hide MLX on Linux). |
| | Prevents crashes on AVX-only CPUs (Intel Sandy/Ivy Bridge) and fixes VRAM reporting on AMD GPUs. |
| | Debug agents & fine-tuning with memory-based request/response logging. |
| | Ultra-fast transcription engine for low-end devices. |
| | Lightweight, high-fidelity text-to-speech with voice cloning. |
| | We now build backends and images for vulkan on arm64 as well |
The free, Open Source OpenAI alternative. Drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required.
RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Works alongside LocalAI and LocalAGI.
LocalAI is a true FOSS movement — built by contributors, powered by community.
If you believe in privacy-first AI:
✅ Star the repo
💬 Contribute code, docs, or feedback
📣 Share with others
Your support keeps this stack alive.
✅ Full Changelog
📋 Click to expand full changelog
What's Changed
Bug fixes :bug:
fix(ui): correctly parse import errors by @mudler in https://github.com/mudler/LocalAI/pull/7726
fix(cli): import via CLI needs system state by @mudler in https://github.com/mudler/LocalAI/pull/7746
fix(amd-gpu): correctly show total and used vram by @mudler in https://github.com/mudler/LocalAI/pull/7761
fix: add nil checks before mergo.Merge to prevent panic in gallery model installation by @majiayu000 in https://github.com/mudler/LocalAI/pull/7785
fix: Usage for image generation is incorrect (and causes error in LiteLLM) by @majiayu000 in https://github.com/mudler/LocalAI/pull/7786
fix: propagate validation errors by @majiayu000 in https://github.com/mudler/LocalAI/pull/7787
fix: Failed to download checksums.txt when using launch to install localai by @majiayu000 in https://github.com/mudler/LocalAI/pull/7788
fix(image-gen): fix scrolling issues by @mudler in https://github.com/mudler/LocalAI/pull/7829
fix(llama.cpp/mmproj): fix loading mmproj in nested sub-dirs different from model path by @mudler in https://github.com/mudler/LocalAI/pull/7832
fix: Prevent BMI2 instruction crash on AVX-only CPUs by @coffeerunhobby in https://github.com/mudler/LocalAI/pull/7817
fix: Highly inconsistent agent response to cogito agent calling MCP server - Body "Invalid http method" by @majiayu000 in https://github.com/mudler/LocalAI/pull/7790
fix(chat/ui): record model name in history for consistency by @mudler in https://github.com/mudler/LocalAI/pull/7845
fix(ui): fix 404 on API menu link by pointing to index.html by @DEVMANISHOFFL in https://github.com/mudler/LocalAI/pull/7878
fix: BMI2 crash on AVX-only CPUs (Intel Ivy Bridge/Sandy Bridge) by @coffeerunhobby in https://github.com/mudler/LocalAI/pull/7864
fix(model): do not assume success when deleting a model process by @jroeber in https://github.com/mudler/LocalAI/pull/7963
fix(functions): do not duplicate function when valid JSON is inside XML tags by @mudler in https://github.com/mudler/LocalAI/pull/8043
Exciting New Features 🎉
feat: disable force eviction by @mudler in https://github.com/mudler/LocalAI/pull/7725
feat(api): Allow tracing of requests and responses by @richiejp in https://github.com/mudler/LocalAI/pull/7609
feat(UI): image generation improvements by @mudler in https://github.com/mudler/LocalAI/pull/7804
feat(image-gen/UI): move controls to the left, make the page more compact by @mudler in https://github.com/mudler/LocalAI/pull/7823
feat(function): Add tool streaming, XML Tool Call Parsing Support by @mudler in https://github.com/mudler/LocalAI/pull/7865
chore: Update to Ubuntu24.04 (cont #7423) by @richiejp in https://github.com/mudler/LocalAI/pull/7769
feat: package GPU libraries inside backend containers for unified base image by @Copilot in https://github.com/mudler/LocalAI/pull/7891
feat(backends): add moonshine backend for faster transcription by @mudler in https://github.com/mudler/LocalAI/pull/7833
feat: enable Vulkan arm64 image builds by @Copilot in https://github.com/mudler/LocalAI/pull/7912
feat: Add Anthropic Messages API support by @Copilot in https://github.com/mudler/LocalAI/pull/7948
feat: add tool/function calling support to Anthropic Messages API by @Copilot in https://github.com/mudler/LocalAI/pull/7956
feat(api): support 'reasoning' api field by @mudler in https://github.com/mudler/LocalAI/pull/7959
feat: Filter backend gallery by system capabilities by @Copilot in https://github.com/mudler/LocalAI/pull/7950
feat(tts): add pocket-tts backend by @mudler in https://github.com/mudler/LocalAI/pull/8018
feat(diffusers): add support to LTX-2 by @mudler in https://github.com/mudler/LocalAI/pull/8019
feat(ui): add video gen UI by @mudler in https://github.com/mudler/LocalAI/pull/8020
feat(api): add support for open responses specification by @mudler in https://github.com/mudler/LocalAI/pull/8063
🧠 Models
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7801
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7807
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7816
Fix(gallery): Updated checksums for qwen3-vl-30b instruct & thinking by @Nold360 in https://github.com/mudler/LocalAI/pull/7819
chore(model-gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/7821
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7826
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7831
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7840
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7903
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7916
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7922
chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/7954
chore(model gallery): add qwen3-coder-30b-a3b-instruct based on model request by @rampa3 in https://github.com/mudler/LocalAI/pull/8082
📖 Documentation and examples
chore(AGENTS.md): Add section to help with building backends by @richiejp in https://github.com/mudler/LocalAI/pull/7871
[gallery] add JSON schema for gallery model specification by @DEVMANISHOFFL in https://github.com/mudler/LocalAI/pull/7890
chore(doc): put alert on install.sh until is fixed by @mudler in https://github.com/mudler/LocalAI/pull/8042
👒 Dependencies
chore(deps): bump securego/gosec from 2.22.9 to 2.22.11 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7774
chore(deps): bump google.golang.org/grpc from 1.77.0 to 1.78.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7777
chore(deps): bump github.com/schollz/progressbar/v3 from 3.18.0 to 3.19.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7775
chore(deps): bump github.com/modelcontextprotocol/go-sdk from 1.1.0 to 1.2.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7776
chore(deps): bump dependabot/fetch-metadata from 2.4.0 to 2.5.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7876
chore(deps): bump github.com/labstack/echo/v4 from 4.14.0 to 4.15.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7875
chore(deps): bump protobuf from 6.33.2 to 6.33.4 in /backend/python/transformers by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7993
chore(deps): bump github.com/mudler/go-processmanager from 0.0.0-20240820160718-8b802d3ecf82 to 0.1.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/7992
chore(deps): bump github.com/onsi/gomega from 1.38.3 to 1.39.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/8000
chore(deps): bump github.com/gpustack/gguf-parser-go from 0.22.1 to 0.23.1 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/8001
chore(deps): bump fyne.io/fyne/v2 from 2.7.1 to 2.7.2 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/8003
chore(deps): bump github.com/onsi/ginkgo/v2 from 2.27.3 to 2.27.5 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/8004
chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/rerankers in the pip group across 1 directory by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/8066
Other Changes
docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/7716
chore: :arrow_up: Update ggml-org/whisper.cpp to 6114e692136bea917dc88a5eb2e532c3d133d963 by @localai-bot in https://github.com/mudler/LocalAI/pull/7717
chore: :arrow_up: Update ggml-org/llama.cpp to c18428423018ed214c004e6ecaedb0cbdda06805 by @localai-bot in https://github.com/mudler/LocalAI/pull/7718
chore: :arrow_up: Update ggml-org/llama.cpp to 85c40c9b02941ebf1add1469af75f1796d513ef4 by @localai-bot in https://github.com/mudler/LocalAI/pull/7731
chore: :arrow_up: Update ggml-org/llama.cpp to 7ac8902133da6eb390c4d8368a7d252279123942 by @localai-bot in https://github.com/mudler/LocalAI/pull/7740
chore: :arrow_up: Update ggml-org/llama.cpp to a4bf35889eda36d3597cd0f8f333f5b8a2fcaefc by @localai-bot in https://github.com/mudler/LocalAI/pull/7751
chore: :arrow_up: Update ggml-org/llama.cpp to 4ffc47cb2001e7d523f9ff525335bbe34b1a2858 by @localai-bot in https://github.com/mudler/LocalAI/pull/7760
chore(ci): be more precise when detecting existing models by @mudler in https://github.com/mudler/LocalAI/pull/7767
chore: ⬆️ Update leejet/stable-diffusion.cpp to 4ff2c8c74bd17c2cfffe3a01be77743fb3efba2f by @richiejp in https://github.com/mudler/LocalAI/pull/7771
chore: :arrow_up: Update ggml-org/llama.cpp to c9a3b40d6578f2381a1373d10249403d58c3c5bd by @localai-bot in https://github.com/mudler/LocalAI/pull/7778
Revert "chore(deps): bump securego/gosec from 2.22.9 to 2.22.11" by @mudler in https://github.com/mudler/LocalAI/pull/7789
feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/7794
chore: :arrow_up: Update ggml-org/llama.cpp to 0f89d2ecf14270f45f43c442e90ae433fd82dab1 by @localai-bot in https://github.com/mudler/LocalAI/pull/7795
chore: :arrow_up: Update ggml-org/whisper.cpp to e9898ddfb908ffaa7026c66852a023889a5a7202 by @localai-bot in https://github.com/mudler/LocalAI/pull/7810
chore: :arrow_up: Update ggml-org/llama.cpp to 13814eb370d2f0b70e1830cc577b6155b17aee47 by @localai-bot in https://github.com/mudler/LocalAI/pull/7809
feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/7820
chore: :arrow_up: Update ggml-org/llama.cpp to ced765be44ce173c374f295b3c6f4175f8fd109b by @localai-bot in https://github.com/mudler/LocalAI/pull/7822
chore: :arrow_up: Update ggml-org/llama.cpp to 706e3f93a60109a40f1224eaf4af0d59caa7c3ae by @localai-bot in https://github.com/mudler/LocalAI/pull/7836
New Contributors
@majiayu000 made their first contribution in https://github.com/mudler/LocalAI/pull/7785
@coffeerunhobby made their first contribution in https://github.com/mudler/LocalAI/pull/7817
@DEVMANISHOFFL made their first contribution in https://github.com/mudler/LocalAI/pull/7878
@jroeber made their first contribution in https://github.com/mudler/LocalAI/pull/7963
@Divyanshupandey007 made their first contribution in https://github.com/mudler/LocalAI/pull/8050
Full Changelog: https://github.com/mudler/LocalAI/compare/v3.9.0...v3.10.0
feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/7847
chore: :arrow_up: Update ggml-org/llama.cpp to e57f52334b2e8436a94f7e332462dfc63a08f995 by @localai-bot in https://github.com/mudler/LocalAI/pull/7848
chore(Makefile): refactor common make targets by @mudler in https://github.com/mudler/LocalAI/pull/7858
chore: :arrow_up: Update leejet/stable-diffusion.cpp to b90b1ee9cf84ea48b478c674dd2ec6a33fd504d6 by @localai-bot in https://github.com/mudler/LocalAI/pull/7862
chore: :arrow_up: Update ggml-org/llama.cpp to 4974bf53cf14073c7b66e1151348156aabd42cb8 by @localai-bot in https://github.com/mudler/LocalAI/pull/7861
chore: :arrow_up: Update leejet/stable-diffusion.cpp to c5602a676caff5fe5a9f3b76b2bc614faf5121a5 by @localai-bot in https://github.com/mudler/LocalAI/pull/7880
chore: :arrow_up: Update ggml-org/whisper.cpp to 679bdb53dbcbfb3e42685f50c7ff367949fd4d48 by @localai-bot in https://github.com/mudler/LocalAI/pull/7879
chore: :arrow_up: Update ggml-org/llama.cpp to e443fbcfa51a8a27b15f949397ab94b5e87b2450 by @localai-bot in https://github.com/mudler/LocalAI/pull/7881
chore(image-ui): simplify interface by @mudler in https://github.com/mudler/LocalAI/pull/7882
chore(llama.cpp/flags): simplify conditionals by @mudler in https://github.com/mudler/LocalAI/pull/7887
chore: :arrow_up: Update ggml-org/llama.cpp to ccbc84a5374bab7a01f68b129411772ddd8e7c79 by @localai-bot in https://github.com/mudler/LocalAI/pull/7894
chore: :arrow_up: Update leejet/stable-diffusion.cpp to 9be0b91927dfa4007d053df72dea7302990226bb by @localai-bot in https://github.com/mudler/LocalAI/pull/7895
chore(dockerfile): drop driver-requirements section by @mudler in https://github.com/mudler/LocalAI/pull/7907
chore(detection): detect GPU vendor from files present in the system by @mudler in https://github.com/mudler/LocalAI/pull/7908
chore(ci): restore building of GPU vendor images by @mudler in https://github.com/mudler/LocalAI/pull/7910
chore(Dockerfile): restore GPU vendor specific sections by @mudler in https://github.com/mudler/LocalAI/pull/7911
fix(intel): Add ARG for Ubuntu codename in Dockerfile by @mudler in https://github.com/mudler/LocalAI/pull/7917
chore: :arrow_up: Update ggml-org/llama.cpp to ae9f8df77882716b1702df2bed8919499e64cc28 by @localai-bot in https://github.com/mudler/LocalAI/pull/7915
chore(ci): use latest jetpack image for l4t by @mudler in https://github.com/mudler/LocalAI/pull/7926
chore(l4t-12): do not use python 3.12 (wheels are only for 3.10) by @mudler in https://github.com/mudler/LocalAI/pull/7928
chore(docs): Add Crush and VoxInput to the integrations by @richiejp in https://github.com/mudler/LocalAI/pull/7924
Optimize GPU library copying to preserve symlinks and avoid duplicates by @Copilot in https://github.com/mudler/LocalAI/pull/7931
chore(uv): add --index-strategy=unsafe-first-match to l4t by @mudler in https://github.com/mudler/LocalAI/pull/7934
chore: :arrow_up: Update leejet/stable-diffusion.cpp to 0e52afc6513cc2dea9a1a017afc4a008d5acf2b0 by @localai-bot in https://github.com/mudler/LocalAI/pull/7930
chore(ci): roll back l4t-cuda12 configurations by @mudler in https://github.com/mudler/LocalAI/pull/7935
Revert "chore(uv): add --index-strategy=unsafe-first-match to l4t" by @mudler in https://github.com/mudler/LocalAI/pull/7936
chore(deps): Bump llama.cpp to '480160d47297df43b43746294963476fc0a6e10f' by @mudler in https://github.com/mudler/LocalAI/pull/7933
chore(llama.cpp): propagate errors during model load by @mudler in https://github.com/mudler/LocalAI/pull/7937
chore: :arrow_up: Update ggml-org/llama.cpp to 593da7fa49503b68f9f01700be9f508f1e528992 by @localai-bot in https://github.com/mudler/LocalAI/pull/7946
feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/7964
chore: :arrow_up: Update ggml-org/llama.cpp to b1377188784f9aea26b8abde56d4aee8c733eec7 by @localai-bot in https://github.com/mudler/LocalAI/pull/7965
fix(l4t-12): use pip to install python deps by @mudler in https://github.com/mudler/LocalAI/pull/7967
chore: :arrow_up: Update ggml-org/llama.cpp to 0c3b7a9efebc73d206421c99b7eb6b6716231322 by @localai-bot in https://github.com/mudler/LocalAI/pull/7978
chore: :arrow_up: Update leejet/stable-diffusion.cpp to 885e62ea822e674c6837a8225d2d75f021b97a6a by @localai-bot in https://github.com/mudler/LocalAI/pull/7979
chore(backends): do not bundle cuda target directory by @mudler in https://github.com/mudler/LocalAI/pull/7982
chore(vulkan): bump vulkan-sdk to 1.4.335.0 by @mudler in https://github.com/mudler/LocalAI/pull/7981
chore: :arrow_up: Update ggml-org/llama.cpp to bcf7546160982f56bc290d2e538544bbc0772f63 by @localai-bot in https://github.com/mudler/LocalAI/pull/7991
chore: :arrow_up: Update leejet/stable-diffusion.cpp to 7010bb4dff7bd55b03d35ef9772142c21699eba9 by @localai-bot in https://github.com/mudler/LocalAI/pull/8013
chore: :arrow_up: Update ggml-org/whisper.cpp to a96310871a3b294f026c3bcad4e715d17b5905fe by @localai-bot in https://github.com/mudler/LocalAI/pull/8014
chore: :arrow_up: Update ggml-org/llama.cpp to e4832e3ae4d58ac0ecbdbf4ae055424d6e628c9f by @localai-bot in https://github.com/mudler/LocalAI/pull/8015
chore: :arrow_up: Update ggml-org/whisper.cpp to 47af2fb70f7e4ee1ba40c8bed513760fdfe7a704 by @localai-bot in https://github.com/mudler/LocalAI/pull/8039
chore: :arrow_up: Update ggml-org/llama.cpp to d98b548120eecf98f0f6eaa1ba7e29b3afda9f2e by @localai-bot in https://github.com/mudler/LocalAI/pull/8040
fix: reduce log verbosity for /api/operations polling by @Divyanshupandey007 in https://github.com/mudler/LocalAI/pull/8050
chore: :arrow_up: Update ggml-org/whisper.cpp to 2eeeba56e9edd762b4b38467bab96c2517163158 by @localai-bot in https://github.com/mudler/LocalAI/pull/8052
chore: :arrow_up: Update ggml-org/llama.cpp to 785a71008573e2d84728fb0ba9e851d72d3f8fab by @localai-bot in https://github.com/mudler/LocalAI/pull/8053
fix(ci): use more beefy runner for expensive jobs by @mudler in https://github.com/mudler/LocalAI/pull/8065
Revert "chore(deps): bump torch from 2.3.1+cxx11.abi to 2.8.0 in /backend/python/rerankers in the pip group across 1 directory" by @mudler in https://github.com/mudler/LocalAI/pull/8072
chore: :arrow_up: Update ggml-org/llama.cpp to 388ce822415f24c60fcf164a321455f1e008cafb by @localai-bot in https://github.com/mudler/LocalAI/pull/8073
chore: :arrow_up: Update ggml-org/whisper.cpp to f53dc74843e97f19f94a79241357f74ad5b691a6 by @localai-bot in https://github.com/mudler/LocalAI/pull/8074
chore(ui): add video generation link by @mudler in https://github.com/mudler/LocalAI/pull/8079
chore: :arrow_up: Update ggml-org/llama.cpp to 2fbde785bc106ae1c4102b0e82b9b41d9c466579 by @localai-bot in https://github.com/mudler/LocalAI/pull/8087
chore: :arrow_up: Update leejet/stable-diffusion.cpp to 9565c7f6bd5fcff124c589147b2621244f2c4aa1 by @localai-bot in https://github.com/mudler/LocalAI/pull/8086
