Changes

• Better handle multi-GPU setups when using Transformers with bitsandbytes (load-in-8bit and load-in-4bit).
• Implement the /v1/internal/logits endpoint for the exllamav3 and exllamav3_hf loaders.
• Make profile picture uploading safer.
• Add fla to the requirements for Exllamav3 to support qwen3-next models.

Bug fixes

• Fix an issue with loading certain chat histories in Instruct mode. Thanks, @Remowylliams.
• Fix portable builds for macOS x86 missing llama.cpp binaries (#7238). Thanks, @IonoclastBrigham.

Backend updates

• Update llama.cpp to https://github.com/ggml-org/llama.cpp/tree/d00cbea63c671cd85a57adaa50abf60b3b87d86f.
• Update transformers to 4.57.
• Update exllamav3 0.0.7.
• Update bitsandbytes to 0.48.

Portable builds

Below you can find self-contained packages that work with GGUF models (llama.cpp) and require no installation! Just download the right version for your system, unzip, and run.