Llamafile

Llamafile v0.9.2 is a significant release. It adds support for:

• DeepSeek Distil R1 Models
• Gemma 3
• IBM Granite

LocalScore

In addition 0.9.2 introduces LocalScore, a benchmarking utility and website.

LocalScore is an open-source tool that both benchmarks how fast Large Language Models (LLMs) run on your specific hardware and serves as a repository for these results. We created LocalScore to provide a simple, portable way to evaluate computer performance across various LLMs while making it easy to share and browse hardware performance data.

LocalScore is now part of the release of Llamafile under the new CLI utility localscore

You can run it ./localscore -m <model>. It is also included in every llamafile so you can benchmark models on your hardware easily using ./llamafile --localscore.

LocalScore was created with support from Mozilla Builders.

What's Changed

• [llamafiler] doc/v1_chat_completions.md: remove duplicate entry by @mseri in https://github.com/Mozilla-Ocho/llamafile/pull/607
• Update server readme with code completion (FIM) example by @heaversm in https://github.com/Mozilla-Ocho/llamafile/pull/637
• URL constructor to get a clean url_prefix (fix #640) by @sizvix in https://github.com/Mozilla-Ocho/llamafile/pull/641
• Fix translation bug from cpp to js in TS highlight by @emilbayes in https://github.com/Mozilla-Ocho/llamafile/pull/681
• Add whisperfile server documentation by @alonsosilvaallende in https://github.com/Mozilla-Ocho/llamafile/pull/707
• Unify button look and rearrange buttons to make them more compact by @corebonts in https://github.com/Mozilla-Ocho/llamafile/pull/712
• add stable-diffusion.cpp to install target (fix #580) by @rgroesslinger in https://github.com/Mozilla-Ocho/llamafile/pull/635
• Improve OpenAI compatibility for /v1/* endpoints by @corebonts in https://github.com/Mozilla-Ocho/llamafile/pull/713
• Update WSL troubleshooting in README.md by @halter73 in https://github.com/Mozilla-Ocho/llamafile/pull/585
• Granite three support by @gabe-l-hart in https://github.com/Mozilla-Ocho/llamafile/pull/608
• Initial support for Gemma 3 models by @corebonts in https://github.com/Mozilla-Ocho/llamafile/pull/717
• Add copy and info buttons to the chat window and improve small screen UX by @corebonts in https://github.com/Mozilla-Ocho/llamafile/pull/719
• Avoid streaming incomplete UTF-8 characters by @corebonts in https://github.com/Mozilla-Ocho/llamafile/pull/727
• Introduce LocalScore CLI by @cjpais in https://github.com/Mozilla-Ocho/llamafile/pull/734

New Contributors

• @mseri made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/607
• @heaversm made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/637
• @sizvix made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/641
• @emilbayes made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/681
• @alonsosilvaallende made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/707
• @corebonts made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/712
• @rgroesslinger made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/635
• @halter73 made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/585
• @gabe-l-hart made their first contribution in https://github.com/Mozilla-Ocho/llamafile/pull/608

Full Changelog: https://github.com/Mozilla-Ocho/llamafile/compare/0.9.1...0.9.2