FastChat v0.2.36

Breaking Changes

• Gradio server migrated to OpenAI v1 API client

Features

• SGLang worker for vision language models — lower latency, higher throughput
• Vision language WebUI with Gradio support
• OpenAI-compatible API server now accepts image input
• LightLLM worker integration for higher throughput
• Apple MLX worker with async support
• Yuan 2.0...

What's Changed

• add dolphin by @infwinston in https://github.com/lm-sys/FastChat/pull/2794
• Fix tiny typo by @bofenghuang in https://github.com/lm-sys/FastChat/pull/2805
• Add instructions for evaluating on MT bench using vLLM by @iojw in https://github.com/lm-sys/FastChat/pull/2770
• fix missing op | for py3.8 by @dumpmemory in https://github.com/lm-sys/FastChat/pull/2800
• Add SOLAR-10...

What's Changed

• fix tokenizer.pad_token attribute error by @wangshuai09 in https://github.com/lm-sys/FastChat/pull/2710
• support stable-vicuna model by @hi-jin in https://github.com/lm-sys/FastChat/pull/2696
• Exllama cache 8bit by @mjkaye in https://github.com/lm-sys/FastChat/pull/2719
• Add Yi support by @infwinston in https://github.com/lm-sys/FastChat/pull/2723
• Add Hermes 2.5 [fixe...

What's Changed

• fix: Fix for OpenOrcaAdapter to return correct conversation template by @vjsrinath in https://github.com/lm-sys/FastChat/pull/2613
• Make fastchat.serve.model_worker to take debug argument by @uinone in https://github.com/lm-sys/FastChat/pull/2628
• openchat 3.5 model support by @imoneoi in https://github.com/lm-sys/FastChat/pull/2638
• xFastTransformer framework support by...

What's Changed

• Fix for single turn dataset by @toslunar in https://github.com/lm-sys/FastChat/pull/2509
• replace os.getenv with os.path.expanduser because the first one doesn… by @khalil-Hennara in https://github.com/lm-sys/FastChat/pull/2515
• Fix arena by @merrymercy in https://github.com/lm-sys/FastChat/pull/2522
• Update Dockerfile by @dubaoquan404 in https://github.com/lm-sys/FastCh...

What's Changed

• Support new models
• Bug fixes

New Contributors

• @wangxiyuan made their first contribution in https://github.com/lm-sys/FastChat/pull/2404
• @wangzhen263 made their first contribution in https://github.com/lm-sys/FastChat/pull/2402
• @karshPrime made their first contribution in https://github.com/lm-sys/FastChat/pull/2406
• @obitolyz made their first contribution i...

What's Changed

• Multiple UI updates, performance improvements and bug fixes
• New model support (Spicyboros + airoboros 2.2, VMware's OpenLLaMa OpenInstruct)
• Add sponsors (Kaggle, MBZUAI, AnyScale, and Huggingface)

New Contributors

• @nicobasile made their first contribution in https://github.com/lm-sys/FastChat/pull/2278
• @zeyugao made their first contribution in https://github....

What's Changed

• Support new models (Qwen, WizardCoder, Llama2-Chinese, BAAI/AquilaChat-7B, OpenOrca, BGE)
• Improve performance and usability. Fix bugs.
• Reduce dependency by making some required packages optional

New Contributors

• @azshue made their first contribution in https://github.com/lm-sys/FastChat/pull/2169
• @liunux4odoo made their first contribution in https://github.co...

• Released Vicuna v1.5 based on Llama 2 with 4K and 16K context lengths. Download weights
• Released Chatbot Arena Conversations, a dataset containing 33k conversations with human preferences. Download it here.
• Serving
  • Add a multi-model worker th...

• Release MT-bench code and data
  • Demo, Leaderboard, Human annotation data
• Release new models
  • Vicuna-33B v1.3 and [LongChat](https://lmsys.org/blog/2023-06-29-...