Release Gateway-v0.2.2
🚀 SGLang Model Gateway v0.2.2 Released!
✨ Features
🎯 Industry-First Responses API for All Models We're bringing OpenAI's Responses API to the entire open-source ecosystem! Now enjoy native support for Llama, DeepSeek, Qwen, and more – with built-in chat history management, multi-turn conversations, and seamless MCP integration. This is the first solution to democratize advanced conversation management across all OSS models.
☸️ Production-Ready Kubernetes Operations Taking large-scale deployments seriously! We now support native gRPC health check endpoints, making it effortless to deploy and operate SGLang at scale on Kubernetes with proper health monitoring and orchestration.
🔐 Your Network, Your Control
- mTLS Support: Secure gateway-to-SGLang communication whether you're running on edge, remote cloud, multi-cloud, or hybrid environments – we've got you covered
- MCP Proxy Enhancements: Configure proxies globally or per-individual MCP server – complete network control in your hands
🤖 Harmony Pipeline Introducing our unified OpenAI-native architecture with GPT OSS model support for both Responses API and Chat Completion – fully integrated with MCP and intelligent storage management.
🌍 Universal Platform Support A major leap in accessibility! SGLang Model Gateway now runs on nearly every operating system and architecture: Linux, Windows, Mac, x86, and ARM. Even better – we support all Python versions from 3.8 to 3.14 in a single wheel file, while reducing wheel size by more than 40%. Deploy anywhere, on any Python version, with unprecedented efficiency!
⚡ Additional Enhancements
- Multi-worker URL support for better load distribution
- Connection pooling and tool inventory for MCP
- Native OpenAI web search tool support and function calling for OpenAI router
🐛 Stability Improvements
We've squashed numerous bugs including background task handling, tool call IDs, conversation management, and installation dependencies.
Try it now: pip install sglang-router==0.2.2
What's Changed in Gateway
Gateway Changes (48 commits)
- [router] 0.2.2 release (#12399) by @slin1237 in https://github.com/sgl-project/sglang/pull/12399
- [router] web_search_preview tool basic implementation (#12290) by @key4ng in https://github.com/sgl-project/sglang/pull/12290
- [router] Function call support for openai router Responses API (#12386) by @key4ng in https://github.com/sgl-project/sglang/pull/12386
- [router] Fix safety_identifier missing (#12404) by @key4ng in https://github.com/sgl-project/sglang/pull/12404
- [router] use safety_identifier replace user on chat history storage (#12185) by @lengrongfu in https://github.com/sgl-project/sglang/pull/12185
- [router] harmony responses api streaming support (#12395) by @slin1237 in https://github.com/sgl-project/sglang/pull/12395
- [router] Harmony Pipeline: Chat Completion & Responses API with MCP Support (#12153) by @slin1237 in https://github.com/sgl-project/sglang/pull/12153
- [bug] fix router installation to include additional dependency (#12348) by @slin1237 in https://github.com/sgl-project/sglang/pull/12348
- [router] refactor mcp to use LRU and fix pooling bug (#12346) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12346
- [bug] fix router pypi license file (#12345) by @slin1237 in https://github.com/sgl-project/sglang/pull/12345
- [router] fix router release workflow and add build test in PR (#12315) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12315
- [Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang (#12338) by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/12338
- [router][grpc] Fix inconsistent behavior of conversation_id not found (#12299) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12299
- [router] support arm, windows, mac, linux, reduce wheel size and number (#12285) by @slin1237 in https://github.com/sgl-project/sglang/pull/12285
- [rust][ci] Add end-to-end tests for Oracle history backend (#12233) by @key4ng in https://github.com/sgl-project/sglang/pull/12233
- [router] upgrade grpc dependency and py 3.13 3.14 support (#12284) by @slin1237 in https://github.com/sgl-project/sglang/pull/12284
- [router] Fix type unmatch during validation (#12257) by @key4ng in https://github.com/sgl-project/sglang/pull/12257
- [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/10804
- [router] configure workflow retries and timeout based on routerConfig (#12252) by @slin1237 in https://github.com/sgl-project/sglang/pull/12252
- [router] use mcp struct from sdk and clean up code across codebase (#12249) by @slin1237 in https://github.com/sgl-project/sglang/pull/12249
- [router] remove code duplication (#12245) by @slin1237 in https://github.com/sgl-project/sglang/pull/12245
New Contributors
- @lengrongfu made their first contribution in https://github.com/sgl-project/sglang/commit/09af0a7b5
- @sufeng-buaa made their first contribution in https://github.com/sgl-project/sglang/commit/ea9610600
Paths Included
sgl-routerpython/sglang/srt/grpcpython/sglang/srt/entrypoints/grpc_server.py
Full Changelog: https://github.com/sgl-project/sglang/compare/gateway-v0.2.1...gateway-v0.2.2