🌍 Universal Platform Support A major leap in accessibility! SGLang Model Gateway now runs on nearly every operating system and architecture: Linux, Windows, Mac, x86, and ARM. Even better – we support all Python versions from 3.8 to 3.14 in a single wheel file, while reducing wheel size by more than 40%. Deploy anywhere, on any Python version, with unprecedented efficiency!

⚡ Additional Enhancements

• Multi-worker URL support for better load distribution
• Connection pooling and tool inventory for MCP
• Native OpenAI web search tool support and function calling for OpenAI router

🐛 Stability Improvements

We've squashed numerous bugs including background task handling, tool call IDs, conversation management, and installation dependencies.

Try it now: pip install sglang-router==0.2.2

What's Changed in Gateway

Gateway Changes (48 commits)

• [router] 0.2.2 release (#12399) by @slin1237 in https://github.com/sgl-project/sglang/pull/12399
• [router] web_search_preview tool basic implementation (#12290) by @key4ng in https://github.com/sgl-project/sglang/pull/12290
• [router] Function call support for openai router Responses API (#12386) by @key4ng in https://github.com/sgl-project/sglang/pull/12386
• [router] Fix safety_identifier missing (#12404) by @key4ng in https://github.com/sgl-project/sglang/pull/12404
• [router] use safety_identifier replace user on chat history storage (#12185) by @lengrongfu in https://github.com/sgl-project/sglang/pull/12185
• [router] harmony responses api streaming support (#12395) by @slin1237 in https://github.com/sgl-project/sglang/pull/12395
• [router] Harmony Pipeline: Chat Completion & Responses API with MCP Support (#12153) by @slin1237 in https://github.com/sgl-project/sglang/pull/12153
• [bug] fix router installation to include additional dependency (#12348) by @slin1237 in https://github.com/sgl-project/sglang/pull/12348
• [router] refactor mcp to use LRU and fix pooling bug (#12346) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12346
• [bug] fix router pypi license file (#12345) by @slin1237 in https://github.com/sgl-project/sglang/pull/12345
• [router] fix router release workflow and add build test in PR (#12315) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12315
• [Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang (#12338) by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/12338
• [router][grpc] Fix inconsistent behavior of conversation_id not found (#12299) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12299
• [router] support arm, windows, mac, linux, reduce wheel size and number (#12285) by @slin1237 in https://github.com/sgl-project/sglang/pull/12285
• [rust][ci] Add end-to-end tests for Oracle history backend (#12233) by @key4ng in https://github.com/sgl-project/sglang/pull/12233
• [router] upgrade grpc dependency and py 3.13 3.14 support (#12284) by @slin1237 in https://github.com/sgl-project/sglang/pull/12284
• [router] Fix type unmatch during validation (#12257) by @key4ng in https://github.com/sgl-project/sglang/pull/12257
• [Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 (#10804) by @sufeng-buaa in https://github.com/sgl-project/sglang/pull/10804
• [router] configure workflow retries and timeout based on routerConfig (#12252) by @slin1237 in https://github.com/sgl-project/sglang/pull/12252
• [router] use mcp struct from sdk and clean up code across codebase (#12249) by @slin1237 in https://github.com/sgl-project/sglang/pull/12249
• [router] remove code duplication (#12245) by @slin1237 in https://github.com/sgl-project/sglang/pull/12245
• [sgl-route] Optimize the use of constant slices and retain to simplif… (#12159) by @lengrongfu in https://github.com/sgl-project/sglang/pull/12159
• [router] Remove SharedXxxStorage type aliases to make Arc explicit (#12171) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12171
• [router][grpc] Add ResponsesContext and fix error propagation in responses api (#12164) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12164
• [misc][grpc] Remove duplicate log (#12168) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12168
• [router] centralize mcp tool args handling (#12155) by @slin1237 in https://github.com/sgl-project/sglang/pull/12155
• [router][grpc] Fix tool call id in parse_json_schema_response (#12152) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12152
• [router] cleaned up all the redundant comments in the config module (#12147) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12147
• [router] MCP Manager Refactoring - Flat Architecture with Connection Pooling (#12097) by @slin1237 in https://github.com/sgl-project/sglang/pull/12097
• [router] Refactor data connector architecture with unified storage modules (#12096) by @key4ng in https://github.com/sgl-project/sglang/pull/12096
• [router][grpc] Remove gpt_oss parsers and remove _parser suffix in tool parser files (#12091) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12091
• [router] migrate app context to builder pattern 2/n (#12089) by @slin1237 in https://github.com/sgl-project/sglang/pull/12089
• [router] migrate app context to builder pattern 1/n (#12086) by @slin1237 in https://github.com/sgl-project/sglang/pull/12086
• [router] fix ut router config init to use build pattern (#12084) by @slin1237 in https://github.com/sgl-project/sglang/pull/12084
• [router] implement response api get input item function and refactor input/output store (#11924) by @key4ng in https://github.com/sgl-project/sglang/pull/11924
• [router] Add mTLS Support for Router-to-Worker Communication (#12019) by @slin1237 in https://github.com/sgl-project/sglang/pull/12019
• [router] Add builder pattern for RouterConfig with zero duplication (#12030) by @slin1237 in https://github.com/sgl-project/sglang/pull/12030
• [router][CI] Clean up imports and prints statements in sgl-router/py_test (#12024) by @CatherineSue in https://github.com/sgl-project/sglang/pull/12024
• [router] change ci names and update log level in ci (#12021) by @slin1237 in https://github.com/sgl-project/sglang/pull/12021
• [Router] Consolidate ConnectionMode enum to core module (#11937) by @YouNeedCryDear in https://github.com/sgl-project/sglang/pull/11937
• [router] Add comprehensive E2E tests for Response API (#11988) by @key4ng in https://github.com/sgl-project/sglang/pull/11988
• [grpc] Support gRPC standard health check (#11955) by @CatherineSue in https://github.com/sgl-project/sglang/pull/11955
• [router] create worker removal step and clean up worker manager (#11921) by @slin1237 in https://github.com/sgl-project/sglang/pull/11921
• [router] Support multiple worker URLs for OpenAI router (#11723) by @key4ng in https://github.com/sgl-project/sglang/pull/11723
• [router][grpc] Fix background tasks stored with wrong id (#11945) by @CatherineSue in https://github.com/sgl-project/sglang/pull/11945
• [router] Add gRPC E2E test suite (#11790) by @key4ng in https://github.com/sgl-project/sglang/pull/11790
• [router][grpc] Support v1/responses API (#11926) by @CatherineSue in https://github.com/sgl-project/sglang/pull/11926
• Fix openai input_text type compatibility (#11935) by @key4ng in https://github.com/sgl-project/sglang/pull/11935

New Contributors

• @lengrongfu made their first contribution in https://github.com/sgl-project/sglang/commit/09af0a7b5
• @sufeng-buaa made their first contribution in https://github.com/sgl-project/sglang/commit/ea9610600

Paths Included

• sgl-router
• python/sglang/srt/grpc
• python/sglang/srt/entrypoints/grpc_server.py

Full Changelog: https://github.com/sgl-project/sglang/compare/gateway-v0.2.1...gateway-v0.2.2