Routing Policies | Scenario | Decision Basis | Applicable Scope |
Route by model name | Precise matching for multi-model services | Match the model field in the request with the model name configured in the model service. | A single-model API needs to access the same model from different suppliers. |
Weight-based routing | Load balancing, A/B testing, and canary release | Randomly allocate traffic according to the configured weight ratio. | Multi-instance deployment of the same model or hybrid deployment across multiple suppliers |
Intent-based routing | Classify and distribute based on semantic content | Call an intent recognition model to analyze the semantics of the request, and then route based on the identified intent. | Multi-task model, service tiering by problem complexity |
Parameter-based routing | Multi-tenant isolation, environment isolation, and user traffic distribution | Match routing rules based on the values of the request Header parameters. | Route traffic to different services based on tenant/environment/user type. |
Tag-based routing | Filtering by model tags | Automatically filter available services through model service tags (Key-Value). | A large number of models require filtering by tags. |
Latency-first routing | Cross-region multi-path routing and response experience optimization | Select the optimal path based on real-time latency monitoring data of the model service. | Multi-region deployment, multi-IDC access scenarios |
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback