tencent cloud

Cloud Native Intelligent Gateway

DokumentasiCloud Native Intelligent GatewayAI GatewayOperation GuideIntelligent RoutingModel Name Routing

Model Name Routing

Unduh
Mode fokus
Ukuran font
Terakhir diperbarui: 2026-09-22 18:43:15
Diterjemahkan oleh AI

Feature Overview

Routing by model name automatically matches client requests to the corresponding model service based on the model field in the request. This routing policy applies to the following scenarios:
A single API must support multiple different models (such as gpt-4o, gpt-4o-mini, and claude-3-5-sonnet).
Different models are provided by different vendors or service instances.
You need to precisely control the routing target for each model.
Use wildcards to match a group of models (for example, gpt-4-* matches all gpt-4 series models).

Configuration Steps

Step 1: Creating a Model Service

Before configuring a routing policy, you must first create a model service:
1. Log in to the Microservices Platform console. In the left sidebar, click AI Gateway > Instance List.
2. On the instance list page, click the ID of the gateway instance you want to configure to go to its basic information page.
3. In the left sidebar, click Model Management, and then click the Model Service tab.
4. In the service list, click New and configure information such as the vendor and access credentials.
5. Save the model service.

Step 2: Configuring Model Name Routing in the Model API

1. On the Model Management > Model API tab, click New.
2. After configuring the Basic Information, go to the Step 2: Select Model Service page to configure the service type and routing policy.
3. Select Service Type as Multi-Model Service.
4. Select the Routing Policy as Model Name Routing.
5. Add a service and configure routing rules in the model service table.
Configuration Parameter Description:
Parameter
Description
Example
Required
Model Service
Select from the list of created model services.
gpt-4o-openai
Yes
Model Name to Match
Matching rule for the model parameter in client requests, supporting exact matching and wildcard matching (*).
gpt-4o or gpt-4-*
Yes
Model Name to Rewrite
When the request is forwarded to the backend service, rewrite the model parameter in the request to a specified value. If the model parameter is left blank, the original value is passed through.
gpt-4o or leave it blank
No

Step 3: Saving and Publishing

After the configuration is complete, click OK to save it. The routing rules take effect immediately.

Configuration Example

Example 1: Exact Matching of Multiple Models

Scenario: A single API must support three different models, each provided by a different service.
Configuration:
Model Service
Matching Model Name
Rewritten Model Name
gpt-4o-openai
gpt-4o
(Leave blank)
gpt-4o-mini-openai
gpt-4o-mini
(Leave blank)
claude-3-5-sonnet-anthropic
claude-3-5-sonnet
(Leave blank)
Test Request:
# Request 1: Route to gpt-4o-openai
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization:Bearer {API_Key}" \\
  -H "Content-Type:application/json" \\
  -d '{"model":"gpt-4o", "messages":[{"role":"user", "content":"Hello"}]}'

# Request 2: Route to claude-3-5-sonnet-anthropic
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization:Bearer {API_Key}" \\
  -H "Content-Type:application/json" \\
  -d '{"model":"claude-3-5-sonnet", "messages": [{"role": "user", "content": "Hello"}]}'

Example 2: Wildcard Matching of Model Series

Scenario: Route all gpt-4-* series models to the same service.
Configuration:
Model Service
Matching Model Name
Rewritten Model Name
gpt-4-family-openai
gpt-4-*
(Leave blank)
gpt-3.5-turbo-openai
gpt-3.5-turbo
(Leave blank)
Test Request:
# Request 1: Match the wildcard rule, route to gpt-4-family-openai, and pass through model=gpt-4o.
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization: Bearer {API_Key}" \\
  -H "Content-Type: application/json" \\
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

# Request 2: Match the wildcard rule, route to gpt-4-family-openai, and pass through model=gpt-4-turbo.
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization: Bearer {API_Key}" \\
  -H "Content-Type: application/json" \\
  -d '{"model": "gpt-4-turbo", "messages": [{"role": "user", "content": "Hello"}]}'

# Request 3: Exact match, route to gpt-3.5-turbo-openai.
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization: Bearer {API_Key}" \\
  -H "Content-Type: application/json" \\
  -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hello"}]}'

Example 3: Model Name Rewriting

Scenario: The client uses a custom model name, but the backend service only recognizes standard model names.
Configuration:
Model Service
Matching Model Name
Rewritten Model Name
gpt-4o-openai
my-custom-gpt4
gpt-4o
claude-3-5-sonnet-anthropic
my-custom-claude
claude-3-5-sonnet-20241022
Test Request:
# When the client requests model=my-custom-gpt4, the gateway rewrites it to model=gpt-4o during forwarding.
curl -X POST https://{Gateway Domain}/ai/llm/v1/chat/completions \\
  -H "Authorization: Bearer {API_Key}" \\
  -H "Content-Type: application/json" \\
  -d '{"model": "my-custom-gpt4", "messages": [{"role": "user", "content": "Hello"}]}'

# Request body forwarded to the backend OpenAI service:
# {"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}

Routing match rule

Matching Priority

After the model_name_route routing policy is enabled, the gateway starts from the first rule in the configuration list and matches the model parameter in the request in top-down order. The first matching rule takes effect.

1. Matching by Configuration Order

Exact matching and wildcard matching have no built-in priority difference. Once a rule is matched successfully, the gateway stops matching subsequent rules.
Therefore, it is recommended to place rules with a narrower, more precise scope before those with a broader, wildcard scope.
For example, configure them in the following order:
Order
Model Name Rule
Matching Type
Target Model Service
1
gpt-4o
Exact matching
Service A
2
gpt-4*
Wildcard matching
Service B
When a request contains model=gpt-4o, the first rule is matched and the request is routed to Service A. When a request contains model=gpt-4-turbo, the second rule is matched and the request is routed to Service B.
If gpt-4* is configured before gpt-4o, a request with model=gpt-4o will first be matched by gpt-4* and then routed to Service B.
Attention:
The hyphen in gpt-4-* is a regular character. This rule can match gpt-4-turbo but cannot match gpt-4o.

2. Multiple Wildcard Rules Matching Simultaneously

When multiple wildcard rules can match the same model name, the rule that appears first in the configuration list is selected.
For example, both gpt-* and gpt-4* can match gpt-4o. The model service corresponding to the rule that appears first in the list is used.

3. No Rules Matched

When a request carries a valid string-type model parameter but fails to match any configured rules, the gateway returns HTTP 404

Response Format Example:
{
  "error": {
    "message": "model 'unknown-model' is not supported by any configured model_name_route rule"
  }
}
When the model parameter is missing, model is an empty string, or model is not a string, it is considered a request parameter exception and does not fall under the "model name not matched" scenario.

Wildcard Rules

Supported Wildcards:
Wildcard
Description
Example
Match Result
*
Matches any character (zero or more)
gpt-4-*
Matches gpt-4-turbo, gpt-4o, and gpt-4-0125-preview
gpt-*
Prefix matching
gpt-*
Matches all models starting with gpt-
Attention:
The wildcard * can only be used in the matching model name field and is not supported in the rewritten model name field.
A single matching rule can contain only one wildcard *.
Wildcard matching is case-sensitive. GPT-* does not match gpt-4o.

Model Name Rewriting Logic

Rewrite Timing:
Before forwarding a request to the backend model service, the gateway replaces the model field in the request body with the value configured for "Rewritten Model Name".
If the "Rewritten Model Name" is left empty, the original `model` value from the client request is passed through.
Typical Scenarios:
Scenario
Matching Model Name
Rewritten Model Name
Client Request model
model Forwarded to Backend
Pass through the original model
gpt-4o
(Leave blank)
gpt-4o
gpt-4o
Unify model identifiers
gpt-4-*
gpt-4-turbo-2024-04-09
gpt-4-turbo
gpt-4-turbo-2024-04-09
Custom alias
my-gpt4
gpt-4o
my-gpt4
gpt-4o
Unmatched Request Handling
If the requested model name does not match any routing rule, the gateway returns:
{
  "error": {
    "code": "model_not_found",
    "message": "Model 'gpt-5' does not exist. Please check whether the model name is correct. Available models: gpt-4o, gpt-4-*, claude-3-5-sonnet",
    "type": "invalid_request_error"
  }
}
The error message lists all matching rules supported by the current API, facilitating client-side troubleshooting.
Topik sebelumnya: OverviewTopik selanjutnya: Weighted Routing

Bantuan dan Dukungan

Apakah halaman ini membantu?

masukan

Search
    Enter keywords to search documentation