Model routes

A model route maps a public model name to an upstream provider, provider model identifier, endpoint, and server-side credential reference. Clients select the public name. They cannot supply a provider secret or choose an unapproved upstream endpoint.

The current development configuration is TOML. The example file is config/niu.example.toml. Static credentials are referenced by environment variable name and loaded at startup.

Set public_catalog = true on a route to publish its alias, declared capabilities and descriptive model metadata at /models/ and GET /catalog/v1/models. This is an explicit public disclosure. The catalog omits provider names, upstream model identifiers, endpoints, credentials and pricing. Routes default to hidden.

The public model directory filters published aliases by alias, display name, input/output modality, and declared API capability. Entries show available descriptions, context and output limits, and links to the supported protocol references. Missing metadata is omitted rather than estimated. Public discovery does not grant workspace access or establish live Provider verification; workspace keys and customer prices remain separate.

TOML

[models.fast]
provider = "openrouter"
upstream_model = "openai/gpt-4.1-mini"
api_key_env = "OPENROUTER_API_KEY"

openrouter uses OpenAI-compatible chat completions at https://openrouter.ai/api/v1 when api_base is omitted. OpenAI remains supported with its own default endpoint. Provider names remain distinct in administration and route records.

Managed providers

Platform administrators manage shared Providers and their model routes in the dashboard or through the compatibility endpoint /api/v1/vendors. A Provider is the business supplying model access; a Provider identifies the upstream API adapter. OpenRouter can be a Provider using the openrouter Provider adapter. Model developers offered through that Provider do not require separate Niu credentials. Provider records, enabled state, model mappings, and revisions persist in PostgreSQL. Updates apply to new requests. Organization and workspace operators cannot manage platform-wide Providers.

Managed configurations accept openai, openrouter and anthropic adapters. For native Anthropic Messages, set api_base to https://api.anthropic.com/v1 and opt the mapping into supports_messages. Connection checks use native Anthropic authentication/version headers and a bounded 1,000-model page. An incomplete catalog is not presented as a complete inventory; a missing model in an incomplete check remains unknown. Configuration persistence and actual invalid-credential refusal are verified; successful direct Anthropic generation and catalog discovery remain unverified. See the native protocol boundary.

Set NIU_VENDOR_ENCRYPTION_KEY to a separate, high-entropy secret of at least 32 characters before storing a Provider credential. Niu encrypts credentials using AES-256-GCM; API responses never return them. Keep the encryption key in your deployment secret store and back it up separately from PostgreSQL. Changing it without re-encrypting stored credentials makes them unreadable; automatic master-key rotation is not implemented.

A database model alias overrides the same alias in TOML, including when the database route is disabled. A configuration with an empty [models] section can use only managed routes. Static file routes remain supported.

Deployment operators may set NIU_VENDOR_BOOTSTRAP_FILE to a JSON seed file. It contains Provider metadata, model routes, and the name of the environment variable holding the initial credential. It must contain no secret values. The seed runs once per bootstrap Provider identity and preserves later edits, renames, credential rotations and disabled state.

Keep credentials out of TOML, source control, browser storage and public catalog responses.

Model Pools

A Model Pool publishes a stable custom model name (slug, such as smart-coder) backed by an ordered set of candidate model routes, conditional routing rules, and automatic failover. Applications and workspace keys call the pool name directly in the model field of POST /v1/chat/completions, POST /v1/responses, and POST /v1/messages, without hardcoding individual upstream aliases in client code.

Platform administrators configure Model Pools in Admin → Model Pools or via /api/v1/pools:

1. Callable identity and catalog visibility. Each enabled pool has a human-readable slug that cannot collide with an existing model route alias. Enable Publish as callable model (publish_as_model) to list the pool in authorized GET /v1/models responses, and optionally enable Public catalog (public_catalog) to disclose its public alias and capabilities at /models/ without exposing candidate routes or Provider identities.
2. Conditional rules and fallback strategy. Rules evaluate incoming request traits in priority order—such as estimated context_length_tokens, modality, thinking, tool_calls, or structured_output—and narrow selection to matching candidate routes before applying the pool’s strategy (priority_failover, weighted_random, round_robin, or lowest_cost). Use POST /api/v1/pools/{id}/preview to dry-run rule matching before rollout.
3. Dual billing modes and strategy audit. Choose pool_tariff to charge workspaces a single published customer selling price under the pool’s own alias, or candidate_tariff to treat the pool as a pure routing layer and bill each request at the selected candidate model’s customer tariff. Pre-dispatch quota reservations hold the conservative upper bound across eligible candidates so failover cannot bypass workspace spending limits. Platform administrators can inspect GET /api/v1/pools/{id}/audit to audit candidate hit distribution, failover rates, customer revenue, and platform margin sustainability; confidential Provider costs never appear in customer workspace views or responses.

Function tools and structured JSON

Function tools and structured output are disabled by default. They are supported only on OpenAI-compatible routes that have been verified by the operator. OpenRouter routes must declare each capability explicitly:

TOML

[models.fast]
provider = "openrouter"
upstream_model = "openai/gpt-4.1-mini"
api_key_env = "OPENROUTER_API_KEY"
supports_tool_calls = true
supports_streaming_tool_calls = true
supports_structured_output = true

supports_tool_calls enables non-streaming function definitions and tool-call responses. supports_streaming_tool_calls separately enables incremental tool-call deltas and requires supports_tool_calls. Niu forwards these declarations to the configured provider; it does not run the requested functions. Keep tool execution and authorization in your application.

supports_structured_output enables response_format with json_object or json_schema for buffered and streaming requests. json_object requires a JSON object. For json_schema, Niu compiles the self-contained schema before admission and checks returned JSON against it; invalid schemas receive 400, and a mismatched buffered upstream response receives 502. Nonempty upstream refusals remain refusals. Schemas are limited to 64 KiB, 4096 JSON nodes and depth 32; regular expression compilation and backtracking are bounded. HTTP and file reference retrieval are disabled, while local references can resolve within the schema. Verify actual endpoint and model behavior before enabling the flag. Streamed JSON deltas remain provisional until [DONE]; Niu validates the complete content before sending that terminal marker, retaining at most 1 MiB of content/refusal bytes across 128 choices. Invalid final content emits an upstream_invalid_response SSE error without [DONE] and records an upstream-response failure without discarding reported terminal usage or its charge. Native Anthropic/Bedrock tool or structured-output routes and combined tools plus structured JSON remain unsupported.

Responses API text subset

The gateway implements a text-only subset of POST /v1/responses, with buffered and streamed output, for routes that explicitly declare Responses support. This is disabled by default on OpenRouter and must be enabled only after verifying the selected model and upstream endpoint:

TOML

[models.fast]
provider = "openai"
upstream_model = "gpt-4.1-mini"
api_base = "https://api.openai.com/v1"
api_key_env = "OPENAI_API_KEY"
supports_responses = true

Requests accept one non-empty string in input, plus optional string instructions, max_output_tokens, temperature, top_p, string-valued metadata, and user. Responses are forwarded to the configured /responses endpoint with the upstream model name, then returned with Niu’s public alias. Niu validates text-message, reasoning and refusal output items and keeps provider-reported input_tokens and output_tokens as the attempt’s prompt/completion usage. Missing usage remains unknown.

Set stream: true to receive Responses Server-Sent Events. Consume the terminal response event; an initial HTTP 200 does not establish completion. Reported terminal usage supplies accounting evidence, while missing usage remains unknown. Stopping the client does not prove upstream cancellation. See the Responses API subset for response and failure semantics.

Input arrays, images and audio, function tools, previous-response/conversation state, prompt templates and other fields are not supported. Unknown fields receive 400; requesting Responses on a route without supports_responses receives 501 before an attempt is created. This flag is an operator declaration, not provider conformance evidence. Native Anthropic/Bedrock routes cannot declare it.

Invalid request shapes receive 400. A valid feature request for a route without the corresponding capability receives 501 before an operation or attempt is recorded. Enabled route capabilities participate in the immutable route revision used for attempt evidence. Request-level tool definitions and schemas are forwarded as part of each request.

Text embeddings

The gateway implements POST /v1/embeddings for OpenAI-compatible routes (openai or openrouter). Embeddings are disabled by default, including on OpenRouter; declare support on a dedicated model route only after verifying that the configured upstream endpoint and model implement the requested behavior:

TOML

[models.embedding]
provider = "openai"
upstream_model = "text-embedding-3-small"
api_base = "https://api.openai.com/v1"
api_key_env = "OPENAI_API_KEY"
supports_embeddings = true
supports_embedding_dimensions = true
supports_embedding_base64 = true

The last two flags independently enable the optional dimensions and encoding_format = "base64" request fields. They default to false. For an OpenRouter route, set provider = "openrouter", use an OpenRouter model ID, and verify embedding support before setting supports_embeddings = true. These flags are operator declarations, not provider conformance evidence. Routes using another provider, or routes without the required declared capability, receive 501 before an operation or attempt is recorded. Chat completion capability does not imply embedding support.

Requests accept one non-empty text string or an array of 1–2048 non-empty strings. encoding_format is float by default and may be base64; dimensions must be an integer from 1 to 65536. Token-ID arrays, nested input, multimodal input, streaming, and unknown request fields are not supported. The gateway maps the configured upstream model, validates the response item count and vector shape, then returns the public model alias. Provider usage is required for settled usage accounting; missing usage leaves the attempt unsettled. When pricing is configured, the total UTF-8 input byte count must fit max_input_tokens, the full configured input bound is reserved before dispatch, and only reported prompt tokens are settled (completion tokens are zero by contract).

The optional route is shown commented out in config/niu.example.toml. No provider-specific conformance suite is implied by enabling it.

Optional text-route pricing

Supported OpenAI-compatible routes can reserve and settle a workspace’s platform upstream budget. This installation-only procurement configuration is separate from customer selling tariffs and invoice charges. Configure integer rates in currency nanounits per million tokens:

TOML

[models.fast.pricing]
currency = "USD"
api_prompt_rate = 1000000000
api_completion_rate = 2000000000
cash_prompt_rate = 1000000000
cash_completion_rate = 2000000000
max_input_tokens = 32768
max_output_tokens = 4096

These illustrative rates mean USD 1 per million input tokens and USD 2 per million output tokens. Replace them with the Provider’s verified rates. The compatibility API-equivalent and cash schedules are independent upstream accounting fields; neither defines customer prices. Create a matching-currency workspace upstream budget before sending priced requests.

The gateway reserves the full configured input and output bounds, not an estimated token count. The input bound is an operator assertion about the provider’s maximum billable input for this route; it is not a locally verified tokenizer limit. Choose bounds from the actual provider/model contract. Understated bounds can incur an overrun: Niu records the full liability and flags the entry instead of hiding the excess. Provider contract validation remains required before production use.

An omitted output limit is set to max_output_tokens using the upstream max_completion_tokens field. Explicit output limits must be positive and no larger than the configured bound. Providers must support that field. Pricing supports one text chat completion, including optional streaming, the text-only Responses subset above, or text embeddings. Chat pricing rejects audio, images, multiple choices and other unsupported billing dimensions; supported text-only function tools are described below. On a priced Responses request, the gateway compares the combined UTF-8 byte length of input and instructions with max_input_tokens, injects the configured output bound when omitted, and reserves both limits before dispatch. Priced embeddings compare total input bytes with max_input_tokens, reserve input capacity only, and settle provider-reported prompt usage. These bounds remain operator assertions, not tokenizer measurements. A workspace with an upstream budget cannot dispatch an unpriced request without a trusted held reservation.

Text-only function calls and tool-result conversations are also supported on token-priced Chat routes with the corresponding tool capability enabled. Niu forwards function definitions and results; your application executes and authorizes functions. Serialized messages, tool definitions, tool choice and response-format instructions count toward the conservative input byte guard. Hosted tools and additional billable modalities are unsupported. See Chat function tools for validation and streaming limits.

The immutable price revision includes the route configuration and pricing. Identical publications reuse a revision. Reservations happen before dispatch; insufficient funds or a missing matching budget prevent the upstream call. Complete provider usage settles cash and API-equivalent cost; missing usage, timeout and interrupted streams retain exposure. A recovery worker retries completed priced settlements without repeating inference.

Priced streaming requests always request final usage via stream_options.include_usage=true. Missing usage or a missing terminal event still retains the reservation.

Published routing configuration

Static TOML routes and stored Provider mappings are compiled into one immutable routing generation. Disabled stored aliases cannot fall back to a same-name static route. Changes affect new selection; already dispatched work retains its pinned route. Authorization, qualification expiry and credential cooldowns are checked live. Initial Provider imports and their publication commit together, and a normal restart preserves saved administrator edits.

Platform administrators can call GET /api/v1/routing-registry, or NiuAdminClient.getRoutingRegistryStatus(), to read the current generation and whether the Gateway’s static configuration matches it. The response contains no route definitions, endpoints, secrets or purchase prices.

Changing an already published TOML configuration requires an explicit replacement: read the current generation, update the configuration, and start the Gateway with NIU_ROUTING_EXPECTED_GENERATION set to that value. If another publication has advanced the generation, startup rejects the replacement without changing the active configuration. Reinspect before retrying, and remove the one-time setting after rollout. Restart other Gateways with matching configuration; superseded instances fail new registry reads closed. Never downgrade active route writers without restoring a compatible backup.

See the publication operation and current-input evidence for coverage and remaining qualification limits. Private Codex account routes use a separate selection mechanism and are not assigned a fabricated registry generation.

模型路由将公开的模型名称映射到上游模型供应商、供应商模型标识符、端点以及服务端凭证引用。客户端仅选择公开模型名称，不能自行提供供应商密钥或选择未经批准的上游端点。

当前的开发配置文件使用 TOML 格式，示例文件为 config/niu.example.toml。静态凭证通过环境变量名引用，并在启动时加载。

在路由上设置 public_catalog = true 可在 /models/ 和 GET /catalog/v1/models 发布其别名、声明的能力及描述性模型元数据。这是一项显式的公开披露操作。公开目录会省略供应商名称、上游模型标识符、端点、凭证和定价信息。路由默认保持隐藏状态。

公开模型目录支持按别名、显示名称、输入/输出模态以及声明的 API 能力筛选已发布的别名。目录条目会展示可用的描述、上下文与输出上限，以及指向受支持协议参考文档的链接。缺失的元数据会被省略，而不会进行估算。公开目录发现不会授予工作空间访问权限，也不代表已完成实时的模型供应商验证；工作空间密钥与客户价格仍然相互独立。

TOML

[models.fast]
provider = "openrouter"
upstream_model = "openai/gpt-4.1-mini"
api_key_env = "OPENROUTER_API_KEY"

当省略 api_base 时，openrouter 会使用位于 https://openrouter.ai/api/v1 的 OpenAI 兼容对话补全接口。OpenAI 依然通过其自身的默认端点获得支持。在管理后台和路由记录中，供应商名称保持彼此独立区分。

托管模型供应商

平台管理员可以在控制台或通过兼容性端点 /api/v1/vendors 管理共享的模型供应商（Provider）及其模型路由。模型供应商是提供模型访问服务的商业实体，并用于标识上游 API 适配器。OpenRouter 可以作为使用 openrouter 供应商适配器的模型供应商；通过该供应商提供的底层模型开发商无需单独配置 Niu 凭证。供应商记录、启用状态、模型映射和修订版本均持久化存储在 PostgreSQL 中。更新会对新请求生效。组织和工作空间操作员无法管理平台级的模型供应商。

托管配置支持 openai、openrouter 和 anthropic 适配器。对于原生 Anthropic Messages 接口，请将 api_base 设置为 https://api.anthropic.com/v1，并为该映射启用 supports_messages。连接检查使用原生 Anthropic 认证/版本请求头以及上限为 1,000 个模型的分页读取。不完整的目录不会被当作完整清单展示；在不完整检查中未出现的模型会保持为未知状态。配置持久化以及对无效凭证的实际拒绝均已验证；直接调用 Anthropic 生成以及完整目录发现仍处于待验证状态。请参阅原生协议边界。

在存储模型供应商凭证之前，请先将 NIU_VENDOR_ENCRYPTION_KEY 设置为至少 32 个字符的独立高熵密钥。Niu 使用 AES-256-GCM 加密凭证，API 响应绝不会返回明文凭证。请将该加密密钥保存在部署环境的机密存储（secret store）中，并与 PostgreSQL 分开备份。如果在未重新加密已存凭证的情况下更换该密钥，已有凭证将无法读取；目前尚未实现自动主密钥轮换。

数据库中的模型别名会覆盖 TOML 中的同名别名，即使数据库路由处于禁用状态也是如此。若配置文件的 [models] 部分为空，则仅使用托管路由。静态文件路由仍然受支持。

部署运维人员可以将 NIU_VENDOR_BOOTSTRAP_FILE 设置为一个 JSON 种子文件。该文件包含模型供应商元数据、模型路由以及保存初始凭证的环境变量名称，且绝不能包含任何明文密钥值。每个引导供应商身份仅会执行一次种子导入，并会保留后续的编辑、重命名、凭证轮换和禁用状态。

请勿将凭证写入 TOML 文件、源代码版本控制、浏览器存储或公开目录响应中。

模型池

模型池（Model Pool） 将一组候选模型路由、条件分流规则与自动故障切换策略发布为一个稳定的自定义模型名称（slug，例如 smart-coder）。应用程序与工作空间密钥只需在 POST /v1/chat/completions、POST /v1/responses 和 POST /v1/messages 的 model 字段中直接传入模型池名称即可调用，无需在业务代码中硬编码底层模型别名。

平台管理员可在管理后台 → 模型池（Admin → Model Pools）或通过 /api/v1/pools 配置模型池：

1. 可调用模型身份与目录发布。 每个启用的模型池拥有唯一的 slug，且不能与现有模型路由别名冲突。开启发布为可调用模型（publish_as_model）后，已授权的工作空间即可在 GET /v1/models 中发现该模型池；可选开启公开模型目录（public_catalog）将其公开别名与声明能力展示在 /models/ 中，同时隐藏底层候选路由与模型供应商信息。
2. 条件路由规则与容灾策略。 网关按优先级匹配请求特征（如预估 context_length_tokens 上下文长度、modality 模态、thinking 深度思考、tool_calls 工具调用或 structured_output 结构化输出），筛选出符合条件的候选模型后再执行池级调度策略（主备故障切换 priority_failover、权重随机 weighted_random、轮询 round_robin 或最低成本优先 lowest_cost）。上线前可通过 POST /api/v1/pools/{id}/preview 预览规则命中结果。
3. 双计费模式与策略审计。 选择统一池定价（pool_tariff）可将模型池作为独立自定义模型对外统一标价计费；选择按命中候选模型计费（candidate_tariff）则将模型池作为纯路由编排技术层，按实际命中的候选模型客户价目表分别计费。分发前的额度预留会按候选集保守上限冻结预算，确保故障切换不会突破工作空间支出限额。平台管理员可通过 GET /api/v1/pools/{id}/audit 审计候选命中分布、容灾切换率、客户计费收入与平台毛利合理性；机密的模型供应商采购成本绝不会暴露在客户工作空间界面或客户 API 响应中。

函数工具与结构化 JSON

函数工具（Function tools）和结构化输出默认处于禁用状态，仅在已由运维人员验证过的 OpenAI 兼容路由上受支持。OpenRouter 路由必须显式声明每项能力：

TOML

[models.fast]
provider = "openrouter"
upstream_model = "openai/gpt-4.1-mini"
api_key_env = "OPENROUTER_API_KEY"
supports_tool_calls = true
supports_streaming_tool_calls = true
supports_structured_output = true

supports_tool_calls 用于启用非流式函数定义与工具调用响应。supports_streaming_tool_calls 单独用于启用增量工具调用片段（deltas），且需要先启用 supports_tool_calls。Niu 会将这些声明转发给已配置的模型供应商，但不会执行所请求的函数。请将工具执行与权限校验保留在您的应用程序中。

supports_structured_output 为缓冲和流式请求启用带有 json_object 或 json_schema 的 response_format。json_object 要求返回 JSON 对象。对于 json_schema，Niu 会在准入前编译自包含的 Schema，并据此校验返回的 JSON；无效的 Schema 会收到 400 响应，而不匹配的缓冲上游响应会收到 502 响应。非空的上游拒绝响应（refusals）仍保留为拒绝状态。Schema 大小上限为 64 KiB、4096 个 JSON 节点和 32 层深度；正则表达式的编译与回溯均受限。HTTP 和文件外部引用检索已被禁用，但 Schema 内部的本地引用可以正常解析。启用该标志前，请先验证实际端点和模型的行为。流式 JSON 片段在收到 [DONE] 之前均为临时内容；Niu 会在发送该终止标记前校验完整内容，在最多 128 个候选项（choices）中最多保留 1 MiB 的内容/拒绝字节。若最终内容无效，网关会发出不带 [DONE] 的 upstream_invalid_response SSE 错误，并记录一次上游响应失败，同时不会丢弃已报告的最终用量或其对应费用。原生 Anthropic/Bedrock 的工具或结构化输出路由，以及同时组合使用工具与结构化 JSON 的请求目前均不受支持。

Responses API 文本子集

对于显式声明支持 Responses 的路由，网关实现了 POST /v1/responses 的纯文本子集，支持缓冲和流式输出。该功能在 OpenRouter 上默认禁用，仅可在验证所选模型和上游端点后启用：

TOML

[models.fast]
provider = "openai"
upstream_model = "gpt-4.1-mini"
api_base = "https://api.openai.com/v1"
api_key_env = "OPENAI_API_KEY"
supports_responses = true

请求在 input 中接受一个非空字符串，以及可选的字符串参数 instructions、max_output_tokens、temperature、top_p、字符串值的 metadata 和 user。请求会携带上游模型名称转发至已配置的 /responses 端点，随后以 Niu 的公开别名返回。Niu 会校验文本消息、推理和拒绝输出项，并将供应商报告的 input_tokens 和 output_tokens 保存为本次尝试的提示/补全用量。缺失的用量保持为未知状态。

设置 stream: true 可接收 Responses 服务器发送事件（SSE）。请读取终止响应事件；开始时的 HTTP 200 并不代表请求已完成。终止事件报告的用量可作为核算依据，缺失用量仍保持未知。停止客户端并不证明上游已取消。响应与失败语义见 Responses API 文本子集。

不支持输入数组、图像与音频、函数工具、历史响应/会话状态、提示词模板以及其他字段。未知字段会收到 400；在未声明 supports_responses 的路由上请求 Responses 接口，会在创建尝试记录前收到 501。该标志仅代表运维人员的声明，而非供应商合规性证明。原生 Anthropic/Bedrock 路由不能声明此标志。

无效的请求结构会收到 400。对未声明相应能力的路由发起合法的功能请求，会在记录操作或尝试之前收到 501。已启用的路由能力会纳入用于尝试存证的不可变路由修订版本中。请求级别的工具定义和 Schema 会作为每次请求的一部分向下游转发。

文本向量嵌入（Embeddings）

网关针对 OpenAI 兼容路由（openai 或 openrouter）实现了 POST /v1/embeddings。向量嵌入功能默认处于禁用状态（包括在 OpenRouter 上）；仅在验证已配置的上游端点和模型确实实现了所需行为后，才可在专用模型路由上声明支持：

TOML

[models.embedding]
provider = "openai"
upstream_model = "text-embedding-3-small"
api_base = "https://api.openai.com/v1"
api_key_env = "OPENAI_API_KEY"
supports_embeddings = true
supports_embedding_dimensions = true
supports_embedding_base64 = true

最后两个标志分别独立启用可选的 dimensions 和 encoding_format = "base64" 请求字段，默认均为 false。对于 OpenRouter 路由，请设置 provider = "openrouter"，使用 OpenRouter 模型 ID，并在设置 supports_embeddings = true 前验证其向量嵌入支持情况。这些标志仅是运维人员的声明，而非供应商合规性证明。使用其他供应商的路由，或未声明所需能力的路由，会在记录操作或尝试之前收到 501。支持对话补全（Chat completion）并不代表支持向量嵌入。

请求接受单个非空文本字符串，或包含 1–2048 个非空字符串的数组。encoding_format 默认为 float，也可设为 base64；dimensions 必须是 1 到 65536 之间的整数。不支持 Token ID 数组、嵌套输入、多模态输入、流式传输以及未知的请求字段。网关会映射已配置的上游模型，校验响应条目数量与向量维度形状，然后返回公开模型别名。已完成结算的用量核算需要依赖供应商返回的用量数据；缺失用量会使该次尝试保持未结算状态。当配置了定价时，输入的 UTF-8 总字节数不得超过 max_input_tokens，分发前会按配置的完整输入上限预留额度，且仅按报告的提示 Token 进行结算（根据契约，补全 Token 为零）。

该可选路由在 config/niu.example.toml 中以注释形式给出。启用该路由并不意味着内置了针对特定供应商的一致性测试套件。

可选的文本路由定价

受支持的 OpenAI 兼容路由可以预留并结算工作空间的平台上游预算。此项仅限安装实例内部使用的采购配置，与面向客户的销售价目表及账单费用相互独立。费率需配置为整数形式的每百万 Token 货币纳单位（nanounits）：

TOML

[models.fast.pricing]
currency = "USD"
api_prompt_rate = 1000000000
api_completion_rate = 2000000000
cash_prompt_rate = 1000000000
cash_completion_rate = 2000000000
max_input_tokens = 32768
max_output_tokens = 4096

上述示例费率代表每百万输入 Token 1 美元、每百万输出 Token 2 美元。请将其替换为已核实的模型供应商费率。兼容性的 API 等效费率表与现金费率表是彼此独立的上游核算字段，两者均不定义客户价格。在发送计费请求前，请先创建币种匹配的工作空间上游预算。

网关会按已配置的完整输入与输出上限预留额度，而非按估算的 Token 数量预留。输入上限是运维人员对该路由下供应商最大可计费输入的断言，并非本地分词器（tokenizer）校验出的限制。请根据实际的供应商/模型合同选择上限。如果低估了上限，可能会导致超支：Niu 会记录全部实际应付责任并标记该条目，而不会隐藏超出部分。在投入生产使用前，仍需完成供应商合同校验。

当请求未指定输出上限时，网关会使用上游 max_completion_tokens 字段将其设置为 max_output_tokens。显式指定的输出上限必须为正数且不超过配置的上限，同时模型供应商必须支持该字段。定价支持单次文本对话补全（含可选的流式传输）、上述纯文本 Responses 子集，或文本向量嵌入。对话计费会拒绝音频、图像、多候选项（multiple choices）及其他不受支持的计费维度；下文说明支持的纯文本函数工具。在计费的 Responses 请求中，网关会将 input 与 instructions 的合计 UTF-8 字节长度与 max_input_tokens 进行比较，在未指定输出限制时注入已配置的输出上限，并在分发前预留这两项限额。计费的向量嵌入请求会将总输入字节数与 max_input_tokens 比较，仅预留输入容量，并结算供应商报告的提示用量。这些上限仍然是运维人员的断言，而非分词器测量值。配置了上游预算的工作空间在没有可靠冻结预留的情况下，无法分发未定价的请求。

启用相应工具能力后，按 Token 计费的 Chat 路由也支持纯文本函数调用和工具结果对话。Niu 转发函数定义与结果，由您的应用执行函数并校验权限。序列化后的消息、工具定义、工具选择和响应格式指令均计入保守的输入字节限制。托管工具和其他计费模态仍不受支持。校验和流式限制见 Chat 函数工具。

不可变的价格修订版本包含路由配置与定价信息；完全相同的发布内容会复用同一个修订版本。额度预留发生在分发之前；资金不足或缺少匹配预算都会阻止发起上游调用。完整的供应商用量会结算现金成本与 API 等效成本；缺失用量、超时和中断的流则会保留风险敞口。后台恢复工作进程会重试已完成但未结算的计费记录，而不会重复发起模型推理。

计费的流式请求始终会通过 stream_options.include_usage=true 请求最终用量。如果缺失用量或缺失终止事件，预留额度仍会保持冻结。

已发布的路由配置

静态 TOML 路由和已存储的模型供应商映射会被编译为一个不可变的路由代数（routing generation）。已禁用的存储别名不会回退到同名的静态路由。配置更改仅影响新的路由选择；已分发的工作会保留其固定的路由。权限校验、资质有效期与凭证冷却状态均会实时检查。初始模型供应商导入及其发布会在同一事务中提交，正常重启会保留管理员已保存的编辑内容。

平台管理员可以调用 GET /api/v1/routing-registry 或 NiuAdminClient.getRoutingRegistryStatus()，读取当前的路由代数以及网关的静态配置是否与之匹配。该响应不包含任何路由定义、端点、密钥或采购价格。

修改已发布的 TOML 配置需要执行显式替换：先读取当前的 generation，更新配置文件，然后在将 NIU_ROUTING_EXPECTED_GENERATION 设置为该值的情况下启动网关。如果其他发布操作已经推进了路由代数，启动过程会拒绝此次替换，且不会更改当前生效的配置。重试前请重新检查状态，并在发布完成后移除该一次性环境变量设置。随后使用匹配的配置重启其他网关实例；已被取代的旧实例在执行新的注册表读取时会采取故障关闭（fail closed）策略。在未恢复兼容备份的情况下，切勿降级活跃的路由写入实例。

有关覆盖范围与剩余资质限制的详情，请参阅路由发布操作与当前输入依据。私有 Codex 账户路由使用独立的选择机制，不会被分配虚构的注册表代数。