Operate Niu

Performance benchmark

This is a historical synthetic dataset recorded on October 16, 2026 for the pinned versions below. It is not a capacity qualification of the current release or a hosted deployment. Authentication, database work and worker profiles differ between gateways, so ratios describe these configurations only. The recorded PostgreSQL configuration uses synchronous_commit = off: successful commits can precede WAL flush and recent transactions can be lost after a crash. See PostgreSQL’s durability settings. No crash-recovery or cloud-cost qualification is established by these results.

This benchmark compares Niu (v0.2.4, Rust / Axum / Tokio), New API (QuantumNous/new-api commit 45fbb14, Go / Gin / GORM), and LiteLLM Proxy (BerriAI/litellm v1.104.0, Python / FastAPI / Uvicorn + uvloop) using the open-source LiteLLM AI Gateway Benchmark (BerriAI/ai-gateway-bench) methodology.

Unlike stateless proxy micro-benchmarks that only test single-client (c = 1) forwarding without database persistence, this suite evaluates all three gateways across concurrency tiers (c = 1, 4, 16, 64), OpenAI /v1/chat/completions and native Anthropic /v1/messages routes, concurrent SSE token streaming, multi-turn coding-agent sessions (claude_code and codex), memory footprint (RSS).

Headline results

Measured on Apple M4 (10-core, 24 GB RAM, macOS arm64) with PostgreSQL 17.11 and deterministic 0ms mock-upstream on loopback (October 16, 2026).
Dimension	Niu (v0.2.4, Rust)	New API (45fbb14, Go)	LiteLLM Proxy (v1.104.0, Python)	Ratio in this dataset
Peak throughput (/v1/chat/completions, c=64)	3,238.7 RPS	865.6 RPS (peak) / 148.9 RPS (sustained)	302.2 RPS	3.74x New API / 10.72x LiteLLM
Tail latency (p99 at c=64, /v1/chat/completions)	31.54 ms (p50: 18.94 ms)	135.73 ms (peak)	338.74 ms (p50: 193.26 ms)	4.30x lower than New API / 10.74x lower than LiteLLM
Anthropic /v1/messages throughput (c=64)	2,518.8 RPS (p99: 60.61 ms)	136.2 RPS (p99: 1,475.53 ms)	232.2 RPS (p99: 397.88 ms)	10.85x LiteLLM / 18.49x New API
Concurrent SSE streaming (c=16, 40 tok/stream)	235.9 streams/s (9,436 tok/s)	100.9 streams/s (4,036 tok/s)	83.0 streams/s (3,320 tok/s)	2.34x New API / 2.84x LiteLLM
Streaming inter-token latency (ITL p50, c=16)	1.34 ms (43.0 chunks/stream)	3.55 ms (42.0 chunks/stream)	4.62 ms (43.0 chunks/stream)	2.65x faster than New API / 3.45x faster than LiteLLM
16-agent coding session (claude_code, 30 turns)	1,746.0 ms (turn p50: 55.0 ms)	4,864.9 ms (turn p50: 142.0 ms)	3,353.3 ms (turn p50: 103.0 ms)	2.79x faster than New API / 1.92x faster than LiteLLM
16-agent coding session (codex, 30 turns)	1,686.6 ms (turn p50: 54.1 ms)	4,891.9 ms (turn p50: 132.8 ms)	5,803.9 ms (turn p50: 180.8 ms)	2.90x faster than New API / 3.44x faster than LiteLLM
Memory footprint (Idle / Peak RSS)	18.2 MB / 26.4 MB	37.4 MB / 98.5 MB	158.0 MB / 194.6 MB	7.37x lower peak RAM than LiteLLM

Methodology

1. Zero-delay mock upstream (`ai-gateway-bench`)

Real Provider APIs add 200–1,500 ms of model inference latency and network variance, masking gateway overhead. Following BerriAI/ai-gateway-bench, all three gateways route requests to an identical Rust/Axum mock-upstream server on loopback (127.0.0.1:9000) configured with MOCK_TTFT_MS=0, MOCK_ITL_MS=0, and MOCK_OUTPUT_TOKENS=40.

Direct baseline verification: Before benchmarking the gateways, load-probe measured mock-upstream directly at 36,649.3 RPS (p50 = 1.83 ms, p99 = 3.02 ms at c = 64), above the gateway rates recorded in this dataset; this does not establish headroom for other workloads.
Closed-loop HTTP client: The Rust load-probe binary (reqwest + tokio, tcp_nodelay = true, persistent HTTP/1.1 connection pool) executes 32 untimed warmup requests before every timed pass and consumes the complete HTTP response body before recording microsecond latency.

2. Configuration differences & durability limits

Each gateway was compiled in release mode and configured for high-throughput operation on the same host (Apple M4 10-core, 24 GB unified memory, macOS arm64, PostgreSQL 17.11):

Gateway runtime and durability configuration during benchmark runs.
Configuration	Niu (v0.2.4)	New API (45fbb14)	LiteLLM Proxy (v1.104.0)
Runtime & HTTP stack	Rust 1.91 (–release), Axum, Tokio	Go 1.24 (-ldflags=“-s -w”), Gin (release), GORM	Python 3.12, FastAPI, Uvicorn, uvloop, httptools
Authentication check	Workspace API key SHA-256 + 5s in-memory grant cache	User token with MEMORY_CACHE_ENABLED=true	In-memory master_key (no database query)
Per-request PostgreSQL work	Transactional admission & completion: pre-dispatch operation/attempt CTE + micro-batched completion settlement (up to 64 rows / 2ms)	PostgreSQL token/user quota checks + BATCH_UPDATE_ENABLED=true usage log writes	None (stateless mode with spend logs disabled in the tested single-worker profile)

Niu performed admission and completion database work in this run. That does not establish equivalent work across the three configurations or crash-safe commit acknowledgement: PostgreSQL used asynchronous commit. The observed peak throughput ratios are 10.7x versus the tested LiteLLM profile and 3.7x versus the New API peak pass.

Detailed benchmark results

1. Non-streaming `/v1/chat/completions` (`c = 1, 4, 16, 64`)

Non-streaming POST /v1/chat/completions throughput and latency across concurrency tiers (0ms mock upstream, 0% errors across all runs).
Concurrency	Gateway	Throughput (RPS)	Mean (ms)	p50 (ms)	p95 (ms)	p99 (ms)
c = 1	Niu (v0.2.4)	233.6	4.27	4.00	5.88	9.36
c = 1	New API (45fbb14)	187.9	5.31	4.36	9.11	13.43
c = 1	LiteLLM Proxy (v1.104.0)	136.5	7.31	6.41	12.17	15.20
c = 4	Niu (v0.2.4)	300.5	13.28	13.15	17.21	19.51
c = 4	New API (45fbb14)	352.0	11.30	9.42	17.11	57.27
c = 4	LiteLLM Proxy (v1.104.0)	296.9	13.42	13.22	16.64	18.28
c = 16	Niu (v0.2.4)	1,040.0	15.32	15.07	19.31	22.26
c = 16	New API (45fbb14, peak / sustained)	612.4 / 278.7	57.23	46.87	70.92	408.08
c = 16	LiteLLM Proxy (v1.104.0)	328.0	48.62	45.07	57.23	163.16
c = 64	Niu (v0.2.4)	3,238.7	19.50	18.94	26.11	31.54
c = 64	New API (45fbb14, peak / sustained)	865.6 / 148.9	73.93	71.15	111.82	135.73
c = 64	LiteLLM Proxy (v1.104.0)	302.2	209.62	193.26	318.24	338.74

2. Native Anthropic `/v1/messages` (`c = 16` and `c = 64`)

Native Anthropic POST /v1/messages throughput and latency (claude-sonnet-4-20250514 payload).
Concurrency	Gateway	Throughput (RPS)	Mean (ms)	p50 (ms)	p95 (ms)	p99 (ms)
c = 16	Niu (v0.2.4)	911.2	17.46	16.60	23.38	26.31
c = 16	LiteLLM Proxy (v1.104.0)	237.5	66.96	63.76	77.67	175.96
c = 16	New API (45fbb14)	164.5	96.97	74.57	228.57	473.92
c = 64	Niu (v0.2.4)	2,518.8	25.15	21.10	52.97	60.61
c = 64	LiteLLM Proxy (v1.104.0)	232.2	272.18	260.97	380.42	397.88
c = 64	New API (45fbb14)	136.2	468.26	346.36	1,291.77	1,475.53

3. SSE token streaming (`c = 1` and `c = 16`, 40 tokens/stream)

SSE streaming throughput, chunk preservation, TTFT, and Inter-Token Latency (ITL).
Concurrency	Gateway	Streams/sec	Tokens/sec	Chunks/Stream	Stream p50 (ms)	TTFT p50 (ms)	ITL p50 (ms)
c = 1	Niu (v0.2.4)	138.6	5,544	43.0	6.82	6.40	0.008
c = 1	New API (45fbb14)	138.3	5,532	42.0	6.53	4.86	0.036
c = 1	LiteLLM Proxy (v1.104.0)	60.4	2,416	43.0	16.05	7.98	0.194
c = 16	Niu (v0.2.4)	235.9	9,436	43.0	62.17	5.96	1.339
c = 16	New API (45fbb14)	100.9	4,036	42.0	150.33	8.21	3.553
c = 16	LiteLLM Proxy (v1.104.0)	83.0	3,320	43.0	192.65	7.64	4.618

4. Multi-turn coding agent sessions (`claude_code` and `codex`)

We simulated 8 and 16 concurrent coding agents each executing a 30-turn coding session (240 and 480 total SSE streams with conversation context growing from 1.5 KB to 25 KB across turns):

Multi-turn coding agent session wall time and per-turn streaming latency across 8 and 16 concurrent agent sessions.
Workload	Gateway	Wall Time (ms)	Session p50 (ms)	Turn p50 (ms)	Turn p95 (ms)	Turn p99 (ms)
8 agents × 30 turns (claude_code)	Niu (v0.2.4)	1,781.6	1,754.8	54.14	71.84	147.19
8 agents × 30 turns (claude_code)	LiteLLM Proxy (v1.104.0)	2,005.3	1,984.3	62.17	90.01	145.17
8 agents × 30 turns (claude_code)	New API (45fbb14)	2,953.3	2,877.3	82.13	207.00	312.18
16 agents × 30 turns (claude_code)	Niu (v0.2.4)	1,746.0	1,693.4	55.02	68.09	82.73
16 agents × 30 turns (claude_code)	LiteLLM Proxy (v1.104.0)	3,353.3	3,318.5	103.01	141.10	245.68
16 agents × 30 turns (claude_code)	New API (45fbb14)	4,864.9	4,654.4	141.96	258.93	421.51
16 agents × 30 turns (codex)	Niu (v0.2.4)	1,686.6	1,651.7	54.05	66.91	72.50
16 agents × 30 turns (codex)	New API (45fbb14)	4,891.9	4,752.8	132.79	469.25	510.17
16 agents × 30 turns (codex)	LiteLLM Proxy (v1.104.0)	5,803.9	5,795.7	180.75	314.31	328.23

Result interpretation & architectural takeaways

The tables record outcomes for the listed versions and profiles. They do not isolate the effect of language, garbage collection, validation, logging, connection lifetime or batching. Those are hypotheses for controlled profiling, not causes established by these measurements.

Compare the same route, concurrency and pass type. New API’s c = 64 Chat peak and sustained passes differ substantially; a peak pass is not a sustained-capacity promise.
The mock workload excludes real model latency, public-network variance, failure recovery and production traffic distributions. Coding-session profiles simulate control loops; they do not establish real coding-task completion times.
RSS describes process memory in this local run, not total deployment memory or cloud cost. Database, load-generator and upstream resource use require separate accounting.
A new qualification needs pinned gateway and harness revisions, complete configuration and raw per-pass records, matched worker and durability settings, and repeated runs. Crash recovery requires separate failure testing.

For the recorded dataset and reproduction boundaries, see the Gateway performance benchmark reference. For a current local workload procedure, use Gateway load testing.

以下为 2026 年 10 月 16 日针对下方固定版本记录的历史合成基准数据集，不作为当前发布版本或托管部署的容量认证。各网关的鉴权、数据库操作与工作进程配置存在差异，因此比率仅反映对应测试配置下的结果。记录中的 PostgreSQL 配置使用了 synchronous_commit = off：成功提交可能在 WAL 刷盘前返回，进程或系统崩溃后近期事务可能丢失。请参阅 PostgreSQL 持久化设置说明。本结果不构成崩溃恢复或云基础设施成本认证。

本基准测试基于开源的 LiteLLM AI Gateway Benchmark (BerriAI/ai-gateway-bench) 测试方法，对 Niu（v0.2.4，Rust / Axum / Tokio）、New API（QuantumNous/new-api commit 45fbb14，Go / Gin / GORM）和 LiteLLM Proxy（BerriAI/litellm v1.104.0，Python / FastAPI / Uvicorn + uvloop）进行了全面对比。

不同于仅测试单客户端（c = 1）且无数据库持久化的无状态代理微基准测试，本测试套件在多个并发梯度（c = 1, 4, 16, 64）、OpenAI /v1/chat/completions 与原生 Anthropic /v1/messages 路由、并发 SSE Token 流式传输、多轮编程智能体会话（claude_code 与 codex）以及内存占用（RSS）等维度上对三款网关进行了系统评估。

核心测试结果

测试环境：Apple M4（10 核，24 GB 内存，macOS arm64），搭配 PostgreSQL 17.11 以及本地回环（loopback）上的确定性 0ms 模拟上游（2026 年 10 月 16 日）。
测试维度	Niu (v0.2.4, Rust)	New API (45fbb14, Go)	LiteLLM Proxy (v1.104.0, Python)	本数据集比率
峰值吞吐量 (/v1/chat/completions, c=64)	3,238.7 RPS	865.6 RPS（峰值）/ 148.9 RPS（持续）	302.2 RPS	New API 的 3.74 倍 / LiteLLM 的 10.72 倍
尾延迟 (c=64 下的 p99, /v1/chat/completions)	31.54 ms (p50: 18.94 ms)	135.73 ms（峰值期）	338.74 ms (p50: 193.26 ms)	比 New API 低 4.30 倍 / 比 LiteLLM 低 10.74 倍
Anthropic /v1/messages 吞吐量 (c=64)	2,518.8 RPS (p99: 60.61 ms)	136.2 RPS (p99: 1,475.53 ms)	232.2 RPS (p99: 397.88 ms)	LiteLLM 的 10.85 倍 / New API 的 18.49 倍
并发 SSE 流式传输 (c=16, 40 tok/stream)	235.9 streams/s (9,436 tok/s)	100.9 streams/s (4,036 tok/s)	83.0 streams/s (3,320 tok/s)	New API 的 2.34 倍 / LiteLLM 的 2.84 倍
流式 Token 间延迟 (ITL p50, c=16)	1.34 ms (43.0 chunks/stream)	3.55 ms (42.0 chunks/stream)	4.62 ms (43.0 chunks/stream)	比 New API 快 2.65 倍 / 比 LiteLLM 快 3.45 倍
16 智能体并发编程会话 (claude_code, 30 轮)	1,746.0 ms (单轮 p50: 55.0 ms)	4,864.9 ms (单轮 p50: 142.0 ms)	3,353.3 ms (单轮 p50: 103.0 ms)	比 New API 快 2.79 倍 / 比 LiteLLM 快 1.92 倍
16 智能体并发编程会话 (codex, 30 轮)	1,686.6 ms (单轮 p50: 54.1 ms)	4,891.9 ms (单轮 p50: 132.8 ms)	5,803.9 ms (单轮 p50: 180.8 ms)	比 New API 快 2.90 倍 / 比 LiteLLM 快 3.44 倍
内存占用 (空闲 / 峰值 RSS)	18.2 MB / 26.4 MB	37.4 MB / 98.5 MB	158.0 MB / 194.6 MB	峰值内存比 LiteLLM 低 7.37 倍

测试方法学

1. 零延迟模拟上游（`ai-gateway-bench`）

真实的模型供应商 API 会引入 200–1,500 ms 的模型推理延迟和网络波动，从而掩盖网关自身的开销。遵循 BerriAI/ai-gateway-bench 的设计，三款网关均将请求路由至本地回环（127.0.0.1:9000）上同一个 Rust/Axum mock-upstream 服务器，并配置 MOCK_TTFT_MS=0、MOCK_ITL_MS=0 和 MOCK_OUTPUT_TOKENS=40。

直连基线验证：在对网关进行基准测试之前，load-probe 直接测试 mock-upstream 测得 36,649.3 RPS（在 c = 64 下 p50 = 1.83 ms，p99 = 3.02 ms），高于本数据集中记录的各网关速率；该结果不代表其他工作负载下的冗余容量。
闭环 HTTP 客户端：Rust 编写的 load-probe 二进制程序（基于 reqwest + tokio，启用 tcp_nodelay = true 及持久 HTTP/1.1 连接池）在每轮计时测试前先执行 32 次不计时的预热请求，并在完整消费 HTTP 响应正文后记录微秒级延迟。

2. 配置差异与持久化边界

每款网关均以 Release 模式编译，并在同一台主机（Apple M4 10 核，24 GB 统一内存，macOS arm64，PostgreSQL 17.11）上配置为高吞吐量运行模式：

基准测试期间各网关的运行时与持久化配置。
配置项	Niu (v0.2.4)	New API (45fbb14)	LiteLLM Proxy (v1.104.0)
运行时与 HTTP 栈	Rust 1.91 (–release), Axum, Tokio	Go 1.24 (-ldflags=“-s -w”), Gin (release), GORM	Python 3.12, FastAPI, Uvicorn, uvloop, httptools
身份验证检查	工作空间 API 密钥 SHA-256 + 5 秒内存授权缓存	用户令牌，启用 MEMORY_CACHE_ENABLED=true	内存 master_key（无数据库查询）
单次请求 PostgreSQL 操作	事务性准入与完成结算：分发前 operation/attempt CTE 写入 + 微批处理完成结算（最多 64 行 / 2ms）	PostgreSQL 令牌/用户配额检查 + BATCH_UPDATE_ENABLED=true 用量日志写入	无（在所测试的单工作进程配置下禁用消费日志的无状态模式）

Niu 在本次运行中执行了准入与完成数据库操作。这并不代表三套配置执行了等价工作或具备崩溃安全的提交确认：PostgreSQL 使用了异步提交。观察到的峰值吞吐量比率为相对所测 LiteLLM 配置的 10.7 倍，以及相对 New API 峰值轮次的 3.7 倍。

详细基准测试结果

1. 非流式 `/v1/chat/completions`（`c = 1, 4, 16, 64`）

不同并发梯度下非流式 POST /v1/chat/completions 的吞吐量与延迟（0ms 模拟上游，所有测试运行错误率均为 0%）。
并发数	网关	吞吐量 (RPS)	平均值 (ms)	p50 (ms)	p95 (ms)	p99 (ms)
c = 1	Niu (v0.2.4)	233.6	4.27	4.00	5.88	9.36
c = 1	New API (45fbb14)	187.9	5.31	4.36	9.11	13.43
c = 1	LiteLLM Proxy (v1.104.0)	136.5	7.31	6.41	12.17	15.20
c = 4	Niu (v0.2.4)	300.5	13.28	13.15	17.21	19.51
c = 4	New API (45fbb14)	352.0	11.30	9.42	17.11	57.27
c = 4	LiteLLM Proxy (v1.104.0)	296.9	13.42	13.22	16.64	18.28
c = 16	Niu (v0.2.4)	1,040.0	15.32	15.07	19.31	22.26
c = 16	New API (45fbb14，峰值 / 持续)	612.4 / 278.7	57.23	46.87	70.92	408.08
c = 16	LiteLLM Proxy (v1.104.0)	328.0	48.62	45.07	57.23	163.16
c = 64	Niu (v0.2.4)	3,238.7	19.50	18.94	26.11	31.54
c = 64	New API (45fbb14，峰值 / 持续)	865.6 / 148.9	73.93	71.15	111.82	135.73
c = 64	LiteLLM Proxy (v1.104.0)	302.2	209.62	193.26	318.24	338.74

2. 原生 Anthropic `/v1/messages`（`c = 16` 与 `c = 64`）

原生 Anthropic POST /v1/messages 的吞吐量与延迟（使用 claude-sonnet-4-20250514 请求载荷）。
并发数	网关	吞吐量 (RPS)	平均值 (ms)	p50 (ms)	p95 (ms)	p99 (ms)
c = 16	Niu (v0.2.4)	911.2	17.46	16.60	23.38	26.31
c = 16	LiteLLM Proxy (v1.104.0)	237.5	66.96	63.76	77.67	175.96
c = 16	New API (45fbb14)	164.5	96.97	74.57	228.57	473.92
c = 64	Niu (v0.2.4)	2,518.8	25.15	21.10	52.97	60.61
c = 64	LiteLLM Proxy (v1.104.0)	232.2	272.18	260.97	380.42	397.88
c = 64	New API (45fbb14)	136.2	468.26	346.36	1,291.77	1,475.53

3. SSE Token 流式传输（`c = 1` 与 `c = 16`，每流 40 个 Token）

SSE 流式吞吐量、分块（Chunk）保持率、首字延迟（TTFT）与 Token 间延迟（ITL）。
并发数	网关	流数/秒 (Streams/sec)	Tokens/秒	分块数/流	整流 p50 (ms)	TTFT p50 (ms)	ITL p50 (ms)
c = 1	Niu (v0.2.4)	138.6	5,544	43.0	6.82	6.40	0.008
c = 1	New API (45fbb14)	138.3	5,532	42.0	6.53	4.86	0.036
c = 1	LiteLLM Proxy (v1.104.0)	60.4	2,416	43.0	16.05	7.98	0.194
c = 16	Niu (v0.2.4)	235.9	9,436	43.0	62.17	5.96	1.339
c = 16	New API (45fbb14)	100.9	4,036	42.0	150.33	8.21	3.553
c = 16	LiteLLM Proxy (v1.104.0)	83.0	3,320	43.0	192.65	7.64	4.618

4. 多轮编程智能体会话（`claude_code` 与 `codex`）

我们模拟了 8 个和 16 个并发编程智能体，每个智能体连续执行 30 轮编程会话（共计 240 个和 480 个 SSE 流，对话上下文随轮次推进从 1.5 KB 增长至 25 KB）：

8 个和 16 个并发智能体会话下的多轮编程智能体会话总耗时（Wall Time）与单轮流式延迟。
工作负载	网关	总耗时 (ms)	会话 p50 (ms)	单轮 p50 (ms)	单轮 p95 (ms)	单轮 p99 (ms)
8 智能体 × 30 轮 (claude_code)	Niu (v0.2.4)	1,781.6	1,754.8	54.14	71.84	147.19
8 智能体 × 30 轮 (claude_code)	LiteLLM Proxy (v1.104.0)	2,005.3	1,984.3	62.17	90.01	145.17
8 智能体 × 30 轮 (claude_code)	New API (45fbb14)	2,953.3	2,877.3	82.13	207.00	312.18
16 智能体 × 30 轮 (claude_code)	Niu (v0.2.4)	1,746.0	1,693.4	55.02	68.09	82.73
16 智能体 × 30 轮 (claude_code)	LiteLLM Proxy (v1.104.0)	3,353.3	3,318.5	103.01	141.10	245.68
16 智能体 × 30 轮 (claude_code)	New API (45fbb14)	4,864.9	4,654.4	141.96	258.93	421.51
16 智能体 × 30 轮 (codex)	Niu (v0.2.4)	1,686.6	1,651.7	54.05	66.91	72.50
16 智能体 × 30 轮 (codex)	New API (45fbb14)	4,891.9	4,752.8	132.79	469.25	510.17
16 智能体 × 30 轮 (codex)	LiteLLM Proxy (v1.104.0)	5,803.9	5,795.7	180.75	314.31	328.23

结果解读与边界说明

上述表格记录了所列版本与配置下的测试结果，并未单独剥离编程语言、垃圾回收、对象校验、日志记录、连接生命周期或批处理各自的影响。这些因素属于受控性能剖析的待验证假设，而非本测量结果所确立的因果结论。

请在相同路由、并发度与测试轮次类型之间进行对比。New API 在 c = 64 下的 Chat 峰值轮次与持续轮次存在显著差异；峰值轮次不代表持续容量承诺。
模拟工作负载不包含真实模型推理延迟、公网波动、故障恢复以及生产流量分布。编程会话配置仅模拟控制循环，不代表真实编程任务的完成耗时。
RSS 仅反映本次本地运行中的进程常驻内存，不代表整体部署内存或云基础设施成本。数据库、压测端与上游资源开销需另行核算。
新的性能认证需要固定网关与测试套件修订版本、完整的配置与原始分轮记录、匹配的工作进程与持久化设置以及重复运行验证。崩溃恢复需要单独的故障注入测试。

如需查看已记录的数据集与复现边界，请参阅网关性能基准测试参考文档。如需了解当前的本地工作负载测试流程，请参阅网关负载测试。

Performance benchmark

Headline results

Methodology

1. Zero-delay mock upstream (ai-gateway-bench)

2. Configuration differences & durability limits

Detailed benchmark results

1. Non-streaming /v1/chat/completions (c = 1, 4, 16, 64)

2. Native Anthropic /v1/messages (c = 16 and c = 64)

3. SSE token streaming (c = 1 and c = 16, 40 tokens/stream)

4. Multi-turn coding agent sessions (claude_code and codex)