AiT AI GatewayIntelligent IT · MSP control plane

Model Performance — latency & throughput comparison.

P50/P95/P99 latency percentiles, throughput, and success rates across all gateway models. Computed from the last 7 days of traffic.

Window:7d14d30d
Best P95 Latency
620ms
Cerebras
Fastest Avg Latency
310ms
Cerebras
Highest Success Rate
100%
Gemini Pro
Most Used Model
445
reqs — DS Chat

Performance comparison

ModelRequestsP50P95Avg LatencyTokens/reqSuccess %
Cerebras · Llama 3.3 70B312280ms620ms310ms4K99%
Groq · Llama 3.3 70B187310ms780ms360ms4K98%
DeepSeek Chat4451.1s3.2s1.4s5K98%
DeepSeek Reasoner644.2s9.8s5.1s7K98%
Claude Haiku 4.5231850ms2.1s980ms4K98%
Gemini 2.5 Flash98520ms1.4s610ms3K99%
Gemini 2.5 Pro422.2s5.6s2.6s4K100%

Latency percentiles — P50 / P95 / P99

0ms3.6s7.3s10.9s14.5sP50P95P99cerebras/llama-3.3-70b P50: 280mscerebras/llama-3.3-70b P95: 620mscerebras/llama-3.3-70b P99: 940msCerebrasgroq/llama-3.3-70b P50: 310msgroq/llama-3.3-70b P95: 780msgroq/llama-3.3-70b P99: 1.2sGroqdeepseek-chat P50: 1.1sdeepseek-chat P95: 3.2sdeepseek-chat P99: 5.8sDS Chatdeepseek-reasoner P50: 4.2sdeepseek-reasoner P95: 9.8sdeepseek-reasoner P99: 14.5sDS Reasonclaude-haiku-4-5 P50: 850msclaude-haiku-4-5 P95: 2.1sclaude-haiku-4-5 P99: 3.4sHaiku 4.5gemini-2.5-flash P50: 520msgemini-2.5-flash P95: 1.4sgemini-2.5-flash P99: 2.2sGemini Flashgemini-2.5-pro P50: 2.2sgemini-2.5-pro P95: 5.6sgemini-2.5-pro P99: 8.9sGemini Pro

Error & throughput summary

1Cerebras
312 reqs2 err99%
2Groq
187 reqs3 err98%
3DS Chat
445 reqs8 err98%
4DS Reason
64 reqs1 err98%
5Haiku 4.5
231 reqs4 err98%
6Gemini Flash
98 reqs1 err99%
7Gemini Pro
42 reqs0 err100%
AiT AI Gateway · powered by Manny, the Intelligent IT assistant. © Intelligent Group (DBA Intelligent IT). All data on this surface is mock — no real virtual keys are ever exposed in product UI.