Portkey vs LiteLLM in 2026: AI Gateways, Latency & TCO
Engineering and 36-month TCO comparison of Portkey vs LiteLLM vs Helicone and compiled Go/Rust AI gateways in 2026: streaming overhead, enterprise SSO walls, PostgreSQL 17 FORCE RLS token-budget ledgers, and 120M req/mo math.

On this page

The Portkey vs LiteLLM question usually shows up when large language models and AI agents move from a demo to real traffic. Any AI gateway comparison also lists Helicone, Maxim Bifrost, Kong AI Gateway, and Agent Router (formerly Envoy AI Gateway). Direct SDK calls are fine at ten requests per minute. At 4,000 req/min, one Anthropic 529 overload burst or a runaway agent loop gets expensive fast.
An AI gateway is a proxy that sits inline on every LLM call and handles routing, authentication, budgets, caching, guardrails, and failover. Portkey and LiteLLM are the two most compared options: Portkey is a managed or hybrid control plane, while LiteLLM is an open-source SDK and proxy you host yourself.
Our Langfuse vs LangSmith piece covers asynchronous tracing and offline evals. This guide covers the synchronous gateway: virtual keys, atomic quotas, PII and Model Context Protocol (MCP) guardrails, caching, and pre-first-byte failover.
At Idealogic, our AI software development, AI agent development, AI integration services, and custom software development teams build inference planes across fintech, healthtech, and social and creator economy. Below: the gateways compared, a PG17 FORCE RLS ledger with a pre-first-byte SSE router, and a 36-month TCO at 120M req/mo across 45 seats. Still scoping the model layer? Start with our LLM development guide.
The short version
- Portkey, now part of Palo Alto Prisma AIRS. Palo Alto Networks completed the Portkey acquisition on May 29, 2026. Portkey-AI/gateway stays MIT-licensed, while Guardrails and MCP Gateway auth sit in paid tiers.
- LiteLLM: fast to adopt, with an SSO wall. BerriAI/litellm (MIT outside its enterprise folder) unifies 100+ providers. SAML SSO beyond 5 users, JWT auth, RBAC budgets, and audit logs need LiteLLM Enterprise. Its own benchmarks show single-digit-ms overhead at 1,000 RPS.
- Helicone and the compiled gateways cover the rest. Helicone ($79 Pro, $799 Team, Enterprise on request) is an edge proxy with analytics logging. Maxim Bifrost (Go, Apache 2.0) and Agent Router, the renamed Envoy AI Gateway, cover compiled self-hosting.
- At 120M req/mo, 45 seats, and HIPAA plus PCI DSS v4.0.1, an owned gateway wins TCO. In our 36-month model it costs $102,000 (CEE) vs $209,400 for Helicone Ent., $240,900 for LiteLLM Ent., and $267,000 for Portkey Ent.
Which one should you pick?
- Portkey: managed guardrails and MCP governance, inside Prisma AIRS.
- LiteLLM: a self-hosted open-source proxy, with quote-based pricing for SSO and RBAC.
- Owned gateway: tens of millions of requests a month under HIPAA or PCI rules, no per-seat fees.
2026 AI gateway internals & licensing: Portkey vs LiteLLM vs Helicone vs Go/Rust
Model adapters won't settle portkey vs litellm: every gateway routes OpenAI, Anthropic, Bedrock, and Vertex AI. Three-year TCO splits on SSE concurrency, SSO and RBAC walls, MCP auth, and atomic quotas.
1. Portkey (MIT TypeScript core + Prisma AIRS Enterprise): Guardrails & MCP governance
Portkey is the better fit when you'd rather buy guardrails and MCP governance than build them. It splits a stateless TypeScript proxy (Portkey-AI/gateway, routing to 1,600+ LLMs, with sub-millisecond gateway processing when self-hosted) from governance. Hosted Portkey adds network hops, so that isn't end-to-end latency.
- Prisma AIRS acquisition: Palo Alto Networks announced the deal on April 30, 2026 and closed it on May 29, 2026 (about $117M).
Portkey-AI/gatewayandPortkey-AI/modelsstay MIT. Enterprise tiers typically include Hybrid VPC workers, HIPAA BAAs, longer log retention, and SAML/SCIM. - Synchronous Guardrails: Inline
checks(block,mask,flag) screen PII, JSON schemas, and prompt injections before the stream starts. - Centralized MCP Gateway auth handles OAuth/Okta brokering and detailed audit logging across Model Context Protocol (MCP) tools.
Looking at Portkey alternatives after the acquisition? The MIT gateway still runs on its own, and it's one base for the owned build below.
2. LiteLLM vs Portkey: open-source proxy speed vs enterprise walls
LiteLLM is the better fit when your team wants an open-source proxy it fully controls. In litellm vs portkey trials, teams ship litellm fast, then hit two walls as they scale:
- The enterprise license boundary: per the LiteLLM LICENSE, code under
litellm/enterprise/has its own license. That's where SAML/OIDC SSO above 5 users, JWT auth, Org/Team RBAC, audit logs, and Vault integration live. LiteLLM enterprise pricing is quote-based and usage-based; we model $36k/yr as an assumption, not a quote. - Capacity at high streaming concurrency: LiteLLM now describes itself as a Rust core with a Python SDK. Its performance docs report about 3.25 ms added latency, and the README cites about 8 ms P95 at 1,000 RPS. Hundreds of Server-Sent Events (SSE) streams with callbacks and spend logging still need more workers and replicas, so size from your own load test.
Teams looking at LiteLLM alternatives usually do it for the SSO and RBAC license, not for raw speed.
3. Helicone vs Portkey & compiled Go proxies (Bifrost, Agent Router, Kong, Cloudflare)
- Helicone vs Portkey: Helicone is an edge proxy gateway with analytics logging ($0 Hobby with 10,000 free requests, $79/mo Pro, $799/mo Team, usage on top, Enterprise on request). It's good at lean caching and cost tracking; Portkey goes further on MCP auth. Helicone says it's now part of Experiential Labs, so check roadmap and SLAs before a multi-year deal.
- Compiled gateways: Maxim Bifrost (Go, Apache 2.0) ships semantic caching and MCP routing, with a vendor-reported 11 microseconds of overhead at 5,000 RPS. Kubernetes shops run Agent Router (Go, Apache 2.0, built on Envoy Gateway), Kong teams use Kong AI Gateway, and edge apps use Cloudflare AI Gateway.
Production blueprint: PG17 token-budget ledger and pre-first-byte SSE failover
Across multi-tenant SaaS and AI API integration stacks, naive proxies ship two bugs. The first is a budget overrun race (TOCTOU): twenty parallel SSE calls each check "spent is under the cap" and all pass before any stream finishes. The second is mid-stream provider splicing. Failing over after the gateway has sent 200 OK corrupts tool JSON and throws away Anthropic prompt caching (cache reads typically bill at 0.1x of the base input price, though some newer models differ) and AWS Bedrock caching, which Bedrock bills separately.
We fix both with a PostgreSQL 17 ledger that stores money as integer microdollars (one dollar is 1,000,000 micros), Row-Level Security with FORCE RLS, and a router that commits to a provider only after the first clean SSE frame. The same pattern runs on PostgreSQL 18. These are the decisions that matter:
The router is the part most gateways get wrong. It skips any provider whose circuit breaker is open, sends the request, and treats a 429, 529, 5xx or empty body as a failure it can safely retry on the next target, because nothing has reached the client yet. It then reads the first SSE frame before answering HTTP 200 and fails over if that frame is an overload or error event. Once a clean first frame arrives, the provider is locked for that request, and if every target fails the gateway returns a 503.
Because the gateway reads the first frame before it answers the client, a 429 or 529 can still fail over without splicing JSON mid-sentence (429 is a rate limit and 529 a temporary overload, and both are retryable with backoff). OpenTelemetry GenAI spans (the semantic conventions are still in development) export asynchronously, and the hold ledger covers OWASP LLM10:2025 Unbounded Consumption for spend.
Portkey vs LiteLLM TCO: 36 months at 120M requests per month
Prices checked November 2026. We model a regulated B2B SaaS gateway over 36 months: 120M monthly LLM requests (about 2,780 req/min around the clock, 4,000 req/min in daytime or about 67 RPS, with bursts assumed at 260+ RPS of SSE), 45 seats, Okta SAML SSO, inline PII guardrails, 90-day logs, and SOC 2 + HIPAA BAA + PCI DSS v4.0.1 VPC residency. Model tokens are excluded as equal. Sizing includes headroom, since the workload never reaches 500+ RPS.
Warning: Portkey, LiteLLM, and Helicone don't publish enterprise prices. Vendor figures below are our planning assumptions, not quotes.
Labor compares $75/hr senior CEE with $165/hr US agency rates (2.2x). For context on regional labour cost differences, see the Eurostat hourly labour cost table (lc_lci_lev) and US DOL / BLS Software Developers (15-1252.00):
- Option A: Portkey Ent. Hybrid VPC (Prisma AIRS + 120M req/mo + 45 seats):
- Setup (80h @ $75/hr CEE = $6,000; $165/hr US = $13,200): VPC workers, PrivateLink, SAML/SCIM, MCP rules, and PG17 webhooks.
- 36-mo SaaS + VPC ($6,800/mo x 36 = $244,800): Portkey Ent. ($5,900/mo, our planning assumption, not a quote) + VPC/ALB ($900/mo) + 6h/mo (216h) ops ($16,200 CEE / $35,640 US) = $267,000 ($293,640 US).
- Option B: LiteLLM Ent. Self-Hosted (BerriAI key + EKS):
- Setup (140h @ $75/hr CEE = $10,500; $165/hr US = $23,100): EKS
uvicornpods, RDS, Valkey, SAML/JWT RBAC, and Vault. - 36-mo key + AWS ($5,500/mo x 36 = $198,000): LiteLLM key ($3,000/mo = $36k/yr, assumed; real pricing is quote-based) + AWS ($2,500/mo: 8x c7g.2xlarge at $1,480/mo as our conservative sizing, RDS $520/mo, Valkey $260/mo, ALB/NAT $240/mo) + 12h/mo (432h) SRE ($32,400 CEE / $71,280 US) = $240,900 ($292,380 US).
- Setup (140h @ $75/hr CEE = $10,500; $165/hr US = $23,100): EKS
- Option C: Helicone Ent. (120M req/mo + 90-day logs + HIPAA / SSO):
- Setup (80h @ $75/hr CEE = $6,000; $165/hr US = $13,200): Edge headers, SAML SSO, cache rules, and webhooks.
- 36-mo SaaS ($5,200/mo x 36 = $187,200): Team already lists SOC 2 and HIPAA, but SAML SSO needs Enterprise, assumed at $62.4k/yr for 120M req/mo and 45 seats + 6h/mo (216h) ops ($16,200 CEE / $35,640 US) = $209,400 ($236,040 US).
- Option D: Owned Gateway (Bifrost /
Portkey-AI/gateway+ PG17 FORCE RLS + Valkey):- Build (520h @ $75/hr CEE = $39,000; $165/hr US = $85,800): Compiled VPC proxy (no license or seat fees), PG17 ledger, SSE failover, PII/MCP guards, OIDC UI, and OTel.
- 36-mo AWS ($1,150/mo x 36 = $41,400): 3x c8g.large ($290/mo), Multi-AZ RDS PG17 ($420/mo), Valkey ($240/mo), and ALB/KMS/S3 ($200/mo) + 8h/mo (288h) ops ($21,600 CEE / $47,520 US) = $102,000 ($174,720 US).
Sensitivity check: Sub-5M req/mo crossover & CapEx payback
- Below 5M req/mo (5 or fewer seats, no HIPAA/PCI VPC rule), skip the custom build. Run Helicone Pro ($79/mo), Cloudflare AI Gateway, or OSS
Portkey-AI/gateway/ LiteLLM, plus our $9,000 PG17 ledger sprint. - Payback at 120M req/mo ($75/hr CEE): Option D OpEx is $1,750/mo ($1,150 AWS + $600 ops) vs $7,250/mo (Portkey), $6,400/mo (LiteLLM), and $5,650/mo (Helicone). The extra CapEx pays back in 6.00 months vs Portkey ($33,000 / $5,500/mo), 6.13 months vs LiteLLM ($28,500 / $4,650/mo), and 8.46 months vs Helicone ($33,000 / $3,900/mo). Lower real quotes stretch those months.
AI gateway build cost: owned gateway vs Portkey and LiteLLM
Our AI software development, AI agent development, and custom software development teams ship fixed-milestone gateways at $75/hr senior CEE vs $165/hr US agency rates (2.2x). Agent budgets are in our AI agent development cost guide.
Inside the worked 520-hour ($39,000 CEE vs $85,800 US) owned AI gateway build
- 1. Proxy & SSE Failover (25% / 130h / $9,750 CEE vs $21,450 US): Multi-AZ c8g Go (Bifrost) or
Portkey-AI/gatewaypods, breakers, and prefix-safe failover. - 2. PG17 FORCE RLS Ledger (25% / 130h / $9,750 CEE vs $21,450 US): Tenant budget and hold tables, restrictive RLS policies, OIDC UI, and the hold sweeper.
-
- PII & MCP guardrails (20% / 104h / $7,800 CEE vs $17,160 US) cover sub-3ms PAN/SSN/PHI redaction, JSON tool checks, and MCP OAuth brokering.
- 4. Valkey Cache & OTel (20% / 104h / $7,800 CEE vs $17,160 US): Tenant-scoped semantic caching, rate buckets, and
gen_ai.*OTLP export. -
- Load QA and cutover (10% / 52h / $3,900 CEE vs $8,580 US): 1,500 RPS k6 SSE chaos drills (429/529), hold audits, and canary cutover.
Concurrent SSE streams that blow past tenant token caps, or splice JSON mid-sentence, cost far more than a millisecond of proxy overhead. In our 120M-request model, a compiled proxy plus a PostgreSQL 17 microdollar hold ledger removes per-seat SSO fees and pays back in about six months.
Portkey vs LiteLLM decision checklist for CTOs
Where we've built quota-governed AI systems:
- Multi-tenant LLM SaaS (Pressmerit): Quota-metered inference and prompt caching across 26 industries.
- Creator monetization (OnePage): High-concurrency streaming and ledgers for social and creator economy bursts.
- HIPAA & fintech ledgers (AddMed & Fese): Zero-PHI egress and settlement under HIPAA and PCI DSS v4.0.1.
Five checks settle Portkey vs LiteLLM vs Helicone vs an owned gateway in 2026:
- Under 5M req/mo with 5 or fewer seats? Run OSS LiteLLM or Helicone Pro ($79/mo).
- Already standardized on Prisma AIRS? Portkey Ent. Hybrid VPC is the natural fit.
- Mostly need caching and cost analytics? Helicone Team or Enterprise, after a roadmap check.
- On Kubernetes Gateway API? Agent Router (formerly Envoy AI Gateway) or Maxim Bifrost, both Apache 2.0.
- At 25M to 120M+ req/mo with 20+ SSO seats and HIPAA/PCI DSS v4.0.1? Build an owned gateway with a PG17 FORCE RLS ledger ($102k 36-mo TCO in our model). Contact our team for a 48-hour audit.

Deputy CEO at Idealogic. Overseeing operational execution, product discovery, client partnerships, and engineering squad delivery.
Frequently asked questions
Choose Portkey for a managed or hybrid-VPC control plane with synchronous PII and prompt-injection guardrails, Model Context Protocol governance, and enterprise security inside Palo Alto Networks Prisma AIRS. Choose LiteLLM for an MIT-licensed SDK and self-hosted proxy across 100+ providers, with no external SaaS hop for telemetry.
Palo Alto Networks announced the Portkey acquisition on April 30, 2026 and completed it on May 29, 2026. Portkey is now the AI gateway inside Prisma AIRS, while the Portkey-AI/gateway repository and the Portkey-AI/models pricing registry stay MIT-licensed on GitHub. Hosted Portkey is still sold, but enterprise deals may bundle broader Prisma security features, so confirm pricing with sales.
LiteLLM's SDK and proxy are MIT-licensed, except the code under litellm/enterprise/, which has its own commercial license. SAML and OIDC single sign-on beyond five users, JWT auth, Organization and Team admin RBAC, audit logs, and secret-manager integrations such as HashiCorp Vault need a paid LiteLLM Enterprise contract.
BerriAI doesn't publish a price list. The LiteLLM docs describe Enterprise pricing as usage-based and quoted on request. Our TCO model below assumes $36,000 per year as a planning number, not a quote. Get a written quote first.
Not as much as older comparisons claim. LiteLLM now describes itself as a Rust core with a Python SDK, and its published benchmarks show about 8 ms P95 overhead at 1,000 RPS, with a 4 vCPU, 8 GB, 4-worker setup reaching roughly 1,200 RPS. Overhead grows with long-lived SSE streams, logging callbacks, and spend writes, so load-test your own traffic and plan extra replicas.
Helicone is an edge proxy gateway with analytics logging, published tiers ($79 Pro and $799 Team per month, plus usage), and simple header-based setup. Portkey goes deeper on runtime orchestration: conditional fallback trees, synchronous guardrail hooks, and Model Context Protocol tool governance. Helicone fits lean cost observability, Portkey fits policy-heavy agent routing. Helicone is now part of Experiential Labs.
A plain SELECT that checks spent_usd against monthly_limit before each LLM call creates a time-of-check to time-of-use race: twenty parallel agent requests can all pass before any stream finishes. In PostgreSQL 17 (the same pattern works on PostgreSQL 18), use FORCE ROW LEVEL SECURITY, reserve worst-case microdollars atomically before opening the stream, and settle the exact input, cached, and output microdollars in a locked hold row when the stream closes or aborts.
Once your gateway has sent HTTP 200 text/event-stream headers and the first content or tool-call token, switching from Anthropic Claude to OpenAI GPT mid-sentence corrupts partial JSON arguments, breaks tool-call IDs, and produces incoherent prose. Fail over only on HTTP 429, 529, or 5xx errors and TTFT timeouts that happen before the first content chunk goes out.
At Idealogic's senior Central and Eastern European rate of $75 per hour (versus $165 per hour at US agencies), a PostgreSQL 17 token-budget ledger sprint costs $9,000 to $12,000. A managed Portkey, LiteLLM, or Helicone enterprise setup runs $13,500 to $21,000. A 520-hour owned compiled gateway costs $39,000 ($85,800 at US rates) and, in our model, saves $107,400 to $165,000 over thirty-six months.
More from the journal

Merge vs Paragon in 2026: Unified API, Embedded iPaaS & TCO
Engineering and 36-month TCO comparison of Merge.dev vs Paragon vs Nango in 2026: per-linked-account vs per-tenant billing at 800 connections, Common Models vs ActionKit, an AWS KMS envelope OAuth2 vault on PostgreSQL FORCE RLS, and itemized payback math.

Deepgram vs AssemblyAI in 2026: Latency, VAD & ASR TCO
Deepgram vs AssemblyAI and OpenAI Realtime ASR in 2026, compared on engineering and 36-month TCO: Flux native End-of-Turn vs Universal-3.6 Pro Realtime, Nova-3 keyterm prompting, PII audio redaction, silence billing on streaming, Silero VAD, and L4 GPU math.

PlanetScale vs Neon in 2026: Metal NVMe, CoW & PG17 TCO
Engineering and 36-month TCO teardown of PlanetScale for Postgres vs Neon (Databricks) and AWS Aurora I/O-Optimized in 2026: local NVMe Metal unlimited IOPS vs disaggregated Pageservers, CoW branching, Neki sharding preview, FORCE RLS on PG17, and 1.2 TB math.