Three dialects, streaming
POST /v1/responses, /v1/chat/completions, and Anthropic /v1/messages — streaming and non-streaming, shipped and E2E-verified against Claude Code. Point your agent at one local endpoint.
Free-tier models.
One endpoint.
ModelPanel is a bring-your-own-key gateway for free LLM tiers — one OpenAI- and Anthropic-compatible local endpoint over your own free keys, with streaming, automatic fallback, and a fusion panel that measurably beats any single free model.
Gateway M2 — Sep 2026 What's new
What it gives you
An endpoint that survives a dead model.
Health-aware routing
Task aliases and direct ids resolve through fallback chains scored by probe health, latency, and rate budget — with fallback before the first token, so a dead model never breaks your stream.
Fusion panel — measured
modelpanel/fusion fans a request across a task-routed panel and synthesizes with a judge: 95.0% vs 90.0% for the best single model on clean GSM1K. Free quotas make the retries free.
Tier-aware keys
Every key carries a tier — free-permanent, free-credits, discounted, paid. Routing defaults to free + discounted; paid keys never fire unless you opt them in with paid: allow.
Notices, not surprises
Failure notices when a model dies mid-request, and opportunity notices when the model you're using also lives somewhere healthier — one header, one body field, never an auto-switch.
Live in minutes
npx modelpanel add-key groq, npx modelpanel serve, paste two env lines into Claude Code. Keys stay in ~/.modelpanel/keys.json (0600), redacted from logs, sent only to their provider.
Architecture
Two planes: request handling and free-tier truth.
A TypeScript gateway owns ingress, canonical normalization, tier-aware routing, and config-driven egress. A Python pipeline owns what is free — probes write JSON files the gateway hot-reloads. The fusion sidecar fuses panel answers beside the gateway.
Ingress
Responses API, Chat Completions, and Anthropic Messages — streaming and non-streaming — parse into one CanonicalRequest and remember the caller's dialect for the return leg.
Router
Registry-driven chains scored by health, latency, and rate budget; tier policy filters candidates (free + discounted by default); fallback before the first token on dead models, 5xx, and 429s.
Egress adapters
Config-driven OpenAI-compatible adapters cover Groq, NIM, Cerebras, OpenRouter, Mistral, Gemini, and LLM7 — provider quirks live in config, not adapter code.
Fusion sidecar
A Python stdlib sidecar serves modelpanel/fusion: task-routed panel recipes, streaming judge synthesis, and failure masks — OpenAI-compatible on the wire.
Vault + tiers
Keys in ~/.modelpanel/keys.json (0600) with tier flags and env overrides; paid keys are listed only when explicitly allowed.
File contract
registry.json, probe_results.json — schema-versioned, fs.watch hot-reload, so a fresh probe run changes routing with no restart.
Why it matters
The moat is live free-tier data, not the proxy.
Free tiers are a moving target: DeepSeek, xAI, Alibaba, Nscale, Nebius, and AI21 all sat at 0 free models online in one weekly snapshot.
BYOK keeps compliance on your own accounts — the gateway never resells tokens and never runs your keys.
Normalize-to-canonical means an Anthropic request can fall back to OpenAI egress — cross-format routing is the point, not a hack.
Fusion is measured, not marketed: panel synthesize 95.0% vs 90.0% for the best single free model on matched GSM1K.
Opportunity notices tell you when your request could be served somewhere healthier — one header, one body field, never an auto-switch.
Evidence feeds the router: benchmark runs and probe results update the registry the gateway hot-reloads — no restart, no stale chains.
Compared to alternatives
Free-tier gateways are all static. That is the bug.
BYOK proxies exist. What none of them ship is live verification of the free tiers they route to.
Versus LiteLLM, Portkey, and OpenRouter
LiteLLM self-hosts static routes with zero markup; Portkey governs enterprise fleets; OpenRouter marks up every request. None of them verify free tiers — ModelPanel's probes and measured fusion panel are the difference.
Versus a hardcoded free-tier script
A hand-written fallback list breaks the day a provider renames or drops a model — NVIDIA's glm-5.2 vanished with an HTTP 410 mid-week. Probes and chains survive weekly churn.
Versus OpenRouter alone
OpenRouter's :free list churns and caps at 1K RPD without a topup. ModelPanel puts Groq, Gemini, NIM, Cerebras, Mistral, and more behind the same alias, each with its own budget.
Versus a cloud API gateway
Vercel and Cloudflare AI gateways proxy to paid keys and live in their cloud. ModelPanel binds 127.0.0.1, keeps keys on your machine, and treats probes as the source of truth.
Use it when
Use it when the endpoint must not break on Monday.
Use it for Claude Code or coding agents on free models without fixing configs every week — the Anthropic dialect is E2E-tested against Claude Code.
Use it when keys must stay on your machine — 127.0.0.1 bind, no cloud, no telemetry.
The gateway is the delivery mechanism; the probe pipeline is the moat — probes are truth, scrapes are the slow backstop.
Not for reselling tokens or multi-user hosted SaaS — explicit non-goals.
FAQ
Questions, answered.
What is ModelPanel?
ModelPanel is a bring-your-own-key gateway for free LLM tiers. It exposes one OpenAI- and Anthropic-compatible local endpoint over your own free-tier keys, with streaming, automatic fallback, and a fusion panel that measurably beats any single free model.
Which providers does ModelPanel support?
The gateway ships with Groq, NVIDIA NIM, Cerebras, OpenRouter, Mistral, Google Gemini, and LLM7 — and any provider in the tracker is addable via a registry entry, no code change. 30+ providers are tracked in the free-tier index.
Is ModelPanel free?
The gateway is open source and local-first. You bring your own free-tier API keys — ModelPanel never resells tokens and never charges per request.
How does ModelPanel route around dead models?
A Python pipeline probes each provider every 8 hours and marks models dead on failures. The router scores healthy candidates by health, latency, and rate budget and falls back before the first token — with X-ModelPanel-Provider and -Model headers showing who served.
Does ModelPanel work with Claude Code?
Yes — it's the first-class use case and E2E-tested. Point ANTHROPIC_BASE_URL at the local gateway and Claude Code streams off free models. OpenAI SDK apps work via /v1/responses and /v1/chat/completions.
Are my API keys safe with ModelPanel?
Keys live in ~/.modelpanel/keys.json (chmod 600) or env vars, are redacted from logs, and are sent only to the provider that owns them. The gateway binds 127.0.0.1 by default.
How is ModelPanel different from LiteLLM or OpenRouter?
LiteLLM and OpenRouter assume paid keys and ship static model lists. ModelPanel's moat is live free-tier verification plus a measured fusion panel — 95.0% vs 90.0% for the best single free model on clean GSM1K.
What are the key tiers?
Every key carries a tier: free-permanent, free-credits, discounted, or paid. Routing considers free and discounted tiers by default; paid keys appear in /v1/models and fire only when you set paid: allow on them.
Benchmark
Fusion, measured — not marketed.
Matched GSM1K (contamination-free math, n=40): the panel with judge synthesis against the strongest single free model and self-consistency voting on the same data.
ModelPanel fusion panel 95.0%
Best single free model 90.0%
Self-consistency baseline 82.5%
Panel members picked from live probe evidence by panel_selector — the router and the benchmarks read from the same registry.
Point your agent at one endpoint. Stop fixing dead models.
ModelPanel: an OpenAI- and Anthropic-compatible gateway on your own free-tier keys — streaming, tier-aware routing, and a fusion panel that measurably beats any single free model.
Read the design spec