Quickstart

Install the gateway, add your keys, seed the registry, serve, and point Claude Code or any OpenAI SDK app at one local endpoint.

Install

1. Install the gateway

   Requires Node 22+ and pnpm (plus Python 3 for the fusion sidecar):

   cd gateway
   pnpm install
   pnpm build        # or use `pnpm dev` (tsx, no build step)

2. Add your provider keys (BYOK)

   Any provider with an OpenAI-compatible endpoint works — these seven ship with adapters:

   Provider	Key URL	Notes
   Groq	console.groq.com/keys	fast inference, generous limits
   Google Gemini	aistudio.google.com/app/apikey	1M-context Flash models
   NVIDIA NIM	build.nvidia.com/settings/api-keys	100+ models, no daily cap
   OpenRouter	openrouter.ai/workspaces/default/keys	one key, many models
   Cerebras	cloud.cerebras.ai	email-only signup
   Mistral	console.mistral.ai/api-keys	large monthly allowance
   LLM7	token.llm7.io	30 RPM (120 with token)

   Add each key:

   npx modelpanel add-key groq          # prompted; stored in ~/.modelpanel/keys.json (chmod 600)
   npx modelpanel add-key nvidia_nim
   # or via env: MODELPANEL_GROQ_KEY=gsk_... — env wins over the vault file

   Each key carries a tier flag (free-permanent, free-credits, discounted, paid) — most of these providers have usable free tiers, and that’s the cheapest way to start. Paid keys work too; they only fire when you opt them in with paid: allow.

3. Seed the registry

   The gateway boots from registry/registry.json and fail-fasts at boot without it — a fresh clone has none:

   mkdir -p registry && cp gateway/test/fixtures/registry.json registry/registry.json

   That ships a sample roster; the registry-gen + prober pipeline owns the real, probe-verified catalog.

4. Serve

   pnpm dev &        # or: npx modelpanel serve
   # → modelpanel gateway on http://127.0.0.1:8787 (sidecar: spawned)

   serve also spawns the fusion sidecar (Python, 127.0.0.1:8791) — the engine behind modelpanel/fusion. /health reports both.

5. Point your tools at it

   Claude Code (Anthropic dialect):

   export ANTHROPIC_BASE_URL=http://127.0.0.1:8787
   export ANTHROPIC_AUTH_TOKEN=local   # any non-empty token; the gateway has no auth
   export ANTHROPIC_MODEL=modelpanel/fusion

   OpenAI SDK apps / Codex-style (streaming included):

   export OPENAI_BASE_URL=http://127.0.0.1:8787/v1

Smoke test

curl -s http://127.0.0.1:8787/health
curl -s http://127.0.0.1:8787/v1/models | jq '.data[].id'

# streamed chat
curl -N http://127.0.0.1:8787/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"free/reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Every response carries x-modelpanel-provider / x-modelpanel-model headers naming who served; x-modelpanel-fallback-from appears when the first choice failed over.

Try the fusion panel

curl -N http://127.0.0.1:8787/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"modelpanel/fusion","stream":true,"messages":[{"role":"user","content":"hi"}]}'

Continue to Core concepts for routing and notices, or the API reference for every command and endpoint.

Source of truth for this page: ModelPanel · open-assistants-lab/ModelPanel