docs: update README for current features, drop unused admin token
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
This commit is contained in:
42
README.md
42
README.md
@@ -66,8 +66,11 @@ Two auth systems that never mix:
|
||||
| Proxy routes (`/v1/*`) | `Bearer sk-xxx` | sha256 of the key → lookup in `virtual_keys`; check revoked / expired / model allowlist |
|
||||
| Admin routes (`/api/admin/*`) | `Bearer <OIDC JWT>` | `go-oidc` `Verify()` → `groups` claim → must contain the admin group |
|
||||
|
||||
**Session keys:** `POST /auth/exchange` trades an Authentik OIDC JWT for an
|
||||
`sk-sess-xxx` virtual key scoped to that group's model allowlist.
|
||||
Dashboard login uses the OIDC authorization code flow: `GET /auth/login`
|
||||
redirects to the provider, and `GET /auth/callback` exchanges the code server
|
||||
side and hands the ID token to the SPA. Every signed-in user can create keys,
|
||||
see their own logs and use the test chat. Members of the admin group also
|
||||
manage providers, the model catalog, routing and settings.
|
||||
|
||||
Provider credentials are encrypted with AES-256-GCM using the 32-byte key in
|
||||
`GATEWAY_SECURITY_ENCRYPTION_KEY` and decrypted only in memory at request time.
|
||||
@@ -83,14 +86,17 @@ is loaded automatically; in production, set real environment variables.
|
||||
|---|---|---|---|
|
||||
| `GATEWAY_SERVER_PORT` | `8080` | no | HTTP listen port |
|
||||
| `GATEWAY_SERVER_ENV` | `development` | no | `development` \| `production` (controls log format/level) |
|
||||
| `GATEWAY_SERVER_PUBLIC_URL` | — | **yes** | canonical external URL, used for the OIDC redirect URI |
|
||||
| `GATEWAY_DB_URL` | — | **yes** | `postgres://user:pass@host:5432/db` |
|
||||
| `GATEWAY_DB_MAX_CONNS` | `20` | no | pool max connections |
|
||||
| `GATEWAY_DB_MIN_CONNS` | `2` | no | pool min connections |
|
||||
| `GATEWAY_DB_MAX_CONN_LIFETIME` | `30` | no | minutes |
|
||||
| `GATEWAY_OIDC_ISSUER` | — | **yes** | OIDC issuer URL |
|
||||
| `GATEWAY_OIDC_CLIENT_ID` | — | **yes** | OIDC client ID |
|
||||
| `GATEWAY_OIDC_CLIENT_SECRET` | — | **yes** | OIDC client secret (confidential client) |
|
||||
| `GATEWAY_OIDC_ADMIN_GROUP` | — | **yes** | group required for admin access |
|
||||
| `GATEWAY_SECURITY_ENCRYPTION_KEY` | — | **yes** | 64 hex chars (32 bytes) for AES-256-GCM |
|
||||
| `GATEWAY_CATALOG_SYNC_INTERVAL` | `30m` | no | how often provider model lists are refreshed |
|
||||
|
||||
Config is validated on startup; missing required values or a wrong-length
|
||||
encryption key cause the server to exit immediately.
|
||||
@@ -166,8 +172,8 @@ migrations/000002_virtual_keys.down.sql
|
||||
| `ollama` | local | OpenAI-compatible |
|
||||
| `openrouter` | api key | OpenAI-compatible; HTTP-Referer/X-Title attribution; reports pricing |
|
||||
| `openai` | api key | reads `cached_tokens` from the usage response |
|
||||
| `anthropic` | api key | Messages API ↔ OpenAI translation; injects `cache_control` on the system prompt + penultimate message |
|
||||
| `github-copilot` | subscription | exchanges a GitHub PAT for a ~30-min Copilot token, mutex-guarded refresh |
|
||||
| `anthropic` | api key | Messages API ↔ OpenAI translation, including streamed tool calls; injects `cache_control` on the system prompt + penultimate message |
|
||||
| `github_copilot` | subscription | exchanges a GitHub token for a ~30-min Copilot token, mutex-guarded refresh |
|
||||
|
||||
## Model addressing
|
||||
|
||||
@@ -179,11 +185,12 @@ timer and cached in memory.
|
||||
|
||||
- **`any/<slug>`** — a reserved prefix that fans out across every provider
|
||||
serving that model and picks one by the configured **routing strategy**
|
||||
(`round_robin` | `lowest_cost` | `priority`, in `settings`).
|
||||
- **Curated aliases** — admins can define a friendly name mapping to one or more
|
||||
catalog entries with priority/weight for local-first/cloud-backup failover.
|
||||
(`round_robin` | `lowest_cost`, set from the dashboard Routing page).
|
||||
- **Admin overrides** — per-model `enabled`/`blocked`, parameter overrides,
|
||||
pricing, and rate limits live alongside the discovered data and survive syncs.
|
||||
Clearing an override in the dashboard falls back to the provider-reported price.
|
||||
- **Curated aliases** — the schema has `models` / `model_targets` tables for
|
||||
friendly names with priority/weight failover. No code uses them yet.
|
||||
|
||||
## Frontend
|
||||
|
||||
@@ -212,17 +219,12 @@ backs the UI in development.
|
||||
|
||||
## Project status
|
||||
|
||||
Implemented: configuration, the database pool + migration runner, the server
|
||||
entrypoint (logging, health checks, graceful shutdown), the embedded dashboard
|
||||
handler, secret encryption (`internal/crypto`), the modular provider layer
|
||||
(`internal/providers` — OpenAI, OpenRouter, llama.cpp, Ollama, Anthropic, GitHub
|
||||
Copilot, with chat streaming + non-streaming and usage accounting), and the
|
||||
runtime model **catalog** (`internal/catalog` + `internal/store`: discovery
|
||||
sync, in-memory cache, `<provider>/<slug>` + `any/<slug>` routing) — all
|
||||
verified against Postgres.
|
||||
Implemented end to end and verified against Postgres: configuration, migrations,
|
||||
the OIDC login flow, virtual keys with model allowlists, the `/v1` proxy
|
||||
(chat completions with streaming, embeddings, model listing), provider dispatch
|
||||
for all six provider types, the runtime catalog with `any/<slug>` routing,
|
||||
rate limits, request and content logging with retention, billing, and the
|
||||
management dashboard.
|
||||
|
||||
Still to build: wiring the catalog syncer into the running server (build live
|
||||
providers from DB rows + decrypt credentials), and the HTTP API (`internal/api`
|
||||
— virtual-key auth, the `/v1` proxy that ties keys → catalog resolution →
|
||||
provider dispatch, admin CRUD for providers/models/overrides, OIDC exchange),
|
||||
billing writes into `request_logs`, and the dashboard UI itself.
|
||||
Not built yet: curated aliases (`models` / `model_targets`), group-scoped
|
||||
session keys, and per-key budgets.
|
||||
|
||||
@@ -30,8 +30,6 @@ GATEWAY_OIDC_ADMIN_GROUP=llm-admins
|
||||
# --- Security ---
|
||||
# 64 hex chars (32 bytes) for AES-256-GCM. Generate with: openssl rand -hex 32
|
||||
GATEWAY_SECURITY_ENCRYPTION_KEY=
|
||||
# Static bearer token for admin API. Generate with: openssl rand -hex 32
|
||||
GATEWAY_SECURITY_ADMIN_TOKEN=
|
||||
|
||||
# --- Catalog ---
|
||||
# How often to sync model lists from providers. Default: 30m
|
||||
|
||||
@@ -63,7 +63,6 @@ dev-stop: ## Stop and remove the dev Postgres container
|
||||
echo ""; \
|
||||
echo " cp .env.example .env"; \
|
||||
echo " echo GATEWAY_SECURITY_ENCRYPTION_KEY=\$$(openssl rand -hex 32) >> .env"; \
|
||||
echo " echo GATEWAY_SECURITY_ADMIN_TOKEN=\$$(openssl rand -hex 16) >> .env"; \
|
||||
echo ""; exit 1
|
||||
|
||||
run: ## Run the gateway (migrations applied automatically on startup)
|
||||
|
||||
Reference in New Issue
Block a user