docs: update README

This commit is contained in:
2026-06-28 23:15:33 +02:00
parent 6227a49992
commit 13e8849047

View File

@@ -40,7 +40,14 @@ behind the scenes.
│ ├── cmd/server/ # entrypoint: config → db → migrate → HTTP server
│ ├── internal/
│ │ ├── api/ # HTTP handlers + middleware (proxy, admin, auth)
│ │ ├── providers/ # provider abstraction (llamacpp, ollama, openai, …)
│ │ ├── providers/ # modular provider layer (registry + capability ifaces)
│ │ │ ├── openaicompat/ # shared OpenAI-compatible transport
│ │ │ ├── openai|openrouter|llamacpp|ollama|copilot/ # OpenAI-compatible
│ │ │ ├── anthropic/ # Messages API + OpenAI<->Anthropic translation
│ │ │ └── all/ # blank-import aggregator (self-registration)
│ │ ├── catalog/ # runtime model library: cache, sync, any/<slug> routing
│ │ ├── store/ # hand-written pgx data-access layer
│ │ ├── crypto/ # AES-256-GCM seal/open for provider secrets
│ │ ├── config/ # viper config, GATEWAY_ env prefix
│ │ ├── db/ # pgxpool setup + migration runner
│ │ ├── billing/ # cost calculation + async usage logging
@@ -157,10 +164,27 @@ migrations/000002_virtual_keys.down.sql
|---|---|---|
| `llamacpp` | local | OpenAI-compatible; routes through llama-swap |
| `ollama` | local | OpenAI-compatible |
| `openrouter` | api key | OpenAI-compatible; HTTP-Referer/X-Title attribution; reports pricing |
| `openai` | api key | reads `cached_tokens` from the usage response |
| `anthropic` | api key | injects `cache_control` on the system prompt + penultimate message |
| `anthropic` | api key | Messages API ↔ OpenAI translation; injects `cache_control` on the system prompt + penultimate message |
| `github-copilot` | subscription | exchanges a GitHub PAT for a ~30-min Copilot token, mutex-guarded refresh |
## Model addressing
Models are addressed as **`<provider>/<slug>`**, where `<provider>` is the
configured provider instance name and `<slug>` is the upstream model id (which
may itself contain slashes, e.g. `openrouter/anthropic/claude-3.7-sonnet`). The
**catalog** is built at runtime by syncing each provider's model listing on a
timer and cached in memory.
- **`any/<slug>`** — a reserved prefix that fans out across every provider
serving that model and picks one by the configured **routing strategy**
(`round_robin` | `lowest_cost` | `priority`, in `settings`).
- **Curated aliases** — admins can define a friendly name mapping to one or more
catalog entries with priority/weight for local-first/cloud-backup failover.
- **Admin overrides** — per-model `enabled`/`blocked`, parameter overrides,
pricing, and rate limits live alongside the discovered data and survive syncs.
## Frontend
The management UI is a Vite + React + TypeScript project in
@@ -188,7 +212,17 @@ backs the UI in development.
## Project status
Early scaffolding. Implemented so far: configuration, the database connection
pool + migration runner, and the server entrypoint (logging, health checks,
graceful shutdown). The `api`, `providers`, `billing`, and `web`
packages and the `dashboard/` UI are still to be built out.
Implemented: configuration, the database pool + migration runner, the server
entrypoint (logging, health checks, graceful shutdown), the embedded dashboard
handler, secret encryption (`internal/crypto`), the modular provider layer
(`internal/providers` — OpenAI, OpenRouter, llama.cpp, Ollama, Anthropic, GitHub
Copilot, with chat streaming + non-streaming and usage accounting), and the
runtime model **catalog** (`internal/catalog` + `internal/store`: discovery
sync, in-memory cache, `<provider>/<slug>` + `any/<slug>` routing) — all
verified against Postgres.
Still to build: wiring the catalog syncer into the running server (build live
providers from DB rows + decrypt credentials), and the HTTP API (`internal/api`
— virtual-key auth, the `/v1` proxy that ties keys → catalog resolution →
provider dispatch, admin CRUD for providers/models/overrides, OIDC exchange),
billing writes into `request_logs`, and the dashboard UI itself.