docs: add dashboard screenshots and architecture notes

Describe the request path, the provider capability interfaces, the
catalog, auth, and logging for contributors. Reword the deploy section so
the author's registry and cluster read as an example.
This commit is contained in:
2026-10-04 17:32:44 +02:00
parent 353af96238
commit fd0451f02b
5 changed files with 187 additions and 6 deletions

View File

@@ -11,6 +11,21 @@ Clients talk to it exactly as they would to OpenAI (`POST /v1/chat/completions`,
SSE streaming included); the gateway handles credentials, routing, and accounting
behind the scenes.
![LMR dashboard overview](docs/screenshots/overview.png)
<details>
<summary>More screenshots</summary>
Models page, with discovered models, pricing and per-model overrides:
![Models page](docs/screenshots/models.png)
Request log, with per-request tokens, cost and latency:
![Logs page](docs/screenshots/logs.png)
</details>
## Features
- **OpenAI-compatible proxy** — drop-in `/v1/chat/completions` with streaming,
@@ -157,13 +172,19 @@ Docker/systemd, pass it with `env_file` / `EnvironmentFile` instead.
## Deploy
A push to `master` runs the Woodpecker pipeline in `.woodpecker.yml`. It builds
the root `Dockerfile` and pushes `registry.govekar.net/lovrog05/localmodelroutex`
with the tags `latest` and the commit SHA.
Any OCI image host works. Build the root `Dockerfile` and run the image with the
`GATEWAY_*` variables from [`.env.example`](.env.example). `compose.yml` runs it
next to your own Postgres, and `compose.bundled.yml` runs Postgres inside the
container for a single-host setup.
The gateway runs on k3s from [`deploy/k8s/lmr.yaml`](deploy/k8s/lmr.yaml). The
manifest expects `configmap/lmr-config`, `secret/lmr-secret` and the pull secret
`regcred2` to exist in the cluster. To roll out a new image:
The files below are the author's own deployment. They are tracked as a worked
example. Replace the registry and hostnames with yours.
- `.woodpecker.yml` runs the tests on every push and pull request, then builds
the image and pushes it to a private registry on `master`.
- [`deploy/k8s/lmr.yaml`](deploy/k8s/lmr.yaml) runs the gateway on k3s. The
manifest expects `configmap/lmr-config`, `secret/lmr-secret` and a registry
pull secret to exist in the cluster. To roll out a new image:
```bash
kubectl apply -f deploy/k8s/lmr.yaml # only when the manifest changed
@@ -252,3 +273,15 @@ management dashboard.
Not built yet: curated aliases (`models` / `model_targets`), group-scoped
session keys, and per-key budgets.
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for the development setup, the test and
lint commands, and the pull request checklist. Report security problems as
described in [SECURITY.md](SECURITY.md), not in public issues.
## License
LMR is released under the GNU Affero General Public License, version 3. See
[LICENSE](LICENSE). If you run a modified LMR as a service for others, the AGPL
requires you to offer them the source of your modified version.

148
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,148 @@
# Architecture
This page explains how a request moves through LMR and where to add things.
Read the [README](../README.md) first for the feature list and the repo layout.
## One request, end to end
A client sends `POST /v1/chat/completions` with `Authorization: Bearer sk-...`
and `"model": "openrouter/anthropic/claude-sonnet-4.5"`.
1. **Key auth** (`internal/api/middleware.go`). The middleware hashes the
bearer token with SHA-256 and loads the matching virtual key with its
allowlist rules. Revoked and expired keys stop here.
2. **Resolve** (`internal/catalog`). The model address is split into
`<provider>/<slug>` and looked up in the in-memory catalog cache. The
reserved `any/<slug>` prefix picks one provider by the routing strategy.
3. **Allowlist and limits** (`internal/api`). The key's rules are matched
against the resolved address. Then the per-model RPM and TPM limits from
the catalog entry are checked in `internal/ratelimit`.
4. **Dispatch** (`internal/providermgr`, `internal/providers`). The provider
instance is fetched by name and type-asserted for the capability the route
needs, for example `ChatCompleter`. The request payload is copied, the
model is swapped for the upstream slug, and admin parameter overrides are
merged in.
5. **Relay**. The provider sends the upstream call and returns either a body
or an SSE stream. The handler writes it to the client unchanged, apart from
adding a `created` timestamp when the upstream omits it.
6. **Account** (`internal/billing`, `internal/store`). Token usage is parsed
from the response or the final stream chunk. Cost is the upstream-reported
dollar amount when the provider gives one, else tokens times the catalog
prices. The log row and the prompt and completion text are written in a
goroutine so they never delay the response.
`/v1/embeddings` and `/v1/images/generations` follow the same six steps with
their own capability interfaces.
## Components
```
cmd/server/main.go config → db → migrate → providers → catalog → HTTP
internal/config viper, GATEWAY_* environment variables
internal/db pgxpool and the golang-migrate runner
internal/store hand-written SQL, one file per table group
internal/crypto AES-256-GCM seal and open for provider credentials
internal/providermgr builds live provider clients from DB rows
internal/providers provider abstraction and the per-type packages
internal/catalog model library: sync, cache, address resolution
internal/ratelimit per-model fixed-window RPM and TPM counters
internal/billing cost calculation
internal/api chi routes: /v1 proxy, /api/admin, /auth
internal/oidcauth OIDC discovery, ID token verification, group check
internal/web go:embed of the built dashboard, served as a SPA
dashboard/ Vite + React + TypeScript management UI
migrations/ numbered SQL pairs, applied at startup
```
## The provider abstraction
`providers.Provider` is small on purpose. It has a name, a type, and a list of
capabilities. Everything a route needs lives in a capability interface:
| Interface | Route | Method |
|---|---|---|
| `ChatCompleter` | `/v1/chat/completions` | `ChatCompletion`, `ChatCompletionStream` |
| `Embedder` | `/v1/embeddings` | `Embeddings` |
| `ImageGenerator` | `/v1/images/generations` | `ImageGeneration` |
| `ModelLister` | catalog sync | `ListModels` |
A handler type-asserts for the interface it needs and answers 501 when the
provider lacks it. New modalities add a new interface and a new route. Existing
providers do not change.
Each provider type is a package under `internal/providers/` that registers a
factory in `init()`. `internal/providers/all` blank-imports every package so
the server binary links them all. The registry maps the `type` column of the
providers table to the factory.
The OpenAI-compatible types (OpenAI, OpenRouter, llama.cpp, Ollama, Copilot)
embed `openaicompat.Transport`, which holds the HTTP client, base URL, auth
hook, static headers, and the flags for usage and cost reporting. A type only
adds what differs. OpenRouter adds attribution headers and the image
translation. Copilot adds a token exchange in its auth hook. Anthropic is the
one type that does not embed the transport. It translates between the OpenAI
and Messages schemas in both directions, including streamed tool calls.
### Why the image route translates
OpenRouter has no `/images/generations` endpoint. Its image models generate
through chat completions with `modalities: ["image", "text"]` and return the
image in `message.images`. The OpenRouter provider's `ImageGeneration` builds
that chat request, maps OpenAI `size` and `quality` to OpenRouter's
`image_config`, and renders the reply in the OpenAI images schema. Clients
written for the OpenAI Images API work unchanged.
## The catalog
Admins configure providers, not models. On a timer, and after every admin
change, the syncer asks each provider for its model list and upserts the rows
into `provider_models`. The syncer extracts what the provider reports: display
name, context length, pricing, and output modalities. Output modalities become
the row's capabilities, which is how the image route knows which models accept
an image request.
Admin overrides live in separate columns on the same row and survive syncs:
enabled, blocked, parameter overrides, prices, and rate limits. A model that
disappears from the upstream list is marked unavailable, not deleted.
The cache holds every servable row in memory, keyed by address. It is rebuilt
after each sync, so the request path never queries the database for routing.
## Authentication
Two systems that do not mix:
- **Proxy routes** under `/v1` take a virtual key. Keys are stored as hashes.
Each key is either allow-all or carries a list of address patterns, where
`openrouter/*` matches a whole provider.
- **Admin and user routes** under `/api` take an OIDC ID token. The login flow
is the authorization code flow. The server does the code exchange and hands
the ID token to the dashboard. Admin access needs the configured group in
the `groups` claim. The dashboard's test chat uses a synthetic allow-all key
bound to the signed-in user, so it needs no real API key.
Provider credentials are sealed with AES-256-GCM using the 32-byte key from
`GATEWAY_SECURITY_ENCRYPTION_KEY`. They are opened in memory when the provider
manager builds a client and are never returned by the admin API.
## Logging and retention
Every proxy request writes one row to `request_logs` with the key prefix,
model, provider, status, token counts, cost, latency, and time to first token
for streams. The prompt and completion text go to a second table. Inline
image data in prompts is replaced by a size marker before the write.
A daily job deletes rows older than the `log.retention_days` setting. Zero
keeps everything. Settings are rows in a key-value table and are edited from
the dashboard.
## Dashboard
The dashboard is a Vite project in `backend/dashboard`. `npm run build` writes
to `backend/internal/web/dist`, and the Go binary embeds that folder. In
development, Vite proxies `/api`, `/v1`, and `/auth` to the Go server on port
8080, so the UI hot-reloads against real data.
Pages map to admin routes one to one: Providers, Models, Routing, API Keys,
Logs, Analytics, Settings, and a Chat page for testing a model through the
same proxy path that clients use.

BIN
docs/screenshots/logs.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 380 KiB

BIN
docs/screenshots/models.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 336 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 259 KiB