frontend

README.md

@@ -0,0 +1,181 @@
+# LMR — Local Model Route
+
+An OpenAI-compatible **LLM gateway**. It puts a single, authenticated, observable
+endpoint in front of any number of model backends — local servers and commercial
+APIs alike — and adds virtual-key auth, provider routing, request logging,
+response caching, usage-based billing, and a management UI.
+
+Meant to run locally in a homelab environment as a way to securely-ish expose your local models and back them up with the big boys.
+
+Clients talk to it exactly as they would to OpenAI (`POST /v1/chat/completions`,
+SSE streaming included); the gateway handles credentials, routing, and accounting
+behind the scenes.
+
+## Features
+
+- **OpenAI-compatible proxy** — drop-in `/v1/chat/completions` with streaming.
+- **Virtual keys** — issue, scope, revoke, and expire `sk-…` keys per user/group,
+  each with its own model allowlist.
+- **Multi-provider routing** — local (llama.cpp / Ollama) and API providers
+  (OpenAI, Anthropic, GitHub Copilot) behind one interface.
+- **Provider abstraction** — per-provider quirks handled internally (Anthropic
+  `cache_control` injection, OpenAI `cached_tokens`, Copilot token exchange).
+- **Response cache** — exact-match (sha256) and semantic (pgvector ANN) caching.
+- **Usage & billing** — per-request cost from a pricing table, written
+  asynchronously so it never blocks the request path.
+- **OIDC admin auth** — Authentik / any OIDC provider, group-gated admin access.
+- **Encrypted secrets** — provider credentials stored AES-256-GCM encrypted at rest.
+- **Single binary** — the React management UI is embedded via `go:embed`.
+
+## Stack
+
+- **Go** (chi, pgx/v5, viper, go-oidc, golang-migrate) — see [`backend/go.mod`](backend/go.mod)
+- **PostgreSQL 16** + **pgvector** — keys, request logs, cache, pricing
+- **React + TypeScript** (Vite) — management UI, built into the server binary
+
+## Repo layout
+
+```
+.
+├── backend/                 # Go module: git.govekar.net/LovroG05/localmodelroutex
+│   ├── cmd/server/          # entrypoint: config → db → migrate → HTTP server
+│   ├── internal/
+│   │   ├── api/             # HTTP handlers + middleware (proxy, admin, auth)
+│   │   ├── providers/       # provider abstraction (llamacpp, ollama, openai, …)
+│   │   ├── config/          # viper config, GATEWAY_ env prefix
+│   │   ├── db/              # pgxpool setup + migration runner
+│   │   ├── billing/         # cost calculation + async usage logging
+│   │   ├── cache/           # exact (sha256) + semantic (pgvector) response cache
+│   │   └── web/             # go:embed of the built UI, served as SPA
+│   ├── migrations/          # golang-migrate SQL, applied automatically at startup
+│   └── dashboard/           # Vite + React TS project (builds into internal/web/dist)
+└── README.md
+```
+
+## Authentication
+
+Two auth systems that never mix:
+
+| Surface | Header | Verification |
+|---|---|---|
+| Proxy routes (`/v1/*`) | `Bearer sk-xxx` | sha256 of the key → lookup in `virtual_keys`; check revoked / expired / model allowlist |
+| Admin routes (`/api/admin/*`) | `Bearer <OIDC JWT>` | `go-oidc` `Verify()` → `groups` claim → must contain the admin group |
+
+**Session keys:** `POST /auth/exchange` trades an Authentik OIDC JWT for an
+`sk-sess-xxx` virtual key scoped to that group's model allowlist.
+
+Provider credentials are encrypted with AES-256-GCM using the 32-byte key in
+`GATEWAY_SECURITY_ENCRYPTION_KEY` and decrypted only in memory at request time.
+
+## Configuration
+
+All settings come from environment variables prefixed `GATEWAY_`. Dot-notation
+config keys map to underscores in env (e.g. `oidc.client_id` →
+`GATEWAY_OIDC_CLIENT_ID`). In development, a `.env` file in the working directory
+is loaded automatically; in production, set real environment variables.
+
+| Variable | Default | Required | Notes |
+|---|---|---|---|
+| `GATEWAY_SERVER_PORT` | `8080` | no | HTTP listen port |
+| `GATEWAY_SERVER_ENV` | `development` | no | `development` \| `production` (controls log format/level) |
+| `GATEWAY_DB_URL` | — | **yes** | `postgres://user:pass@host:5432/db` |
+| `GATEWAY_DB_MAX_CONNS` | `20` | no | pool max connections |
+| `GATEWAY_DB_MIN_CONNS` | `2` | no | pool min connections |
+| `GATEWAY_DB_MAX_CONN_LIFETIME` | `30` | no | minutes |
+| `GATEWAY_OIDC_ISSUER` | — | **yes** | OIDC issuer URL |
+| `GATEWAY_OIDC_CLIENT_ID` | — | **yes** | OIDC client ID |
+| `GATEWAY_OIDC_ADMIN_GROUP` | — | **yes** | group required for admin access |
+| `GATEWAY_SECURITY_ENCRYPTION_KEY` | — | **yes** | 64 hex chars (32 bytes) for AES-256-GCM |
+
+Config is validated on startup; missing required values or a wrong-length
+encryption key cause the server to exit immediately.
+
+Generate an encryption key:
+
+```bash
+openssl rand -hex 32
+```
+
+## Running locally
+
+Requires Go (see the `go` directive in [`backend/go.mod`](backend/go.mod)) and a
+PostgreSQL 16 instance with the `pgvector` extension available.
+
+```bash
+# 1. Start Postgres (pgvector image includes the extension)
+docker run -d --name lmr-postgres \
+  -e POSTGRES_USER=gateway -e POSTGRES_PASSWORD=gateway -e POSTGRES_DB=gateway \
+  -p 5432:5432 pgvector/pgvector:pg16
+
+# 2. Configure
+cd backend
+cp .env .env.local   # or edit .env directly; fill in OIDC + encryption key
+
+# 3. Run — migrations are applied automatically on startup
+go run ./cmd/server
+```
+
+The server listens on `GATEWAY_SERVER_PORT` (default `:8080`).
+
+### Health checks
+
+| Endpoint | Meaning |
+|---|---|
+| `GET /healthz` | Liveness — process is up |
+| `GET /readyz` | Readiness — database reachable (pings the pool) |
+
+## Database & migrations
+
+SQL migrations live in [`backend/migrations/`](backend/migrations/) using
+golang-migrate's `{version}_{name}.{up,down}.sql` naming. They are **embedded into
+the binary** (`go:embed`) and **applied automatically on startup**, so a deployed
+binary is self-contained — no migration files needed on the host.
+
+To add a migration, create a new numbered pair, e.g.:
+
+```
+migrations/000002_virtual_keys.up.sql
+migrations/000002_virtual_keys.down.sql
+```
+
+## Providers
+
+| Name | Kind | Notes |
+|---|---|---|
+| `llamacpp` | local | OpenAI-compatible; routes through llama-swap |
+| `ollama` | local | OpenAI-compatible |
+| `openai` | api key | reads `cached_tokens` from the usage response |
+| `anthropic` | api key | injects `cache_control` on the system prompt + penultimate message |
+| `github-copilot` | subscription | exchanges a GitHub PAT for a ~30-min Copilot token, mutex-guarded refresh |
+
+## Frontend
+
+The management UI is a Vite + React + TypeScript project in
+[`backend/dashboard/`](backend/dashboard/). Its production build is written into
+`backend/internal/web/dist/` (set via `build.outDir` in
+[`vite.config.ts`](backend/dashboard/vite.config.ts)) and embedded into the server
+binary, which serves it as a single-page app with history-API fallback (API
+routes take precedence).
+
+```bash
+cd backend/dashboard
+npm install
+npm run build      # type-checks (tsc), then outputs to ../internal/web/dist
+```
+
+During development, run the Vite dev server alongside the gateway:
+
+```bash
+npm run dev        # serves the UI on http://localhost:5173 with hot reload
+```
+
+The dev server proxies `/v1`, `/api`, and `/auth` to `http://localhost:8080`
+(configured under `server.proxy` in `vite.config.ts`), so the running gateway
+backs the UI in development.
+
+## Project status
+
+Early scaffolding. Implemented so far: configuration, the database connection
+pool + migration runner, and the server entrypoint (logging, health checks,
+graceful shutdown). The `api`, `providers`, `billing`, `cache`, and `web`
+packages and the `dashboard/` UI are still to be built out.

backend/.gitignore

@@ -0,0 +1,5 @@
+# Compiled server binary
+/bin/
+
+# Embedded UI build output (generated by `cd dashboard && npm run build`)
+/internal/web/dist/

backend/dashboard/.gitignore

@@ -0,0 +1,13 @@
+# Logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+
+.DS_Store

backend/dashboard/README.md

@@ -0,0 +1,46 @@
+# Getting Started with Create React App
+
+This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
+
+## Available Scripts
+
+In the project directory, you can run:
+
+### `npm start`
+
+Runs the app in the development mode.\
+Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
+
+The page will reload if you make edits.\
+You will also see any lint errors in the console.
+
+### `npm test`
+
+Launches the test runner in the interactive watch mode.\
+See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
+
+### `npm run build`
+
+Builds the app for production to the `build` folder.\
+It correctly bundles React in production mode and optimizes the build for the best performance.
+
+The build is minified and the filenames include the hashes.\
+Your app is ready to be deployed!
+
+See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
+
+### `npm run eject`
+
+**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
+
+If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
+
+Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
+
+You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
+
+## Learn More
+
+You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
+
+To learn React, check out the [React documentation](https://reactjs.org/).

backend/dashboard/index.html

@@ -0,0 +1,13 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <link rel="icon" href="/favicon.ico" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>LMR Dashboard</title>
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.tsx"></script>
+  </body>
+</html>

backend/dashboard/package.json

@@ -0,0 +1,22 @@
+{
+  "name": "dashboard",
+  "version": "0.1.0",
+  "private": true,
+  "type": "module",
+  "scripts": {
+    "dev": "vite",
+    "build": "tsc && vite build",
+    "preview": "vite preview"
+  },
+  "dependencies": {
+    "react": "^19.2.0",
+    "react-dom": "^19.2.0"
+  },
+  "devDependencies": {
+    "@types/react": "^19.2.0",
+    "@types/react-dom": "^19.2.0",
+    "@vitejs/plugin-react": "^4.3.4",
+    "typescript": "~5.7.2",
+    "vite": "^7.0.0"
+  }
+}

backend/dashboard/public/manifest.json

@@ -0,0 +1,25 @@
+{
+  "short_name": "React App",
+  "name": "Create React App Sample",
+  "icons": [
+    {
+      "src": "favicon.ico",
+      "sizes": "64x64 32x32 24x24 16x16",
+      "type": "image/x-icon"
+    },
+    {
+      "src": "logo192.png",
+      "type": "image/png",
+      "sizes": "192x192"
+    },
+    {
+      "src": "logo512.png",
+      "type": "image/png",
+      "sizes": "512x512"
+    }
+  ],
+  "start_url": ".",
+  "display": "standalone",
+  "theme_color": "#000000",
+  "background_color": "#ffffff"
+}

backend/dashboard/public/robots.txt

@@ -0,0 +1,3 @@
+# https://www.robotstxt.org/robotstxt.html
+User-agent: *
+Disallow:

backend/dashboard/src/App.css

@@ -0,0 +1,38 @@
+.App {
+  text-align: center;
+}
+
+.App-logo {
+  height: 40vmin;
+  pointer-events: none;
+}
+
+@media (prefers-reduced-motion: no-preference) {
+  .App-logo {
+    animation: App-logo-spin infinite 20s linear;
+  }
+}
+
+.App-header {
+  background-color: #282c34;
+  min-height: 100vh;
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  justify-content: center;
+  font-size: calc(10px + 2vmin);
+  color: white;
+}
+
+.App-link {
+  color: #61dafb;
+}
+
+@keyframes App-logo-spin {
+  from {
+    transform: rotate(0deg);
+  }
+  to {
+    transform: rotate(360deg);
+  }
+}

backend/dashboard/src/App.tsx

@@ -0,0 +1,25 @@
+import logo from './logo.svg';
+import './App.css';
+
+function App() {
+  return (
+    <div className="App">
+      <header className="App-header">
+        <img src={logo} className="App-logo" alt="logo" />
+        <p>
+          Edit <code>src/App.tsx</code> and save to reload.
+        </p>
+        <a
+          className="App-link"
+          href="https://reactjs.org"
+          target="_blank"
+          rel="noopener noreferrer"
+        >
+          Learn React
+        </a>
+      </header>
+    </div>
+  );
+}
+
+export default App;

backend/dashboard/src/index.css

@@ -0,0 +1,13 @@
+body {
+  margin: 0;
+  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
+    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
+    sans-serif;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+}
+
+code {
+  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
+    monospace;
+}

backend/dashboard/src/logo.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 841.9 595.3"><g fill="#61DAFB"><path d="M666.3 296.5c0-32.5-40.7-63.3-103.1-82.4 14.4-63.6 8-114.2-20.2-130.4-6.5-3.8-14.1-5.6-22.4-5.6v22.3c4.6 0 8.3.9 11.4 2.6 13.6 7.8 19.5 37.5 14.9 75.7-1.1 9.4-2.9 19.3-5.1 29.4-19.6-4.8-41-8.5-63.5-10.9-13.5-18.5-27.5-35.3-41.6-50 32.6-30.3 63.2-46.9 84-46.9V78c-27.5 0-63.5 19.6-99.9 53.6-36.4-33.8-72.4-53.2-99.9-53.2v22.3c20.7 0 51.4 16.5 84 46.6-14 14.7-28 31.4-41.3 49.9-22.6 2.4-44 6.1-63.6 11-2.3-10-4-19.7-5.2-29-4.7-38.2 1.1-67.9 14.6-75.8 3-1.8 6.9-2.6 11.5-2.6V78.5c-8.4 0-16 1.8-22.6 5.6-28.1 16.2-34.4 66.7-19.9 130.1-62.2 19.2-102.7 49.9-102.7 82.3 0 32.5 40.7 63.3 103.1 82.4-14.4 63.6-8 114.2 20.2 130.4 6.5 3.8 14.1 5.6 22.5 5.6 27.5 0 63.5-19.6 99.9-53.6 36.4 33.8 72.4 53.2 99.9 53.2 8.4 0 16-1.8 22.6-5.6 28.1-16.2 34.4-66.7 19.9-130.1 62-19.1 102.5-49.9 102.5-82.3zm-130.2-66.7c-3.7 12.9-8.3 26.2-13.5 39.5-4.1-8-8.4-16-13.1-24-4.6-8-9.5-15.8-14.4-23.4 14.2 2.1 27.9 4.7 41 7.9zm-45.8 106.5c-7.8 13.5-15.8 26.3-24.1 38.2-14.9 1.3-30 2-45.2 2-15.1 0-30.2-.7-45-1.9-8.3-11.9-16.4-24.6-24.2-38-7.6-13.1-14.5-26.4-20.8-39.8 6.2-13.4 13.2-26.8 20.7-39.9 7.8-13.5 15.8-26.3 24.1-38.2 14.9-1.3 30-2 45.2-2 15.1 0 30.2.7 45 1.9 8.3 11.9 16.4 24.6 24.2 38 7.6 13.1 14.5 26.4 20.8 39.8-6.3 13.4-13.2 26.8-20.7 39.9zm32.3-13c5.4 13.4 10 26.8 13.8 39.8-13.1 3.2-26.9 5.9-41.2 8 4.9-7.7 9.8-15.6 14.4-23.7 4.6-8 8.9-16.1 13-24.1zM421.2 430c-9.3-9.6-18.6-20.3-27.8-32 9 .4 18.2.7 27.5.7 9.4 0 18.7-.2 27.8-.7-9 11.7-18.3 22.4-27.5 32zm-74.4-58.9c-14.2-2.1-27.9-4.7-41-7.9 3.7-12.9 8.3-26.2 13.5-39.5 4.1 8 8.4 16 13.1 24 4.7 8 9.5 15.8 14.4 23.4zM420.7 163c9.3 9.6 18.6 20.3 27.8 32-9-.4-18.2-.7-27.5-.7-9.4 0-18.7.2-27.8.7 9-11.7 18.3-22.4 27.5-32zm-74 58.9c-4.9 7.7-9.8 15.6-14.4 23.7-4.6 8-8.9 16-13 24-5.4-13.4-10-26.8-13.8-39.8 13.1-3.1 26.9-5.8 41.2-7.9zm-90.5 125.2c-35.4-15.1-58.3-34.9-58.3-50.6 0-15.7 22.9-35.6 58.3-50.6 8.6-3.7 18-7 27.7-10.1 5.7 19.6 13.2 40 22.5 60.9-9.2 20.8-16.6 41.1-22.2 60.6-9.9-3.1-19.3-6.5-28-10.2zM310 490c-13.6-7.8-19.5-37.5-14.9-75.7 1.1-9.4 2.9-19.3 5.1-29.4 19.6 4.8 41 8.5 63.5 10.9 13.5 18.5 27.5 35.3 41.6 50-32.6 30.3-63.2 46.9-84 46.9-4.5-.1-8.3-1-11.3-2.7zm237.2-76.2c4.7 38.2-1.1 67.9-14.6 75.8-3 1.8-6.9 2.6-11.5 2.6-20.7 0-51.4-16.5-84-46.6 14-14.7 28-31.4 41.3-49.9 22.6-2.4 44-6.1 63.6-11 2.3 10.1 4.1 19.8 5.2 29.1zm38.5-66.7c-8.6 3.7-18 7-27.7 10.1-5.7-19.6-13.2-40-22.5-60.9 9.2-20.8 16.6-41.1 22.2-60.6 9.9 3.1 19.3 6.5 28.1 10.2 35.4 15.1 58.3 34.9 58.3 50.6-.1 15.7-23 35.6-58.4 50.6zM320.8 78.4z"/><circle cx="420.9" cy="296.5" r="45.7"/><path d="M520.5 78.1z"/></g></svg>

backend/dashboard/src/main.tsx

@@ -0,0 +1,10 @@
+import React from 'react';
+import ReactDOM from 'react-dom/client';
+import App from './App';
+import './index.css';
+
+ReactDOM.createRoot(document.getElementById('root')!).render(
+  <React.StrictMode>
+    <App />
+  </React.StrictMode>,
+);

backend/dashboard/src/vite-env.d.ts

@@ -0,0 +1 @@
+/// <reference types="vite/client" />

backend/dashboard/tsconfig.json

@@ -0,0 +1,23 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "lib": ["ES2022", "DOM", "DOM.Iterable"],
+    "module": "ESNext",
+    "moduleResolution": "bundler",
+    "jsx": "react-jsx",
+    "useDefineForClassFields": true,
+    "resolveJsonModule": true,
+    "isolatedModules": true,
+    "moduleDetection": "force",
+    "esModuleInterop": true,
+    "skipLibCheck": true,
+    "noEmit": true,
+
+    /* Linting */
+    "strict": true,
+    "noUnusedLocals": true,
+    "noUnusedParameters": true,
+    "noFallthroughCasesInSwitch": true
+  },
+  "include": ["src", "vite.config.ts"]
+}

backend/dashboard/vite.config.ts

@@ -0,0 +1,21 @@
+import { defineConfig } from 'vite';
+import react from '@vitejs/plugin-react';
+
+// https://vite.dev/config/
+export default defineConfig({
+  plugins: [react()],
+  build: {
+    // Build straight into the Go embed package so `go build` bundles the UI.
+    // emptyOutDir is required because the target is outside the project root.
+    outDir: '../internal/web/dist',
+    emptyOutDir: true,
+  },
+  server: {
+    // Forward gateway routes to the Go server during `npm run dev`.
+    proxy: {
+      '/v1': 'http://localhost:8080',
+      '/api': 'http://localhost:8080',
+      '/auth': 'http://localhost:8080',
+    },
+  },
+});