LovroG05
00800d3a3f
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
The sync took meta.n_ctx_train from the model listing, which is the training context, not the -c the server runs with. The llama.cpp provider now asks the server for the live value: /props on a single server, the router's /models status and /props?model=, and /upstream/<model>/props for models llama-swap has loaded. Stopped models are never probed, because that would start them. Admins can also pin the context window per model from the Models page. The override wins over the discovered value and survives syncs.
5 lines
301 B
SQL
5 lines
301 B
SQL
-- Admin override for the context window. Providers report the training
|
|
-- context (llama.cpp meta.n_ctx_train) or a catalog value, which can differ
|
|
-- from what the running instance accepts. NULL means use the discovered value.
|
|
ALTER TABLE provider_models ADD COLUMN context_length_override integer;
|