Commit Graph
3 Commits
Author SHA1 Message Date
spasumar-oai 4ef1d4b89b Increase the app-server model refresh interval (#40498)
## What changed

Change the background model refresh interval from 3 minutes to 4 minutes 30
seconds.

GitOrigin-RevId: dcefe471c91dc94a04ddc74595c3ce00d9061e55
2026-08-24 23:29:10 +00:00
Michael Bolin 5892c7b69d model-provider: route model discovery through HTTP client factory (#31361) 2026-07-09 02:32:10 +00:00
jif 5935a90619 app-server: keep the model cache warm (#28699)
## Why

The app server is long-lived, but its shared model cache otherwise
refreshes only when a caller needs it. Once the five-minute cache
expires, starting a thread or calling `model/list` can wait for
`/models` on the request path.

Refresh the cache in the background before it expires so foreground
callers normally use fresh local state.

## What changed

- Start an app-server worker that refreshes models immediately and then
every three minutes using the existing models-manager API.
- Hold only a weak reference to the models manager between refreshes, so
the worker does not extend its lifetime.
- Stop scheduling refreshes when the app-server lifecycle handle is shut
down or dropped. A refresh already in progress is allowed to finish.
- Adjust affected app-server test fixtures to distinguish the background
`/models` probe from the connection they are testing.

The existing models-manager cache, refresh strategies, auth handling,
ETag behavior, and concurrency semantics are unchanged.

## Testing

-
`models_refresh_worker::tests::refreshes_immediately_periodically_and_stops_when_dropped`
-
`suite::v2::remote_control::listen_off_honors_persisted_remote_control_enable`
-
`suite::v2::attestation::attestation_generate_round_trip_adds_header_to_responses_websocket_handshake`
2026-06-17 16:18:39 +02:00