Commit Graph
3 Commits
Author SHA1 Message Date
Ahmed Ibrahim f31bd3adff Persist provider and auth identity with model catalog caches (#43897)
## What changed

Add a SHA-256 identity derived from provider routing, headers, and authentication scope. Return it with each model catalog response and persist it in `ModelsCacheEntry`. ChatGPT credentials with stable account and user metadata retain the same identity across token refreshes; opaque API credentials contribute to the digest.

## Testing

Add identity tests covering account, user, email, plan, auth mode, provider routing, headers, and API credential changes, plus stability across ChatGPT token refreshes. Update cache tests to include the persisted identity.

GitOrigin-RevId: 3f51c6cabcb01bc03505150a768a61dfe5d6569f
2026-09-08 19:48:37 +00:00
Michael Bolin 5892c7b69d model-provider: route model discovery through HTTP client factory (#31361) 2026-07-09 02:32:10 +00:00
jif 5935a90619 app-server: keep the model cache warm (#28699)
## Why

The app server is long-lived, but its shared model cache otherwise
refreshes only when a caller needs it. Once the five-minute cache
expires, starting a thread or calling `model/list` can wait for
`/models` on the request path.

Refresh the cache in the background before it expires so foreground
callers normally use fresh local state.

## What changed

- Start an app-server worker that refreshes models immediately and then
every three minutes using the existing models-manager API.
- Hold only a weak reference to the models manager between refreshes, so
the worker does not extend its lifetime.
- Stop scheduling refreshes when the app-server lifecycle handle is shut
down or dropped. A refresh already in progress is allowed to finish.
- Adjust affected app-server test fixtures to distinguish the background
`/models` probe from the connection they are testing.

The existing models-manager cache, refresh strategies, auth handling,
ETag behavior, and concurrency semantics are unchanged.

## Testing

-
`models_refresh_worker::tests::refreshes_immediately_periodically_and_stops_when_dropped`
-
`suite::v2::remote_control::listen_off_honors_persisted_remote_control_enable`
-
`suite::v2::attestation::attestation_generate_round_trip_adds_header_to_responses_websocket_handshake`
2026-06-17 16:18:39 +02:00