Commit Graph
4 Commits
Author SHA1 Message Date
rhan-oai c4f0e09912 Allow asynchronous model request budget checks (#60080)
## What changed

Make `ModelRequestContributor::budget_exhausted` return an `ExtensionFuture` and await budget checks before HTTP and WebSocket generation. WebSocket warmup continues to skip budget checks.

If the account changes during the post-connect budget check, rebuild the request setup and reconnect before sending the WebSocket request.

## Testing

Update tests to cover a pending budget check that resolves to exhaustion, warmup connection reuse, and an account switch while the post-connect budget check is awaiting completion.

GitOrigin-RevId: 462c74b86a46a4b2276f901c69187fc7cbc3ff81
2026-11-07 07:45:36 +00:00
rhan-oai 1031a7b9aa Check extension budgets before model generation requests (#59794)
## What changed

Add a default `ModelRequestContributor::budget_exhausted` hook and fail generation with `SessionBudgetExceeded` when any contributor reports an exhausted budget for the thread.

Check budgets before HTTP requests and before WebSocket connection setup, then check again after the connection is established. WebSocket warmup remains available when the budget is exhausted.

## Testing

Add coverage for budget changes during a tool call, verifying that exhaustion prevents the next model request while an available budget allows completion. Extend WebSocket coverage to verify that exhausted generation opens no connection and warmup still permits connection reuse once the budget becomes available.

GitOrigin-RevId: 97ce02dc7f9bd54830e4b73c4f0334a30367eebf
2026-11-06 02:31:54 +00:00
jif fc916a22b5 Expose Guardian assessments in response streams (#57757)
## What changed

Decode `response.metadata` Guardian assessments into typed `GuardianAssessment` events over SSE and WebSocket, carrying `response_id`, `item_id`, and `requires_review`. Ignore assessments with missing, empty, or incorrectly typed required fields while preserving independent delivery of other metadata.

Expose the assessment type to request-scoped extension interceptors. Keep assessments out of conversation history; they do not authorize tool execution.

## Testing

Add SSE coverage for assessment ordering, malformed metadata, and mixed metadata delivery. Extend model request tests to cover SSE and WebSocket interceptor delivery and exclusion from subsequent model input.

GitOrigin-RevId: 282b8817ee97346d76b40bcb412eef023df60ff1
2026-10-28 15:04:29 +00:00
Ruslan Nigmatullin 8de2d336b8 Add extension hooks for model requests and response streams (#47679)
## What changed

Add `ModelRequestContributor` and `ModelResponseInterceptor` to the extension API. Contributors can add filtered `client_metadata` without overwriting existing or reserved fields and create interceptors for individual requests.

Wire contributors into HTTP and WebSocket requests, distinguishing generation from warmup. Apply response interceptors in registration order before tracing and response bookkeeping so transformed output is recorded consistently. Interceptors can inspect, buffer, or transform events, with stream ownership carrying request state and cancellation.

## Testing

Add tests for metadata protection, incremental event transformation, interceptor ordering, upstream cancellation, and consistency between delivered output, traces, and WebSocket bookkeeping. Integration tests cover output gating and interruption over HTTP and WebSocket, including a contributor that bypasses gating for warmup requests.

GitOrigin-RevId: d4bc1745d2cacf8636728dbc0a58ab9f250e9392
2026-09-23 22:10:04 +00:00