Commit Graph
3 Commits
Author SHA1 Message Date
Jennifer Zhao 806d9732c9 Report exec runtime resets before accepting further requests (#52825)
## Why

Replacing a gRPC code-mode runtime loses stored values and running cells. Silently continuing after reconnection hides that loss, while prior external tool operations may already have completed.

## What changed

- Capture the runtime generation when `execute` and `wait` calls are created. If recovery replaces it, reject those calls with an explicit reset error, including concurrent calls and calls first polled after recovery.
- Explain the lost state and advise inspecting prior tool results before retrying. Allow a fresh `exec` call to continue on the replacement runtime.
- Keep `terminate` from reopening the runtime so cleanup cannot hide a reset from the next request.

## Testing

Extend gRPC recovery tests to cover reset errors after host restarts, missing sessions, and route loss, rejection of queued calls, and successful fresh execution with cleared stored state.

GitOrigin-RevId: 914cab3bb45e30a3f3b2afacb5631e42b18e21b0
2026-10-10 08:51:21 +00:00
cassirer-openai 5b3c1522f8 Preserve per-session routing for gRPC code mode (#52381)
## Why

Sessions sharing a transport can be admitted to different hosts. Each session needs its own routing token so subsequent RPCs reach the host holding its state.

## What changed

- Capture the optional `x-code-mode-route` metadata from successful `OpenSession` responses and attach it to subsequent session RPCs. Reject tokens longer than 256 bytes and keep new admission attempts free of existing session routes.
- Invalidate a routed session when an `Unavailable` response explicitly sets `x-code-mode-route-lost: true`, allowing a later operation to reopen it without replaying the failed execution. Ordinary `Unavailable` responses preserve the session.
- Allow up to five admission attempts for `Unavailable` and `ResourceExhausted`, with an immediate first retry followed by jittered exponential backoff.
- Order session failure with shutdown so late callback errors cannot cancel an in-flight `CloseSession` RPC.

## Testing

Add regression coverage for independent routing on a shared provider, route-loss recovery without execution replay, preservation of unaffected session state, and callback failures racing shutdown. Extend admission tests to verify the five-attempt limit and recovery tests to distinguish explicit route loss from ordinary unavailability.

GitOrigin-RevId: 5d60ef1afa56beb8c70ba7ab27865128225b0d3d
2026-10-09 02:50:59 +00:00
cassirer-openai aa6635ece5 Retry transient gRPC code-mode session admission failures (#51185)
## Why

Transient admission errors or connection resets can prevent a code-mode session from opening before any execution starts.

## What changed

- Retry `OpenSession` and the initial lease message on `Unavailable` or `ResourceExhausted`, with at most three attempts and 100 ms and 200 ms delays within a single startup deadline.
- Classify HTTP connection and request failures as `Unavailable` so admission can recover from connection resets, while preserving error sources.
- Keep retries limited to initial admission so established executions are never replayed.

## Testing

Add gRPC integration tests covering transient status recovery, the retry limit, immediate failure for non-retryable statuses, and successful session execution after a connection reset.

GitOrigin-RevId: 3b654bcd377a5addf7c3ed58bb1c72a449b2f7f7
2026-10-05 22:18:55 +00:00