Commit Graph
12 Commits
Author SHA1 Message Date
victor-openai b71af39fe6 Preserve MCP App UI metadata in tool-call events and history (#45805)
## Why

Clients need widget presentation details to render MCP Apps without waiting for the full MCP catalog, including when replaying saved history.

## What changed

- Add `mcpToolCall.mcpAppUi` with the invoked descriptor's `resourceUri` and `preferredModelDisplayMode`, and carry it through tool-call events and saved history.
- Support `fullscreen` and default to `inline` when the display preference is missing or unsupported.
- Keep existing resource URI fields for compatibility. Leave `mcpAppUi` null for older history and tools that declare widgets only in result metadata, where clients still use catalog discovery.
- Update protocol schemas, TypeScript and Python types, and app-server documentation.

## Testing

Add parameterized integration coverage for fullscreen, missing and unsupported preferences, legacy URI metadata, and result-only widgets, verifying consistent tool events and preservation across session resume.

GitOrigin-RevId: 2a9bed804dcea8c2b4903b8406fa6710681f7f5e
2026-09-15 22:52:56 +00:00
Krish Chainani 7b8b17b97a Support image references by file ID in inputs and tool outputs (#45794)
## What changed

- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.

## Testing

Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.

GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
2026-09-15 21:19:08 +00:00
Krish Chainani 5a66d460d3 Refactor image content to use a shared ImageReference type (#45543)
## What changed

Represent images in `ContentItem` and `FunctionCallOutputContentItem` with `ImageReference::Inline`, flattened to preserve the existing `image_url` wire format. Update image producers and consumers and regenerate app-server schemas and SDK artifacts.

Preserve the Python SDK's `InputImageContentItem` and `InputImageFunctionCallOutputContentItem` class names during generation.

## Testing

Add a regression test for stable Python image class names and adapt existing image tests to the shared representation.

GitOrigin-RevId: c38a780ac3314c2ac2deb3afc1b93b94b6f93fec
2026-09-14 23:25:09 +00:00
Owen Lin 3052bbcf8c Remove the deprecated thread/rollback API (#44915)
## What changed

Remove `thread/rollback`, its request and response types, generated bindings, and the core `Op::ThreadRollback` operation. Requests now follow the generic unknown-method rejection path. Document `thread/revert` as the alternative for paginated threads.

Keep historical `ThreadRolledBack` markers and legacy error deserialization so existing rollouts remain compatible with replay and migration.

## Testing

Adapt retained-context and Guardian history tests to append legacy rollback markers and resume threads, preserving coverage of surviving instructions, answers, and review history.

GitOrigin-RevId: b3da1becdf86b1869275aacb0ffc2817cee5af2e
2026-09-11 21:32:04 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
Anton Panasenko 2b554fd3f9 Expose loaded thread environments in app-server responses (#42386)
## What changed

- Add the experimental `Thread.environments` field with each selected environment's ID, working directory, and runtime workspace roots.
- Populate the field from live thread state in start, resume, read, list, and unarchive responses. Use `null` for unloaded threads and an empty list when a loaded thread has no environment selection.
- Document that the field reports selection rather than connection status and does not affect executor selection or resume behavior.

## Testing

- Cover selection updates, unloaded and resumed threads, loaded pathless unarchives, and clients that subscribe after environment connection notifications.

GitOrigin-RevId: a6daa06aff7089d7c5a1e4c7d03109b41f12fa3c
2026-09-02 23:41:14 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
Brent Traut 986ff1cc7c Expose model settings in app-server thread metadata (#42151)
## What changed

- Add nullable `model` and `reasoningEffort` fields to the shared app-server `Thread` object and generated schemas.
- Report current settings for loaded threads and the latest persisted settings for unloaded threads across thread read, list, start, resume, rollback, metadata update, and notification paths.
- Keep unavailable legacy or filesystem-only settings nullable, and allow metadata reads without loading a thread or dispatching queued work.

## Testing

- Cover live-setting precedence, persisted settings after restart, null legacy values, and model metadata returned by thread lifecycle operations.

GitOrigin-RevId: 22ebe8a8d1910a820d90a97730d812a00d8c1372
2026-09-01 22:56:18 +00:00
rka-oai e56e4922eb Support standalone tool outputs in turn/start (#41002)
## What changed

- Add an optional `toolOutput` to `turn/start` for starting or steering a turn with a named function-call output instead of user input.
- Emit standalone outputs as `functionCallOutput` thread items, persist them in durable history, and include them in resumed threads.
- Validate that tool outputs have a name, are not combined with nonempty `input`, satisfy the text-size limit, and use supported image URLs.

## Testing

- Cover standalone tool-output notifications, model input, and resumed history in both legacy and paginated history modes.
- Cover the text-size limit for structured tool output.

GitOrigin-RevId: a1eeec7b9b7de67b1a1c5e829a54552b85ec118b
2026-08-26 23:58:10 +00:00
Francis Chalissery 7276d67081 Expose resumable misalignment details through app server (#40952)
## Why

App-server clients need the explanation and steering instruction returned with a
misalignment policy violation to offer a user-confirmed continuation. Missing or
incomplete details must continue to behave as a terminal block.

## What changed

- Parse optional misalignment classification, explanation, and steering details
  from streamed and HTTP Responses errors and propagate them through core errors.
- Include the details in live app-server `error` and `turn/completed` payloads and
  export the corresponding protocol schemas and TypeScript types.
- Keep explanations and steering messages out of serialized rollout events and
  redact them from debug output.
- Document how clients can resume with `turn/start` after user confirmation.

## Testing

- Cover streamed, HTTP, and WebSocket-wrapped errors, including malformed and
  classification-only details.
- Verify live app-server notifications expose resumable details without writing
  the explanation or steering message to the rollout.

GitOrigin-RevId: 329258a444c2cd91d0c57ab1720830b33ddcfac5
2026-08-26 18:42:44 +00:00
Steve Coffey e0c727de04 Classify streaming rate-limit errors (#40931)
## What changed

- Classify `response.failed` events with the `rate_limit_exceeded` code as a distinct retryable error while preserving any parsed retry delay.
- Expose the error as `rateLimitExceeded` through the core protocol and app-server schemas after stream retries are exhausted.
- Preserve the upstream message for TUI display while keeping it out of telemetry summaries.

## Testing

- Cover SSE classification, retry metadata, protocol conversion and serialization, exhausted stream retries, telemetry redaction, and TUI rendering.

GitOrigin-RevId: 02dab4d3477dcd7653a58c49c4bd38687a616579
2026-08-26 17:45:42 +00:00
Owen Lin 5cb7a35de9 Promote paginated thread history APIs (#40673)
## What changed

- Make `thread/turns/list`, `thread/items/list`, and `thread/revert` available
  without the experimental API capability.
- Stabilize `thread/reverted`, `thread.historyMode`, and the pagination fields
  used by `thread/resume` and `thread/fork`.
- Publish the promoted requests, responses, and fields in the stable JSON and
  TypeScript schemas, and update the app-server documentation accordingly.

GitOrigin-RevId: 18b8c53c265211597bb1a62fd62f66f4a3df5f06
2026-08-25 19:03:27 +00:00