Commit Graph
11 Commits
Author SHA1 Message Date
Matt Kaufer 592964c0cb Record local image inputs on completed image-generation items (#54420)
## What changed

Add optional `inputImages` metadata containing each referenced local image's path and SHA-256 hash of the bytes read before normalization. Compute the hash from the same bytes used to prepare the edit request.

Preserve this metadata through completion events, persisted thread history, resume, and legacy rollout migration. Update app-server schemas and generated TypeScript and Python types; older items continue to deserialize without the field.

## Testing

Add coverage for input ordering, changed contents at the same path, unreadable or invalid references, serialization, persisted history and resume, and legacy migration.

GitOrigin-RevId: 522be8ec1e39c9b2bbe5b9ccb37ea67d17f0e21a
2026-10-15 21:30:56 +00:00
Krish Chainani aabf13307a Store generated image items as attachment references (#54398)
## What changed

- Upload generated images to the attachment store and expose `fileId` in image-generation items when available, with inline `result` fallback on upload or decoding failure.
- Preserve file references through legacy events, persisted rollouts, thread history, and task output summaries. Keep model-facing image preparation separate.
- Add attachment request tags and mark generated-image uploads with `usage=image_gen`, preserving the thread's ephemeral setting.
- Update public schemas and SDK types to accept either image representation, retain the Python `ImageGenerationThreadItem` alias, and prefer `fileId` when reading records containing both fields.

## Testing

Add coverage for file-backed generation and rollout persistence, legacy history replay, inline and transitional serialization, task output visibility, and Python SDK alias compatibility.

GitOrigin-RevId: f7a2bb5707e2f277348a547a988bc33c8e96d891
2026-10-15 19:51:22 +00:00
Steve Coffey b0a6b8d86f Record resolved model and reasoning effort in sub-agent activity (#51463)
## What changed

Add `model` and `reasoningEffort` to `subAgentActivity` items for multi-agent v2 spawns. Capture startup settings before the child begins its turn and resolve the reported effort against model metadata, including model defaults, while preserving the child's configured effort.

Carry these fields through live notifications, persisted activity, thread history, and legacy rollout migration. Update the JSON schemas and generated TypeScript and Python types. Older records and other activity kinds retain null values.

## Testing

Add coverage for resolved spawn settings matching the child's request across legacy and paginated history, preserving configured effort, forwarding metadata through events and history, and reading and migrating older records without the new fields.

GitOrigin-RevId: a9e4241ae21bb8db1ead293f78ffba8257cca0ae
2026-10-06 19:29:40 +00:00
rb-oai 8b6bb1c77d Add a partial answer message phase (#51241)
## Why

Assistant answer text may be followed by more output or tools. A distinct `partial_answer` phase lets callers represent this separately from commentary and a terminal `final_answer`.

## What changed

- Add `MessagePhase::PartialAnswer` and expose `partial_answer` in the app-server schemas and generated TypeScript and Python types.
- Keep the TUI working indicator visible after partial answers and exclude them from the thread history's final-answer summary field.
- Route partial answers to the speakable channel in realtime V3 BEM-tag mode.

## Testing

Add coverage for phase preservation through serialization, input-message conversion, and SSE item start/completion events. Extend TUI snapshot coverage to verify that partial answers keep the working indicator visible while final answers hide it.

GitOrigin-RevId: 1ae3ad9264b6e36ae1cbe6b93df3d345ab4460e6
2026-10-06 02:57:15 +00:00
chiam-oai 772abc9425 Preserve and expose thread item lifecycle timestamps (#47114)
## What changed

- Persist producer-recorded item start and completion times in thread history, preserving existing non-null values on updates.
- Return nullable `startedAtMs` and `completedAtMs` Unix millisecond timestamps from `thread/items/list`, and update the generated schemas and TypeScript and Python types.
- Leave timestamps unknown for existing rows and older writers.

## Testing

Add coverage for exact timestamp persistence, protocol round trips with present and missing timestamps, paginated item responses, and compatibility with older database writers.

GitOrigin-RevId: 894853481085782ff54db1704799e41446d15e51
2026-09-21 22:23:26 +00:00
victor-openai b71af39fe6 Preserve MCP App UI metadata in tool-call events and history (#45805)
## Why

Clients need widget presentation details to render MCP Apps without waiting for the full MCP catalog, including when replaying saved history.

## What changed

- Add `mcpToolCall.mcpAppUi` with the invoked descriptor's `resourceUri` and `preferredModelDisplayMode`, and carry it through tool-call events and saved history.
- Support `fullscreen` and default to `inline` when the display preference is missing or unsupported.
- Keep existing resource URI fields for compatibility. Leave `mcpAppUi` null for older history and tools that declare widgets only in result metadata, where clients still use catalog discovery.
- Update protocol schemas, TypeScript and Python types, and app-server documentation.

## Testing

Add parameterized integration coverage for fullscreen, missing and unsupported preferences, legacy URI metadata, and result-only widgets, verifying consistent tool events and preservation across session resume.

GitOrigin-RevId: 2a9bed804dcea8c2b4903b8406fa6710681f7f5e
2026-09-15 22:52:56 +00:00
Krish Chainani 7b8b17b97a Support image references by file ID in inputs and tool outputs (#45794)
## What changed

- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.

## Testing

Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.

GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
2026-09-15 21:19:08 +00:00
Krish Chainani 5a66d460d3 Refactor image content to use a shared ImageReference type (#45543)
## What changed

Represent images in `ContentItem` and `FunctionCallOutputContentItem` with `ImageReference::Inline`, flattened to preserve the existing `image_url` wire format. Update image producers and consumers and regenerate app-server schemas and SDK artifacts.

Preserve the Python SDK's `InputImageContentItem` and `InputImageFunctionCallOutputContentItem` class names during generation.

## Testing

Add a regression test for stable Python image class names and adapt existing image tests to the shared representation.

GitOrigin-RevId: c38a780ac3314c2ac2deb3afc1b93b94b6f93fec
2026-09-14 23:25:09 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
rka-oai e56e4922eb Support standalone tool outputs in turn/start (#41002)
## What changed

- Add an optional `toolOutput` to `turn/start` for starting or steering a turn with a named function-call output instead of user input.
- Emit standalone outputs as `functionCallOutput` thread items, persist them in durable history, and include them in resumed threads.
- Validate that tool outputs have a name, are not combined with nonempty `input`, satisfy the text-size limit, and use supported image URLs.

## Testing

- Cover standalone tool-output notifications, model input, and resumed history in both legacy and paginated history modes.
- Cover the text-size limit for structured tool output.

GitOrigin-RevId: a1eeec7b9b7de67b1a1c5e829a54552b85ec118b
2026-08-26 23:58:10 +00:00
Owen Lin 5cb7a35de9 Promote paginated thread history APIs (#40673)
## What changed

- Make `thread/turns/list`, `thread/items/list`, and `thread/revert` available
  without the experimental API capability.
- Stabilize `thread/reverted`, `thread.historyMode`, and the pagination fields
  used by `thread/resume` and `thread/fork`.
- Publish the promoted requests, responses, and fields in the stable JSON and
  TypeScript schemas, and update the app-server documentation accordingly.

GitOrigin-RevId: 18b8c53c265211597bb1a62fd62f66f4a3df5f06
2026-08-25 19:03:27 +00:00