Commit Graph
11 Commits
Author SHA1 Message Date
pakrym-oai ce0323f2f2 Support typed content in contextual prompt fragments (#54776)
## Why

Contextual fragments and extension-owned world-state sections currently return strings, limiting them to text content.

## What changed

- Return `ContentItem` from fragment bodies and rendering, preserving classification and attribution. Apply text markers only to `InputText`; pass other content through unchanged.
- Allow extension prompt and world-state contributions to carry typed content, and preserve media in context-window hints.
- Hash non-text world-state fragments by their serialized content and role while retaining the existing text hashing behavior.
- Add shared content-to-tool-output conversion that preserves image references, image detail, and audio.

## Testing

Add coverage for typed world-state hashing and preservation of image content and attribution. Expand code-mode output tests to cover input text, output text, file-backed and inline images, and audio.

GitOrigin-RevId: 2ed00d466a3f169b151202f3eef71e667ea64be1
2026-10-17 04:02:21 +00:00
pakrym-oai c9253c4977 Send base instructions as Responses input messages (#51156)
## What changed

Move base instructions from the top-level `instructions` field into a prefixed `developer` message in `input` for standard Responses requests, matching Responses Lite. Remove the field from HTTP and WebSocket request types and assign instruction messages stable IDs derived from the thread and instruction text.

## Testing

Extend standard and Responses Lite tests to cover instruction message serialization, stable IDs, and resume consistency. Update WebSocket tests to verify that changed instructions trigger a full request without `previous_response_id`.

GitOrigin-RevId: e2de4b8e30a9339377fac133ceda981abd1a112b
2026-10-05 18:52:51 +00:00
Krish Chainani 7b8b17b97a Support image references by file ID in inputs and tool outputs (#45794)
## What changed

- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.

## Testing

Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.

GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
2026-09-15 21:19:08 +00:00
jwang-openai 4d8eca1ff3 Attribute command and plugin analytics to the invoking model (#45445)
## Why

Command execution and plugin measurement events lack model and reasoning-effort labels. Attribution needs to reflect the step that invoked the command, even when model settings change before a background process finishes.

## What changed

- Add `model_slug` and `reasoning_effort` to command execution and plugin measurement analytics.
- Capture model context from resolved step settings and carry it through execution, approval, Guardian review, and plugin metrics collection.
- Preserve the first command-start model context when subsequent start notifications arrive.
- Keep the carried context out of serialized protocol items and generated schemas.

## Testing

Extend analytics tests to cover model switches before invocation and during background execution, default reasoning effort, Guardian-denied commands, and repeated start notifications retaining the original model context.

GitOrigin-RevId: af90e1c0d39bab625f2e89786085b61a9b96c0ce
2026-09-14 15:30:22 +00:00
jif e1eb98461c Route approvals through the extension decision API (#43432)
## What changed

- Let approval extensions choose between cached approval, synchronous review, and a user prompt for tool and permission requests, while core enforces mandatory Guardian and fresh-review requirements.
- Carry an optional `review_reason` on Guardian assessment events and use it to trigger strict-review notifications, retaining the fallback for older events.
- Require synchronous review when cached evidence has an unusable encrypted parent compaction.

## Testing

Add regression coverage for manual prompts and cached approvals with non-UTF-8 working directories. Update approval contributors in existing tests to use the decision API.

GitOrigin-RevId: a4e66416070bdaa881daa41429c49021629f44c1
2026-09-07 11:07:28 +00:00
rka-oai 28327355b8 Update tests for default-enabled update_plan (#41630)
## What changed

- Cover the default, explicitly enabled, and explicitly disabled states of
  `tools.update_plan.enabled`.
- Verify prompt tool lists remain consistent across requests, including when
  custom base and developer instructions are configured.
- Explicitly enable `update_plan` in existing fixtures that depend on the tool.

GitOrigin-RevId: 3be279cb5195dfc5892c2c8ff6a1461fac879309
2026-08-30 07:40:36 +00:00
pakrym-oai 4aa52f2cb2 Gate content item kinds behind a feature flag (#40450)
## What changed

- Add the under-development `content_item_kinds` feature, disabled by default.
- Strip content item classifications from Responses input when the feature is
  disabled while preserving other passthrough metadata.

## Testing

- Add coverage that disables the feature and verifies requests omit content item
  kinds without removing the turn ID.

GitOrigin-RevId: f938feb2895d766d979b1666fc7b97fd128fe8a7
2026-08-24 18:22:13 +00:00
pakrym-oai 068847f76d Classify permission instructions under the permissions namespace (#40295)
## What changed

- Change the permission prompt content kind from
  `generic.permissions_instructions` to `permissions.instructions`.
- Update request annotation tests to expect the new content kind.

GitOrigin-RevId: e1e39c74ac24668ac1f3467bb0cc1335a9408712
2026-08-23 23:38:12 +00:00
pakrym-oai 287587c32c Classify additional generated context fragments (#40275)
## What changed

- Represent compaction summaries and Guardian-approved actions as typed contextual fragments.
- Inject subagent notifications through the same fragment path.
- Annotate the resulting request items with `compaction.summary`, `guardian.approved_action`, and `multi_agent.subagent_notification` content kinds while preserving their existing roles and text.

## Testing

- Verify the content-kind metadata on compacted summaries, approved actions, and subagent notifications.

GitOrigin-RevId: 01c93e7f08b2dc1c1e85ba69e82bfb5f56e6cca5
2026-08-23 20:08:40 +00:00
pakrym-oai 4582c0a498 Annotate user input and contextual fragments with content kinds (#40196)
## What changed

- Preserve each contextual fragment's `ContentItemKind` when converting it to a response message.
- Classify user text, image, and audio content as `user.text`, `user.image`, and `user.audio` in their original order.
- Give additional context key-specific `additional_content.<key>` kinds and retain annotations on generated messages such as image resize notices and interrupted-turn markers.

## Testing

- Expand request-level coverage to verify content-kind annotations across mixed-modality turns, additional context, reminders, budgets, skills, model switching, subagents, and image resizing.

GitOrigin-RevId: 671e4545f39b335525c60d2a5df4608e8068cafc
2026-08-23 06:10:44 +00:00
pakrym-oai c4b434aae2 Preserve context annotations in merged messages (#40184)
## What changed

- Carry each contextual fragment's `ContentItemKind` through initial context assembly and world-state updates.
- Attach the classifications to merged response messages through `content_item_kinds`, preserving their order alongside the message content.
- Convert extension prompt fragments into annotated developer fragments and classify built-in developer and guardian instructions.

## Testing

- Add an integration test covering roles and content annotations in the first request.
- Update world-state and persisted-history tests to account for annotation metadata.

GitOrigin-RevId: 71c48dad5b17bffd47ba03263e3eb6d5e74fbf4b
2026-08-23 04:41:11 +00:00