Commit Graph
6 Commits
Author SHA1 Message Date
malone hedges 24072feb33 Fix realtime input text payloads 2026-06-12 11:50:43 -07:00
guinness-oai 126bd6e7a8 Update realtime handoff transcript handling (#18597)
## Summary

This PR aims to improve integration between the realtime model and the
codex agent by sharing more context with each other. In particular, we
now share full realtime conversation transcript deltas in addition to
the delegation message.

realtime_conversation.rs now turns a handoff into:
```
<realtime_delegation>
  <input>...</input>
  <transcript_delta>...</transcript_delta>
</realtime_delegation>
```

## Implementation notes

The transcript is accumulated in the realtime websocket layer as parsed
realtime events arrive. When a background-agent handoff is requested,
the current transcript snapshot is copied onto the handoff event and
then serialized by `realtime_conversation.rs` into the hidden realtime
delegation envelope that Codex receives as user-turn context.

For Realtime V2, the session now explicitly enables input audio
transcription, and the parser handles the relevant input/output
transcript completion events so the snapshot includes both user speech
and realtime model responses. The delegation `<input>` remains the
actual handoff request, while `<transcript_delta>` carries the
surrounding conversation history for context.

Reviewers should note that the transcript payload is intended for Codex
context sharing, not UI rendering. The realtime delegation envelope
should stay hidden from the user-facing transcript surface, while still
being included in the background-agent turn so Codex can answer with the
same conversational context the realtime model had.
2026-04-20 14:04:09 -07:00
Ahmed Ibrahim 2f9090be62 Add realtime voice selection (#17176)
- Add realtime voice selection for realtime/start.
- Expose the supported v1/v2 voice lists and cover explicit, configured,
default, and invalid voice paths.
2026-04-08 20:19:15 -07:00
Ahmed IbrahimandCodex fb3dcfde1d Add WebRTC transport to realtime start (#16960)
Adds WebRTC startup to the experimental app-server
`thread/realtime/start` method with an optional transport enum. The
websocket path remains the default; WebRTC offers create the realtime
session through the shared start flow and emit the answer SDP via
`thread/realtime/sdp`.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-07 15:43:38 -07:00
Ahmed IbrahimandCodex fbd7f9b986 [stack 2/4] Align main realtime v2 wire and runtime flow (#14830)
## Stack Position
2/4. Built on top of #14828.

## Base
- #14828

## Unblocks
- #14829
- #14827

## Scope
- Port the realtime v2 wire parsing, session, app-server, and
conversation runtime behavior onto the split websocket-method base.
- Branch runtime behavior directly on the current realtime session kind
instead of parser-derived flow flags.
- Keep regression coverage in the existing e2e suites.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-16 21:38:07 -07:00
Ahmed IbrahimandCodex 6f05d8d735 [stack 1/4] Split realtime websocket methods by version (#14828)
## Stack Position
1/4. Base PR in the realtime stack.

## Base
- `main`

## Unblocks
- #14830

## Scope
- Split the realtime websocket request builders into `common`, `v1`, and
`v2` modules.
- Keep runtime behavior unchanged in this PR.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-16 16:00:59 -07:00