It's pretty amazing we have gotten here without the ability for the
model to see image content from MCP tool calls.
This PR builds off of 4391 and fixes#4819. I would like @KKcorps to get
adequete credit here but I also want to get this fix in ASAP so I gave
him a week to update it and haven't gotten a response so I'm going to
take it across the finish line.
This test highlights how absured the current situation is. I asked the
model to read this image using the Chrome MCP
<img width="2378" height="674" alt="image"
src="https://github.com/user-attachments/assets/9ef52608-72a2-4423-9f5e-7ae36b2b56e0"
/>
After this change, it correctly outputs:
> Captured the page: image dhows a dark terminal-style UI labeled
`OpenAI Codex (v0.0.0)` with prompt `model: gpt-5-codex medium` and
working directory `/codex/codex-rs`
(and more)
Before this change, it said:
> Took the full-page screenshot you asked for. It shows a long,
horizontally repeating pattern of stylized people in orange, light-blue,
and mustard clothing, holding hands in alternating poses against a white
background. No text or other graphics-just rows of flat illustration
stretching off to the right.
Without this change, the Figma, Playwright, Chrome, and other visual MCP
servers are pretty much entirely useless.
I tested this change with the openai respones api as well as a third
party completions api
Makes sense to move this struct to `app-server-protocol/` since we want
to serialize as camelCase, but we don't for structs defined in
`protocol/`
It was:
```
export type Account = { "type": "ApiKey", api_key: string, } | { "type": "chatgpt", email: string | null, plan_type: PlanType, };
```
But we want:
```
export type Account = { "type": "apiKey", apiKey: string, } | { "type": "chatgpt", email: string | null, planType: PlanType, };
```
move the truncation logic to conversation history to use on any tool
output. This will help us in avoiding edge cases while truncating the
tool calls and mcp calls.
Follow-up PR to #5569. Add Keyring Support for Auth Storage in Codex CLI
as well as a hybrid mode (default to persisting in keychain but fall
back to file when unavailable.)
It also refactors out the keyringstore implementation from rmcp-client
[here](https://github.com/openai/codex/blob/main/codex-rs/rmcp-client/src/oauth.rs)
to a new keyring-store crate.
There will be a follow-up that picks the right credential mode depending
on the config, instead of hardcoding `AuthCredentialsStoreMode::File`.
This PR introduces a new `Auth Storage` abstraction layer that takes
care of read, write, and load of auth tokens based on the
AuthCredentialsStoreMode. It is similar to how we handle MCP client
oauth
[here](https://github.com/openai/codex/blob/main/codex-rs/rmcp-client/src/oauth.rs).
Instead of reading and writing directly from disk for auth tokens, Codex
CLI workflows now should instead use this auth storage using the public
helper functions.
This PR is just a refactor of the current code so the behavior stays the
same. We will add support for keyring and hybrid mode in follow-up PRs.
I have read the CLA Document and I hereby sign the CLA
This PR does the following:
1. Changes `try_refresh_token` to handle the case where the endpoint
returns a response without an `id_token`. The OpenID spec indicates that
this field is optional and clients should not assume it's present.
2. Changes the `attempt_stream_responses` to propagate token refresh
errors rather than silently ignoring them.
3. Fixes a typo in a couple of error messages (unrelated to the above,
but something I noticed in passing) - "reconnect" should be spelled
without a hyphen.
This PR does not implement the additional suggestion from @pakrym-oai
that we should sign out when receiving `refresh_token_expired` from the
refresh endpoint. Leaving this as a follow-on because I'm undecided on
whether this should be implemented in `try_refresh_token` or its
callers.
This adds an RPC to the app server to the the `ConversationSummary` via
a rollout path. Now that the VS Code extension supports showing the
Codex UI in an editor panel where the URI of the panel maps to the
rollout file, we need to be able to get the `ConversationSummary` from
the rollout file directly.
An AppServer client should be able to use any (`model_provider`, `model`) in the user's config. `NewConversationParams` already supported specifying the `model`, but this PR expands it to support `model_provider`, as well.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/5793).
* #5803
* __->__ #5793
Because conversations that use the Responses API can have encrypted
reasoning messages, trying to resume a conversation with a different
provider could lead to confusing "failed to decrypt" errors. (This is
reproducible by starting a conversation using ChatGPT login and resuming
it as a conversation that uses OpenAI models via Azure.)
This changes `ListConversationsParams` to take a `model_providers:
Option<Vec<String>>` and adds `model_provider` on each
`ConversationSummary` it returns so these cases can be disambiguated.
Note this ended up making changes to
`codex-rs/core/src/rollout/tests.rs` because it had a number of cases
where it expected `Some` for the value of `next_cursor`, but the list of
rollouts was complete, so according to this docstring:
https://github.com/openai/codex/blob/bcd64c7e7231d6316a2377d1525a0fa74f21b783/codex-rs/app-server-protocol/src/protocol.rs#L334-L337
If there are no more items to return, then `next_cursor` should be
`None`. This PR updates that logic.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/5658).
* #5803
* #5793
* __->__ #5658
Revert #5642 because this generates:
```
// GENERATED CODE! DO NOT MODIFY BY HAND!
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
export type GetAccountResponse = Account | null;
```
But `Account` is unknown.
The unique use of `#[ts(export)]` on `GetAccountResponse` is also
suspicious as are the changes to
`codex-rs/app-server-protocol/src/export.rs` since the existing system
has worked fine for quite some time.
Though a pure backout of #5642 puts things in a state where, as the PR
noted, the following does not work:
```
cargo run -p codex-app-server-protocol --bin export -- --out DIR
```
So in addition to the backout, this PR adds:
```rust
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
pub struct GetAccountResponse {
pub account: Account,
}
```
and changes `GetAccount.response` as follows:
```diff
- response: Option<Account>,
+ response: GetAccountResponse,
```
making it consistent with other types.
With this change, I verified that both of the following work:
```
just codex generate-ts --out /tmp/somewhere
cargo run -p codex-app-server-protocol --bin export -- --out /tmp/somewhere-else
```
The generated TypeScript is as follows:
```typescript
// GetAccountResponse.ts
import type { Account } from "./Account";
export type GetAccountResponse = { account: Account, };
```
and
```typescript
// Account.ts
import type { PlanType } from "./PlanType";
export type Account = { "type": "ApiKey", api_key: string, } | { "type": "chatgpt", email: string | null, plan_type: PlanType, };
```
Though while the inconsistency between `"type": "ApiKey"` and `"type":
"chatgpt"` is quite concerning, I'm not sure if that format is ever
written to disk in any case, but @owenlin0, I would recommend looking
into that.
Also, it appears that the types in `codex-rs/protocol/src/account.rs`
are used exclusively by the `app-server-protocol` crate, so perhaps they
should just be moved there?
Gabriel Peal