mirror of
https://github.com/openai/codex.git
synced 2026-09-04 15:08:45 +00:00
## Why Managed-network commands within one Codex conversation share the same HTTP and SOCKS proxy ingress. When several exec calls run concurrently, the proxy sees the requested destination but cannot tell which exec opened the connection. For example: ```text exec A: curl https://example.com/a ─┐ ├─> conversation proxy ─> Guardian exec B: curl https://example.com/b ─┘ host: example.com trigger: unknown ```. Three parallel network execs reached Guardian without their triggering call IDs or commands. Guardian denied the requests, but Codex could not safely associate those outcomes with the individual tool calls. ## What changes Keep the shared proxy ingress and tag each connection at the existing trusted Linux bridge: ```text exec A ─> existing Linux bridge ─> [token A][proxy bytes] ─┐ ├─> shared HTTP/SOCKS ingress exec B ─> existing Linux bridge ─> [token B][proxy bytes] ─┘ │ token A ─> exec A ─────┤ token B ─> exec B ─────┘ ``` The complete path is: ```text active exec registration │ ├─ registers its UUID as a short-lived attribution token ├─ passes the token to the Linux sandbox helper ├─ helper removes the token before launching the user command ├─ existing host bridge prepends the token to each proxy connection ├─ shared proxy consumes the bounded attribution frame └─ proxy attaches the matching execution-scoped state ├─ Guardian receives the exact call ID and command └─ a denial finishes/cancels the matching tool call ``` Dropping the active or deferred exec registration removes the token. Connections that were already accepted retain their resolved attribution; new connections using an expired token fail closed. ## Before and after Before, Guardian could receive only the network destination: ```json { "tool": "network_access", "host": "www.17track.net", "port": 443, "protocol": "https" } ``` After, the same request includes the action that caused it: ```json { "tool": "network_access", "host": "www.17track.net", "port": 443, "protocol": "https", "trigger": { "callId": "exec-network-first", "command": ["/bin/sh", "-c", "curl https://www.17track.net"] } } ``` ## Listener accounting This PR does **not** create proxy listeners per exec. ```text Existing topology: one conversation -> one HTTP listener + optional one SOCKS listener Discarded per-exec approach: one conversation -> existing listener pair + up to one additional listener pair per active exec This PR: one conversation -> existing listener pair only + one small token-map entry per active exec ``` The Linux sandbox already creates a trusted routing bridge for each sandboxed command. This PR adds a short frame write to that bridge rather than introducing another listener, task, or proxy process. The existing conversation-scoped listener pair remains. Making a single proxy service shared across multiple conversations would be a separate multi-tenant architecture change involving per-conversation policy, configuration, audit, and Guardian routing. ## Keeping the implementation small The attribution is bound once, when the TCP connection enters the proxy. The ingress installs an execution-scoped clone of the existing `NetworkProxyState`, so the established HTTP, SOCKS, MITM, policy, audit, and blocked-request paths continue using their existing state lookup. This avoids plumbing a new request-context type through every protocol handler. Outside the two ingress wrappers, protocol-specific request handling is unchanged. ## Security behavior - Tokens are generated from the existing random execution registration IDs. - The trusted Linux helper consumes and removes the token before executing user code. - Attribution frames have a fixed magic prefix, bounded token length, and bounded read timeout. - Unknown or expired tokens close the connection. - A token presented to a proxy for another environment closes the connection. - Existing unframed callers preserve the current conservative attribution behavior. ## Platform scope Exact bridge attribution is enabled on Linux. macOS and Windows retain their current shared-proxy behavior. ## Test coverage The concurrent end-to-end test starts two managed-network execs together and synchronizes them so both are active before either connects. It then inspects the two Guardian requests and compares the complete attribution pairs: ```text (exec-network-first, exact first command) (exec-network-second, exact second command) ``` Focused proxy coverage verifies the bounded frame and that a registered framed connection receives the matching execution and environment state. ## Scope This fixes the Linux network-to-exec attribution path and records a denial against the exact matching tool call. It intentionally does not change: - delivery of an entirely unattributed denial to the parent turn; - how parallel denials count toward the Guardian circuit breaker; - how the UI displays the rejection reason or completed-turn state. Those remain separate concerns from attribution. ## Relationship to #29456 and #29668 #29456 made the proxy environment and sandbox policy come from the same prepared network context. This PR adds the execution token to that prepared launch and consumes it at the shared ingress. This follows #29668's shared-ingress framing direction, but completes the production registration, Linux bridge, core call mapping, denial mapping, and concurrent end-to-end path. It also keeps attribution in the existing per-connection proxy state instead of introducing request-context plumbing through every HTTP, SOCKS, and MITM handler. This PR is intended to supersede #29668 for the Linux attribution fix. --------- Co-authored-by: viyatb-oai <viyatb@openai.com> Co-authored-by: Codex <noreply@openai.com>
1618 lines
54 KiB
Rust
1618 lines
54 KiB
Rust
mod execution_scope;
|
|
|
|
use crate::attribution::PROXY_ATTRIBUTION_TOKEN_ENV_KEY;
|
|
use crate::config;
|
|
use crate::credential_broker::BROKERED_CREDENTIALS_ENV_KEY;
|
|
use crate::credential_broker::CREDENTIAL_BROKER_ACTIVE_ENV_KEY;
|
|
use crate::http_proxy;
|
|
use crate::network_policy::NetworkPolicyDecider;
|
|
use crate::runtime::BlockedRequestObserver;
|
|
use crate::runtime::ConfigState;
|
|
use crate::runtime::unix_socket_permissions_supported;
|
|
use crate::socks5;
|
|
use crate::state::NetworkProxyState;
|
|
use anyhow::Context;
|
|
use anyhow::Result;
|
|
use clap::Parser;
|
|
use codex_utils_absolute_path::AbsolutePathBuf;
|
|
use serde::Deserialize;
|
|
use serde::Serialize;
|
|
use std::collections::HashMap;
|
|
use std::net::SocketAddr;
|
|
use std::net::TcpListener as StdTcpListener;
|
|
use std::sync::Arc;
|
|
use std::sync::Mutex;
|
|
use std::sync::RwLock;
|
|
use tokio::task::JoinHandle;
|
|
use tracing::warn;
|
|
|
|
use self::execution_scope::ExecutionScope;
|
|
|
|
#[derive(Debug, Clone, Parser)]
|
|
#[command(name = "codex-network-proxy", about = "Codex network sandbox proxy")]
|
|
pub struct Args {}
|
|
|
|
#[derive(Debug)]
|
|
struct ReservedListeners {
|
|
http: Mutex<Option<StdTcpListener>>,
|
|
socks: Mutex<Option<StdTcpListener>>,
|
|
}
|
|
|
|
impl ReservedListeners {
|
|
fn new(http: StdTcpListener, socks: Option<StdTcpListener>) -> Self {
|
|
Self {
|
|
http: Mutex::new(Some(http)),
|
|
socks: Mutex::new(socks),
|
|
}
|
|
}
|
|
|
|
fn take_http(&self) -> Option<StdTcpListener> {
|
|
let mut guard = self
|
|
.http
|
|
.lock()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
guard.take()
|
|
}
|
|
|
|
fn take_socks(&self) -> Option<StdTcpListener> {
|
|
let mut guard = self
|
|
.socks
|
|
.lock()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
guard.take()
|
|
}
|
|
}
|
|
|
|
struct ReservedListenerSet {
|
|
http_listener: StdTcpListener,
|
|
socks_listener: Option<StdTcpListener>,
|
|
}
|
|
|
|
impl ReservedListenerSet {
|
|
fn new(http_listener: StdTcpListener, socks_listener: Option<StdTcpListener>) -> Self {
|
|
Self {
|
|
http_listener,
|
|
socks_listener,
|
|
}
|
|
}
|
|
|
|
fn http_addr(&self) -> Result<SocketAddr> {
|
|
self.http_listener
|
|
.local_addr()
|
|
.context("failed to read reserved HTTP proxy address")
|
|
}
|
|
|
|
fn socks_addr(&self, default_addr: SocketAddr) -> Result<SocketAddr> {
|
|
self.socks_listener
|
|
.as_ref()
|
|
.map_or(Ok(default_addr), |listener| {
|
|
listener
|
|
.local_addr()
|
|
.context("failed to read reserved SOCKS5 proxy address")
|
|
})
|
|
}
|
|
|
|
fn into_reserved_listeners(self) -> Arc<ReservedListeners> {
|
|
Arc::new(ReservedListeners::new(
|
|
self.http_listener,
|
|
self.socks_listener,
|
|
))
|
|
}
|
|
}
|
|
|
|
#[derive(Clone)]
|
|
pub struct NetworkProxyBuilder {
|
|
state: Option<Arc<NetworkProxyState>>,
|
|
http_addr: Option<SocketAddr>,
|
|
socks_addr: Option<SocketAddr>,
|
|
managed_by_codex: bool,
|
|
policy_decider: Option<Arc<dyn NetworkPolicyDecider>>,
|
|
blocked_request_observer: Option<Arc<dyn BlockedRequestObserver>>,
|
|
}
|
|
|
|
impl Default for NetworkProxyBuilder {
|
|
fn default() -> Self {
|
|
Self {
|
|
state: None,
|
|
http_addr: None,
|
|
socks_addr: None,
|
|
managed_by_codex: true,
|
|
policy_decider: None,
|
|
blocked_request_observer: None,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl NetworkProxyBuilder {
|
|
pub fn state(mut self, state: Arc<NetworkProxyState>) -> Self {
|
|
self.state = Some(state);
|
|
self
|
|
}
|
|
|
|
pub fn http_addr(mut self, addr: SocketAddr) -> Self {
|
|
self.http_addr = Some(addr);
|
|
self
|
|
}
|
|
|
|
pub fn socks_addr(mut self, addr: SocketAddr) -> Self {
|
|
self.socks_addr = Some(addr);
|
|
self
|
|
}
|
|
|
|
pub fn managed_by_codex(mut self, managed_by_codex: bool) -> Self {
|
|
self.managed_by_codex = managed_by_codex;
|
|
self
|
|
}
|
|
|
|
pub fn policy_decider<D>(mut self, decider: D) -> Self
|
|
where
|
|
D: NetworkPolicyDecider,
|
|
{
|
|
self.policy_decider = Some(Arc::new(decider));
|
|
self
|
|
}
|
|
|
|
pub fn policy_decider_arc(mut self, decider: Arc<dyn NetworkPolicyDecider>) -> Self {
|
|
self.policy_decider = Some(decider);
|
|
self
|
|
}
|
|
|
|
pub fn blocked_request_observer<O>(mut self, observer: O) -> Self
|
|
where
|
|
O: BlockedRequestObserver,
|
|
{
|
|
self.blocked_request_observer = Some(Arc::new(observer));
|
|
self
|
|
}
|
|
|
|
pub fn blocked_request_observer_arc(
|
|
mut self,
|
|
observer: Arc<dyn BlockedRequestObserver>,
|
|
) -> Self {
|
|
self.blocked_request_observer = Some(observer);
|
|
self
|
|
}
|
|
|
|
pub async fn build(self) -> Result<NetworkProxy> {
|
|
let state = self.state.ok_or_else(|| {
|
|
anyhow::anyhow!(
|
|
"NetworkProxyBuilder requires a state; supply one via builder.state(...)"
|
|
)
|
|
})?;
|
|
state
|
|
.set_blocked_request_observer(self.blocked_request_observer.clone())
|
|
.await;
|
|
let current_cfg = state.current_cfg().await?;
|
|
let (requested_http_addr, requested_socks_addr, reserved_listeners) = if self
|
|
.managed_by_codex
|
|
{
|
|
let runtime = config::resolve_runtime(¤t_cfg)?;
|
|
#[cfg(target_os = "windows")]
|
|
let (managed_http_addr, managed_socks_addr) = config::clamp_bind_addrs(
|
|
runtime.http_addr,
|
|
runtime.socks_addr,
|
|
¤t_cfg.network,
|
|
);
|
|
#[cfg(target_os = "windows")]
|
|
let reserved = reserve_windows_managed_listeners(
|
|
managed_http_addr,
|
|
managed_socks_addr,
|
|
current_cfg.network.enable_socks5,
|
|
)
|
|
.context("reserve managed loopback proxy listeners")?;
|
|
#[cfg(not(target_os = "windows"))]
|
|
let reserved = reserve_loopback_ephemeral_listeners(current_cfg.network.enable_socks5)
|
|
.context("reserve managed loopback proxy listeners")?;
|
|
let http_addr = reserved.http_addr()?;
|
|
let socks_addr = reserved.socks_addr(runtime.socks_addr)?;
|
|
(
|
|
http_addr,
|
|
socks_addr,
|
|
Some(reserved.into_reserved_listeners()),
|
|
)
|
|
} else {
|
|
let runtime = config::resolve_runtime(¤t_cfg)?;
|
|
(
|
|
self.http_addr.unwrap_or(runtime.http_addr),
|
|
self.socks_addr.unwrap_or(runtime.socks_addr),
|
|
None,
|
|
)
|
|
};
|
|
|
|
// Reapply bind clamping for caller overrides so unix-socket proxying stays loopback-only.
|
|
let (http_addr, socks_addr) = config::clamp_bind_addrs(
|
|
requested_http_addr,
|
|
requested_socks_addr,
|
|
¤t_cfg.network,
|
|
);
|
|
|
|
Ok(NetworkProxy {
|
|
state,
|
|
http_addr,
|
|
socks_addr,
|
|
socks_enabled: current_cfg.network.enable_socks5,
|
|
socks5_udp_enabled: current_cfg.network.enable_socks5_udp,
|
|
runtime_settings: Arc::new(RwLock::new(NetworkProxyRuntimeSettings::from_config(
|
|
¤t_cfg,
|
|
)?)),
|
|
reserved_listeners,
|
|
policy_decider: self.policy_decider,
|
|
environment_proxies: Arc::new(Mutex::new(HashMap::new())),
|
|
execution_scope: None,
|
|
})
|
|
}
|
|
}
|
|
|
|
fn reserve_loopback_ephemeral_listeners(
|
|
reserve_socks_listener: bool,
|
|
) -> Result<ReservedListenerSet> {
|
|
let http_listener =
|
|
reserve_loopback_ephemeral_listener().context("reserve HTTP proxy listener")?;
|
|
let socks_listener = if reserve_socks_listener {
|
|
Some(reserve_loopback_ephemeral_listener().context("reserve SOCKS5 proxy listener")?)
|
|
} else {
|
|
None
|
|
};
|
|
Ok(ReservedListenerSet::new(http_listener, socks_listener))
|
|
}
|
|
|
|
#[cfg(target_os = "windows")]
|
|
fn reserve_windows_managed_listeners(
|
|
http_addr: SocketAddr,
|
|
socks_addr: SocketAddr,
|
|
reserve_socks_listener: bool,
|
|
) -> Result<ReservedListenerSet> {
|
|
let http_addr = windows_managed_loopback_addr(http_addr);
|
|
let socks_addr = windows_managed_loopback_addr(socks_addr);
|
|
|
|
match try_reserve_windows_managed_listeners(http_addr, socks_addr, reserve_socks_listener) {
|
|
Ok(listeners) => Ok(listeners),
|
|
Err(err) if err.kind() == std::io::ErrorKind::AddrInUse => {
|
|
warn!("managed Windows proxy ports are busy; falling back to ephemeral loopback ports");
|
|
reserve_loopback_ephemeral_listeners(reserve_socks_listener)
|
|
.context("reserve fallback loopback proxy listeners")
|
|
}
|
|
Err(err) => Err(err).context("reserve Windows managed proxy listeners"),
|
|
}
|
|
}
|
|
|
|
#[cfg(target_os = "windows")]
|
|
fn try_reserve_windows_managed_listeners(
|
|
http_addr: SocketAddr,
|
|
socks_addr: SocketAddr,
|
|
reserve_socks_listener: bool,
|
|
) -> std::io::Result<ReservedListenerSet> {
|
|
let http_listener = StdTcpListener::bind(http_addr)?;
|
|
let socks_listener = if reserve_socks_listener {
|
|
Some(StdTcpListener::bind(socks_addr)?)
|
|
} else {
|
|
None
|
|
};
|
|
Ok(ReservedListenerSet::new(http_listener, socks_listener))
|
|
}
|
|
|
|
#[cfg(target_os = "windows")]
|
|
fn windows_managed_loopback_addr(addr: SocketAddr) -> SocketAddr {
|
|
if !addr.ip().is_loopback() {
|
|
warn!(
|
|
"managed Windows proxies must bind to loopback; clamping {addr} to 127.0.0.1:{}",
|
|
addr.port()
|
|
);
|
|
}
|
|
SocketAddr::from(([127, 0, 0, 1], addr.port()))
|
|
}
|
|
|
|
fn reserve_loopback_ephemeral_listener() -> Result<StdTcpListener> {
|
|
StdTcpListener::bind(SocketAddr::from(([127, 0, 0, 1], 0)))
|
|
.context("bind loopback ephemeral port")
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
struct NetworkProxyRuntimeSettings {
|
|
allow_local_binding: bool,
|
|
allow_unix_sockets: Arc<[String]>,
|
|
dangerously_allow_all_unix_sockets: bool,
|
|
mitm_ca_trust_bundle: Option<crate::certs::ManagedMitmCaTrustBundle>,
|
|
}
|
|
|
|
impl NetworkProxyRuntimeSettings {
|
|
fn from_config(config: &config::NetworkProxyConfig) -> Result<Self> {
|
|
let mitm_ca_trust_bundle = if config.network.mitm {
|
|
let env = crate::certs::ca_env_from_process();
|
|
Some(crate::certs::managed_ca_trust_bundle(&env)?)
|
|
} else {
|
|
None
|
|
};
|
|
Ok(Self {
|
|
allow_local_binding: config.network.allow_local_binding,
|
|
allow_unix_sockets: config.network.allow_unix_sockets().into(),
|
|
dangerously_allow_all_unix_sockets: config.network.dangerously_allow_all_unix_sockets,
|
|
mitm_ca_trust_bundle,
|
|
})
|
|
}
|
|
}
|
|
|
|
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
|
struct EnvironmentProxyAddrs {
|
|
http_addr: SocketAddr,
|
|
socks_addr: SocketAddr,
|
|
}
|
|
|
|
/// Portable managed-network facts needed by an operating-system sandbox.
|
|
#[derive(Clone, Debug, Default, Eq, PartialEq, Serialize, Deserialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct ManagedNetworkSandboxContext {
|
|
/// Loopback proxy ports that sandboxed commands may connect to.
|
|
#[serde(default)]
|
|
pub loopback_ports: Vec<u16>,
|
|
/// Whether the command may bind local sockets and exchange loopback traffic.
|
|
#[serde(default)]
|
|
pub allow_local_binding: bool,
|
|
}
|
|
|
|
/// Environment-specific managed-network settings prepared for one command launch.
|
|
#[derive(Clone, Debug, Eq, PartialEq)]
|
|
pub struct PreparedManagedNetwork {
|
|
/// Complete command environment with managed proxy variables applied.
|
|
pub env: HashMap<String, String>,
|
|
/// Matching portable sandbox inputs for the command environment.
|
|
pub sandbox_context: ManagedNetworkSandboxContext,
|
|
}
|
|
|
|
struct EnvironmentProxy {
|
|
addrs: EnvironmentProxyAddrs,
|
|
http_task: JoinHandle<Result<()>>,
|
|
socks_task: Option<JoinHandle<Result<()>>>,
|
|
}
|
|
|
|
#[derive(Clone)]
|
|
pub struct NetworkProxy {
|
|
state: Arc<NetworkProxyState>,
|
|
http_addr: SocketAddr,
|
|
socks_addr: SocketAddr,
|
|
socks_enabled: bool,
|
|
socks5_udp_enabled: bool,
|
|
runtime_settings: Arc<RwLock<NetworkProxyRuntimeSettings>>,
|
|
reserved_listeners: Option<Arc<ReservedListeners>>,
|
|
policy_decider: Option<Arc<dyn NetworkPolicyDecider>>,
|
|
environment_proxies: Arc<Mutex<HashMap<String, EnvironmentProxy>>>,
|
|
execution_scope: Option<Arc<ExecutionScope>>,
|
|
}
|
|
|
|
impl std::fmt::Debug for NetworkProxy {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
// Avoid logging internal state (config contents, derived globsets, etc.) which can be noisy
|
|
// and may contain sensitive paths.
|
|
f.debug_struct("NetworkProxy")
|
|
.field("http_addr", &self.http_addr)
|
|
.field("socks_addr", &self.socks_addr)
|
|
.finish_non_exhaustive()
|
|
}
|
|
}
|
|
|
|
impl PartialEq for NetworkProxy {
|
|
fn eq(&self, other: &Self) -> bool {
|
|
self.http_addr == other.http_addr
|
|
&& self.socks_addr == other.socks_addr
|
|
&& self.runtime_settings() == other.runtime_settings()
|
|
}
|
|
}
|
|
|
|
impl Eq for NetworkProxy {}
|
|
|
|
pub const PROXY_URL_ENV_KEYS: &[&str] = &[
|
|
"HTTP_PROXY",
|
|
"HTTPS_PROXY",
|
|
"WS_PROXY",
|
|
"WSS_PROXY",
|
|
"ALL_PROXY",
|
|
"FTP_PROXY",
|
|
"YARN_HTTP_PROXY",
|
|
"YARN_HTTPS_PROXY",
|
|
"NPM_CONFIG_HTTP_PROXY",
|
|
"NPM_CONFIG_HTTPS_PROXY",
|
|
"NPM_CONFIG_PROXY",
|
|
"BUNDLE_HTTP_PROXY",
|
|
"BUNDLE_HTTPS_PROXY",
|
|
"PIP_PROXY",
|
|
"DOCKER_HTTP_PROXY",
|
|
"DOCKER_HTTPS_PROXY",
|
|
];
|
|
|
|
pub const ALL_PROXY_ENV_KEYS: &[&str] = &["ALL_PROXY", "all_proxy"];
|
|
pub const PROXY_ACTIVE_ENV_KEY: &str = "CODEX_NETWORK_PROXY_ACTIVE";
|
|
pub const ALLOW_LOCAL_BINDING_ENV_KEY: &str = "CODEX_NETWORK_ALLOW_LOCAL_BINDING";
|
|
const ELECTRON_GET_USE_PROXY_ENV_KEY: &str = "ELECTRON_GET_USE_PROXY";
|
|
const NODE_USE_ENV_PROXY_ENV_KEY: &str = "NODE_USE_ENV_PROXY";
|
|
#[cfg(any(target_os = "macos", test))]
|
|
const GIT_SSH_COMMAND_ENV_KEY: &str = "GIT_SSH_COMMAND";
|
|
pub const PROXY_ENV_KEYS: &[&str] = &[
|
|
PROXY_ACTIVE_ENV_KEY,
|
|
CREDENTIAL_BROKER_ACTIVE_ENV_KEY,
|
|
BROKERED_CREDENTIALS_ENV_KEY,
|
|
ALLOW_LOCAL_BINDING_ENV_KEY,
|
|
PROXY_ATTRIBUTION_TOKEN_ENV_KEY,
|
|
ELECTRON_GET_USE_PROXY_ENV_KEY,
|
|
NODE_USE_ENV_PROXY_ENV_KEY,
|
|
"HTTP_PROXY",
|
|
"HTTPS_PROXY",
|
|
"http_proxy",
|
|
"https_proxy",
|
|
"YARN_HTTP_PROXY",
|
|
"YARN_HTTPS_PROXY",
|
|
"npm_config_http_proxy",
|
|
"npm_config_https_proxy",
|
|
"npm_config_proxy",
|
|
"NPM_CONFIG_HTTP_PROXY",
|
|
"NPM_CONFIG_HTTPS_PROXY",
|
|
"NPM_CONFIG_PROXY",
|
|
"BUNDLE_HTTP_PROXY",
|
|
"BUNDLE_HTTPS_PROXY",
|
|
"PIP_PROXY",
|
|
"DOCKER_HTTP_PROXY",
|
|
"DOCKER_HTTPS_PROXY",
|
|
"WS_PROXY",
|
|
"WSS_PROXY",
|
|
"ws_proxy",
|
|
"wss_proxy",
|
|
"NO_PROXY",
|
|
"no_proxy",
|
|
"npm_config_noproxy",
|
|
"NPM_CONFIG_NOPROXY",
|
|
"YARN_NO_PROXY",
|
|
"BUNDLE_NO_PROXY",
|
|
"ALL_PROXY",
|
|
"all_proxy",
|
|
"FTP_PROXY",
|
|
"ftp_proxy",
|
|
];
|
|
|
|
#[cfg(target_os = "macos")]
|
|
pub const PROXY_GIT_SSH_COMMAND_ENV_KEY: &str = GIT_SSH_COMMAND_ENV_KEY;
|
|
|
|
const FTP_PROXY_ENV_KEYS: &[&str] = &["FTP_PROXY", "ftp_proxy"];
|
|
const WEBSOCKET_PROXY_ENV_KEYS: &[&str] = &["WS_PROXY", "WSS_PROXY", "ws_proxy", "wss_proxy"];
|
|
|
|
pub const NO_PROXY_ENV_KEYS: &[&str] = &[
|
|
"NO_PROXY",
|
|
"no_proxy",
|
|
"npm_config_noproxy",
|
|
"NPM_CONFIG_NOPROXY",
|
|
"YARN_NO_PROXY",
|
|
"BUNDLE_NO_PROXY",
|
|
];
|
|
|
|
pub const DEFAULT_NO_PROXY_VALUE: &str = concat!(
|
|
"localhost,127.0.0.1,::1,",
|
|
"10.0.0.0/8,",
|
|
"172.16.0.0/12,",
|
|
"192.168.0.0/16"
|
|
);
|
|
|
|
#[cfg(target_os = "macos")]
|
|
pub const CODEX_PROXY_GIT_SSH_COMMAND_MARKER: &str = "CODEX_PROXY_GIT_SSH_COMMAND=1 ";
|
|
#[cfg(target_os = "macos")]
|
|
const CODEX_PROXY_GIT_SSH_COMMAND_PREFIX: &str =
|
|
"CODEX_PROXY_GIT_SSH_COMMAND=1 ssh -o ProxyCommand='nc -X 5 -x ";
|
|
#[cfg(target_os = "macos")]
|
|
const CODEX_PROXY_GIT_SSH_COMMAND_SUFFIX: &str = " %h %p'";
|
|
|
|
pub fn proxy_url_env_value<'a>(
|
|
env: &'a HashMap<String, String>,
|
|
canonical_key: &str,
|
|
) -> Option<&'a str> {
|
|
if let Some(value) = env.get(canonical_key) {
|
|
return Some(value.as_str());
|
|
}
|
|
let lower_key = canonical_key.to_ascii_lowercase();
|
|
env.get(lower_key.as_str()).map(String::as_str)
|
|
}
|
|
|
|
pub fn has_proxy_url_env_vars(env: &HashMap<String, String>) -> bool {
|
|
PROXY_URL_ENV_KEYS
|
|
.iter()
|
|
.any(|key| proxy_url_env_value(env, key).is_some_and(|value| !value.trim().is_empty()))
|
|
}
|
|
|
|
fn set_env_keys(env: &mut HashMap<String, String>, keys: &[&str], value: &str) {
|
|
for key in keys {
|
|
env.insert((*key).to_string(), value.to_string());
|
|
}
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
fn codex_proxy_git_ssh_command(socks_addr: SocketAddr) -> String {
|
|
format!("{CODEX_PROXY_GIT_SSH_COMMAND_PREFIX}{socks_addr}{CODEX_PROXY_GIT_SSH_COMMAND_SUFFIX}")
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
fn is_codex_proxy_git_ssh_command(command: &str) -> bool {
|
|
command.starts_with(CODEX_PROXY_GIT_SSH_COMMAND_PREFIX)
|
|
&& command.ends_with(CODEX_PROXY_GIT_SSH_COMMAND_SUFFIX)
|
|
}
|
|
|
|
fn apply_proxy_env_overrides(
|
|
env: &mut HashMap<String, String>,
|
|
http_addr: SocketAddr,
|
|
socks_addr: SocketAddr,
|
|
socks_enabled: bool,
|
|
allow_local_binding: bool,
|
|
mitm_ca_trust_bundle: Option<&crate::certs::ManagedMitmCaTrustBundle>,
|
|
) {
|
|
let http_proxy_url = format!("http://{http_addr}");
|
|
let socks_proxy_url = format!("socks5h://{socks_addr}");
|
|
env.insert(PROXY_ACTIVE_ENV_KEY.to_string(), "1".to_string());
|
|
env.insert(
|
|
ALLOW_LOCAL_BINDING_ENV_KEY.to_string(),
|
|
if allow_local_binding {
|
|
"1".to_string()
|
|
} else {
|
|
"0".to_string()
|
|
},
|
|
);
|
|
|
|
// HTTP-based clients are best served by explicit HTTP proxy URLs.
|
|
set_env_keys(
|
|
env,
|
|
&[
|
|
"HTTP_PROXY",
|
|
"HTTPS_PROXY",
|
|
"http_proxy",
|
|
"https_proxy",
|
|
"YARN_HTTP_PROXY",
|
|
"YARN_HTTPS_PROXY",
|
|
"npm_config_http_proxy",
|
|
"npm_config_https_proxy",
|
|
"npm_config_proxy",
|
|
"NPM_CONFIG_HTTP_PROXY",
|
|
"NPM_CONFIG_HTTPS_PROXY",
|
|
"NPM_CONFIG_PROXY",
|
|
"BUNDLE_HTTP_PROXY",
|
|
"BUNDLE_HTTPS_PROXY",
|
|
"PIP_PROXY",
|
|
"DOCKER_HTTP_PROXY",
|
|
"DOCKER_HTTPS_PROXY",
|
|
],
|
|
&http_proxy_url,
|
|
);
|
|
// Some websocket clients look for dedicated WS/WSS proxy environment variables instead of
|
|
// HTTP(S)_PROXY. Keep them aligned with the managed HTTP proxy endpoint.
|
|
set_env_keys(env, WEBSOCKET_PROXY_ENV_KEYS, &http_proxy_url);
|
|
|
|
// Keep loopback and IP-literal private targets direct so local IPC/LAN access avoids the proxy.
|
|
// Do not include hostname suffixes here: those can force clients to resolve internal names
|
|
// locally instead of letting the proxy resolve them.
|
|
set_env_keys(env, NO_PROXY_ENV_KEYS, DEFAULT_NO_PROXY_VALUE);
|
|
|
|
env.insert(
|
|
ELECTRON_GET_USE_PROXY_ENV_KEY.to_string(),
|
|
"true".to_string(),
|
|
);
|
|
// Node.js built-in HTTP clients only honor proxy environment variables when this is enabled.
|
|
env.insert(NODE_USE_ENV_PROXY_ENV_KEY.to_string(), "1".to_string());
|
|
|
|
// Keep HTTP_PROXY/HTTPS_PROXY as HTTP endpoints. A lot of clients break if
|
|
// those vars contain SOCKS URLs. We only switch ALL_PROXY here.
|
|
//
|
|
if socks_enabled {
|
|
set_env_keys(env, ALL_PROXY_ENV_KEYS, &socks_proxy_url);
|
|
set_env_keys(env, FTP_PROXY_ENV_KEYS, &socks_proxy_url);
|
|
} else {
|
|
set_env_keys(env, ALL_PROXY_ENV_KEYS, &http_proxy_url);
|
|
set_env_keys(env, FTP_PROXY_ENV_KEYS, &http_proxy_url);
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
if socks_enabled {
|
|
// Preserve existing SSH wrappers (for example: Secretive/Teleport setups)
|
|
// but refresh a previously injected Codex fallback so it cannot point
|
|
// at a stale proxy port after the proxy is restarted.
|
|
match env.get(GIT_SSH_COMMAND_ENV_KEY) {
|
|
Some(command) if !is_codex_proxy_git_ssh_command(command) => {}
|
|
_ => {
|
|
env.insert(
|
|
GIT_SSH_COMMAND_ENV_KEY.to_string(),
|
|
codex_proxy_git_ssh_command(socks_addr),
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
if let Some(mitm_ca_trust_bundle) = mitm_ca_trust_bundle {
|
|
let managed_path = mitm_ca_trust_bundle.path.to_string_lossy().into_owned();
|
|
for key in crate::certs::CUSTOM_CA_ENV_KEYS {
|
|
if env
|
|
.get(key)
|
|
.filter(|value| !value.is_empty())
|
|
.is_some_and(|value| {
|
|
value != &managed_path
|
|
&& mitm_ca_trust_bundle.startup_env_values.get(key) != Some(value)
|
|
})
|
|
{
|
|
// TODO(winston): Materialize policy-checked per-child bundles for readable
|
|
// startup and command-scoped CA overrides. For now startup overrides are
|
|
// replaced with the default bundle and later command-scoped overrides are
|
|
// preserved, either of which can make intercepted TLS fail.
|
|
continue;
|
|
}
|
|
env.insert(key.to_string(), managed_path.clone());
|
|
}
|
|
}
|
|
}
|
|
|
|
impl NetworkProxy {
|
|
pub fn builder() -> NetworkProxyBuilder {
|
|
NetworkProxyBuilder::default()
|
|
}
|
|
|
|
pub fn http_addr(&self) -> SocketAddr {
|
|
self.http_addr
|
|
}
|
|
|
|
pub fn socks_addr(&self) -> SocketAddr {
|
|
self.socks_addr
|
|
}
|
|
|
|
pub async fn current_cfg(&self) -> Result<config::NetworkProxyConfig> {
|
|
self.state.current_cfg().await
|
|
}
|
|
|
|
pub async fn add_allowed_domain(&self, host: &str) -> Result<()> {
|
|
self.state.add_allowed_domain(host).await
|
|
}
|
|
|
|
pub async fn add_denied_domain(&self, host: &str) -> Result<()> {
|
|
self.state.add_denied_domain(host).await
|
|
}
|
|
|
|
pub fn allow_local_binding(&self) -> bool {
|
|
self.runtime_settings().allow_local_binding
|
|
}
|
|
|
|
pub fn allow_unix_sockets(&self) -> Arc<[String]> {
|
|
self.runtime_settings().allow_unix_sockets
|
|
}
|
|
|
|
pub fn dangerously_allow_all_unix_sockets(&self) -> bool {
|
|
self.runtime_settings().dangerously_allow_all_unix_sockets
|
|
}
|
|
|
|
/// Returns the generated MITM CA bundle path child sandboxes should expose to TLS clients.
|
|
pub fn managed_mitm_ca_trust_bundle_path(&self) -> Option<AbsolutePathBuf> {
|
|
self.runtime_settings()
|
|
.mitm_ca_trust_bundle
|
|
.and_then(|bundle| {
|
|
AbsolutePathBuf::from_absolute_path(bundle.path)
|
|
.map_err(|err| warn!("managed MITM CA trust bundle path is invalid: {err}"))
|
|
.ok()
|
|
})
|
|
}
|
|
|
|
fn prepare_for_addrs(
|
|
&self,
|
|
mut env: HashMap<String, String>,
|
|
addrs: EnvironmentProxyAddrs,
|
|
) -> PreparedManagedNetwork {
|
|
let runtime_settings = self.runtime_settings();
|
|
// Enforce proxying for child processes. Proxy endpoint values are always rewritten;
|
|
// managed MITM CA vars preserve child-scoped overrides after proxy startup.
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
addrs.http_addr,
|
|
addrs.socks_addr,
|
|
self.socks_enabled,
|
|
runtime_settings.allow_local_binding,
|
|
runtime_settings.mitm_ca_trust_bundle.as_ref(),
|
|
);
|
|
self.state.virtualize_child_credentials(&mut env);
|
|
if let Some(execution_scope) = self.execution_scope.as_ref() {
|
|
env.insert(
|
|
PROXY_ATTRIBUTION_TOKEN_ENV_KEY.to_string(),
|
|
execution_scope.attribution_token.clone(),
|
|
);
|
|
}
|
|
let mut loopback_ports = [
|
|
Some(addrs.http_addr),
|
|
self.socks_enabled.then_some(addrs.socks_addr),
|
|
]
|
|
.into_iter()
|
|
.flatten()
|
|
.filter(|addr| addr.ip().is_loopback())
|
|
.map(|addr| addr.port())
|
|
.collect::<Vec<_>>();
|
|
loopback_ports.sort_unstable();
|
|
loopback_ports.dedup();
|
|
PreparedManagedNetwork {
|
|
env,
|
|
sandbox_context: ManagedNetworkSandboxContext {
|
|
loopback_ports,
|
|
allow_local_binding: runtime_settings.allow_local_binding,
|
|
},
|
|
}
|
|
}
|
|
|
|
fn apply_to_env_for_addrs(
|
|
&self,
|
|
env: &mut HashMap<String, String>,
|
|
addrs: EnvironmentProxyAddrs,
|
|
) {
|
|
let prepared = self.prepare_for_addrs(std::mem::take(env), addrs);
|
|
*env = prepared.env;
|
|
}
|
|
|
|
pub fn apply_to_env(&self, env: &mut HashMap<String, String>) {
|
|
self.apply_to_env_for_addrs(
|
|
env,
|
|
EnvironmentProxyAddrs {
|
|
http_addr: self.http_addr,
|
|
socks_addr: self.socks_addr,
|
|
},
|
|
);
|
|
}
|
|
|
|
pub fn apply_to_env_for_environment(
|
|
&self,
|
|
env: &mut HashMap<String, String>,
|
|
environment_id: &str,
|
|
) -> Result<()> {
|
|
let addrs = self.environment_proxy_addrs(environment_id)?;
|
|
self.apply_to_env_for_addrs(env, addrs);
|
|
Ok(())
|
|
}
|
|
|
|
pub fn apply_to_env_for_optional_environment(
|
|
&self,
|
|
env: &mut HashMap<String, String>,
|
|
environment_id: Option<&str>,
|
|
) -> Result<()> {
|
|
match environment_id {
|
|
Some(environment_id) => self.apply_to_env_for_environment(env, environment_id),
|
|
None => {
|
|
self.apply_to_env(env);
|
|
Ok(())
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Applies the environment-specific proxy settings and returns the matching portable sandbox
|
|
/// projection from the same runtime configuration snapshot.
|
|
pub fn prepare_for_optional_environment(
|
|
&self,
|
|
env: HashMap<String, String>,
|
|
environment_id: Option<&str>,
|
|
) -> Result<PreparedManagedNetwork> {
|
|
let addrs = match environment_id {
|
|
Some(environment_id) => self.environment_proxy_addrs(environment_id)?,
|
|
None => EnvironmentProxyAddrs {
|
|
http_addr: self.http_addr,
|
|
socks_addr: self.socks_addr,
|
|
},
|
|
};
|
|
Ok(self.prepare_for_addrs(env, addrs))
|
|
}
|
|
|
|
fn environment_proxy_addrs(&self, environment_id: &str) -> Result<EnvironmentProxyAddrs> {
|
|
if let Some(execution_scope) = self.execution_scope.as_ref() {
|
|
anyhow::ensure!(
|
|
execution_scope.environment_id == environment_id,
|
|
"execution-scoped network proxy belongs to environment `{}`, not `{environment_id}`",
|
|
execution_scope.environment_id
|
|
);
|
|
}
|
|
|
|
let mut proxies = self
|
|
.environment_proxies
|
|
.lock()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
if let Some(proxy) = proxies.get(environment_id) {
|
|
return Ok(proxy.addrs);
|
|
}
|
|
|
|
let runtime = tokio::runtime::Handle::try_current().with_context(|| {
|
|
format!("failed to create network proxy for environment `{environment_id}`")
|
|
})?;
|
|
let listeners =
|
|
reserve_loopback_ephemeral_listeners(self.socks_enabled).with_context(|| {
|
|
format!("failed to reserve network proxy for environment `{environment_id}`")
|
|
})?;
|
|
let http_addr = listeners.http_addr().with_context(|| {
|
|
format!("failed to read HTTP proxy address for environment `{environment_id}`")
|
|
})?;
|
|
let socks_addr = listeners.socks_addr(self.socks_addr).with_context(|| {
|
|
format!("failed to read SOCKS proxy address for environment `{environment_id}`")
|
|
})?;
|
|
let addrs = EnvironmentProxyAddrs {
|
|
http_addr,
|
|
socks_addr,
|
|
};
|
|
let ReservedListenerSet {
|
|
http_listener,
|
|
socks_listener,
|
|
} = listeners;
|
|
|
|
let environment_id = environment_id.to_string();
|
|
let http_state = self.state.clone();
|
|
let http_decider = self.policy_decider.clone();
|
|
let http_environment_id = Some(environment_id.clone());
|
|
let http_task = runtime.spawn(async move {
|
|
http_proxy::run_http_proxy_with_std_listener(
|
|
http_state,
|
|
http_listener,
|
|
http_decider,
|
|
http_environment_id,
|
|
)
|
|
.await
|
|
});
|
|
|
|
let socks_task = if self.socks_enabled {
|
|
let socks_state = self.state.clone();
|
|
let socks_decider = self.policy_decider.clone();
|
|
let socks_environment_id = Some(environment_id.clone());
|
|
let socks5_udp_enabled = self.socks5_udp_enabled;
|
|
socks_listener.map(|listener| {
|
|
runtime.spawn(async move {
|
|
socks5::run_socks5_with_std_listener(
|
|
socks_state,
|
|
listener,
|
|
socks_decider,
|
|
socks_environment_id,
|
|
socks5_udp_enabled,
|
|
)
|
|
.await
|
|
})
|
|
})
|
|
} else {
|
|
None
|
|
};
|
|
|
|
proxies.insert(
|
|
environment_id,
|
|
EnvironmentProxy {
|
|
addrs,
|
|
http_task,
|
|
socks_task,
|
|
},
|
|
);
|
|
Ok(addrs)
|
|
}
|
|
|
|
pub async fn replace_config_state(&self, new_state: ConfigState) -> Result<()> {
|
|
let current_cfg = self.state.current_cfg().await?;
|
|
anyhow::ensure!(
|
|
new_state.config.network.enabled == current_cfg.network.enabled,
|
|
"cannot update network.enabled on a running proxy"
|
|
);
|
|
anyhow::ensure!(
|
|
new_state.config.network.proxy_url == current_cfg.network.proxy_url,
|
|
"cannot update network.proxy_url on a running proxy"
|
|
);
|
|
anyhow::ensure!(
|
|
new_state.config.network.socks_url == current_cfg.network.socks_url,
|
|
"cannot update network.socks_url on a running proxy"
|
|
);
|
|
anyhow::ensure!(
|
|
new_state.config.network.enable_socks5 == current_cfg.network.enable_socks5,
|
|
"cannot update network.enable_socks5 on a running proxy"
|
|
);
|
|
anyhow::ensure!(
|
|
new_state.config.network.enable_socks5_udp == current_cfg.network.enable_socks5_udp,
|
|
"cannot update network.enable_socks5_udp on a running proxy"
|
|
);
|
|
|
|
let settings = NetworkProxyRuntimeSettings::from_config(&new_state.config)?;
|
|
self.state.replace_config_state(new_state).await?;
|
|
let mut guard = self
|
|
.runtime_settings
|
|
.write()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
*guard = settings;
|
|
Ok(())
|
|
}
|
|
|
|
fn runtime_settings(&self) -> NetworkProxyRuntimeSettings {
|
|
self.runtime_settings
|
|
.read()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
|
.clone()
|
|
}
|
|
|
|
pub async fn run(&self) -> Result<NetworkProxyHandle> {
|
|
anyhow::ensure!(
|
|
self.execution_scope.is_none(),
|
|
"execution-scoped network proxy is already running"
|
|
);
|
|
let current_cfg = self.state.current_cfg().await?;
|
|
if !current_cfg.network.enabled {
|
|
warn!("network.enabled is false; skipping proxy listeners");
|
|
return Ok(NetworkProxyHandle::noop());
|
|
}
|
|
|
|
if !unix_socket_permissions_supported() {
|
|
warn!(
|
|
"allowUnixSockets and dangerouslyAllowAllUnixSockets are macOS-only; requests will be rejected on this platform"
|
|
);
|
|
}
|
|
|
|
let reserved_listeners = self.reserved_listeners.as_ref();
|
|
let http_listener = reserved_listeners.and_then(|listeners| listeners.take_http());
|
|
let socks_listener = reserved_listeners.and_then(|listeners| listeners.take_socks());
|
|
|
|
let http_state = self.state.clone();
|
|
let http_decider = self.policy_decider.clone();
|
|
let http_addr = self.http_addr;
|
|
let http_task = tokio::spawn(async move {
|
|
match http_listener {
|
|
Some(listener) => {
|
|
http_proxy::run_http_proxy_with_std_listener(
|
|
http_state,
|
|
listener,
|
|
http_decider,
|
|
/*environment_id*/ None,
|
|
)
|
|
.await
|
|
}
|
|
None => {
|
|
http_proxy::run_http_proxy(
|
|
http_state,
|
|
http_addr,
|
|
http_decider,
|
|
/*environment_id*/ None,
|
|
)
|
|
.await
|
|
}
|
|
}
|
|
});
|
|
|
|
let socks_task = if current_cfg.network.enable_socks5 {
|
|
let socks_state = self.state.clone();
|
|
let socks_decider = self.policy_decider.clone();
|
|
let socks_addr = self.socks_addr;
|
|
let enable_socks5_udp = current_cfg.network.enable_socks5_udp;
|
|
Some(tokio::spawn(async move {
|
|
match socks_listener {
|
|
Some(listener) => {
|
|
socks5::run_socks5_with_std_listener(
|
|
socks_state,
|
|
listener,
|
|
socks_decider,
|
|
/*environment_id*/ None,
|
|
enable_socks5_udp,
|
|
)
|
|
.await
|
|
}
|
|
None => {
|
|
socks5::run_socks5(
|
|
socks_state,
|
|
socks_addr,
|
|
socks_decider,
|
|
/*environment_id*/ None,
|
|
enable_socks5_udp,
|
|
)
|
|
.await
|
|
}
|
|
}
|
|
}))
|
|
} else {
|
|
None
|
|
};
|
|
|
|
Ok(NetworkProxyHandle {
|
|
http_task: Some(http_task),
|
|
socks_task,
|
|
environment_proxies: self.environment_proxies.clone(),
|
|
completed: false,
|
|
})
|
|
}
|
|
}
|
|
|
|
pub struct NetworkProxyHandle {
|
|
http_task: Option<JoinHandle<Result<()>>>,
|
|
socks_task: Option<JoinHandle<Result<()>>>,
|
|
environment_proxies: Arc<Mutex<HashMap<String, EnvironmentProxy>>>,
|
|
completed: bool,
|
|
}
|
|
|
|
impl NetworkProxyHandle {
|
|
fn noop() -> Self {
|
|
Self {
|
|
http_task: Some(tokio::spawn(async { Ok(()) })),
|
|
socks_task: None,
|
|
environment_proxies: Arc::new(Mutex::new(HashMap::new())),
|
|
completed: true,
|
|
}
|
|
}
|
|
|
|
pub async fn wait(mut self) -> Result<()> {
|
|
let http_task = self.http_task.take().context("missing http proxy task")?;
|
|
let socks_task = self.socks_task.take();
|
|
let http_result = http_task.await;
|
|
let socks_result = match socks_task {
|
|
Some(task) => Some(task.await),
|
|
None => None,
|
|
};
|
|
self.completed = true;
|
|
abort_environment_proxies(self.environment_proxies.clone()).await;
|
|
http_result??;
|
|
if let Some(socks_result) = socks_result {
|
|
socks_result??;
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
pub async fn shutdown(mut self) -> Result<()> {
|
|
abort_tasks(self.http_task.take(), self.socks_task.take()).await;
|
|
abort_environment_proxies(self.environment_proxies.clone()).await;
|
|
self.completed = true;
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
async fn abort_task(task: Option<JoinHandle<Result<()>>>) {
|
|
if let Some(task) = task {
|
|
task.abort();
|
|
let _ = task.await;
|
|
}
|
|
}
|
|
|
|
async fn abort_tasks(
|
|
http_task: Option<JoinHandle<Result<()>>>,
|
|
socks_task: Option<JoinHandle<Result<()>>>,
|
|
) {
|
|
abort_task(http_task).await;
|
|
abort_task(socks_task).await;
|
|
}
|
|
|
|
async fn abort_environment_proxies(
|
|
environment_proxies: Arc<Mutex<HashMap<String, EnvironmentProxy>>>,
|
|
) {
|
|
let proxies = {
|
|
let mut guard = environment_proxies
|
|
.lock()
|
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
|
guard.drain().map(|(_, proxy)| proxy).collect::<Vec<_>>()
|
|
};
|
|
for proxy in proxies {
|
|
abort_task(Some(proxy.http_task)).await;
|
|
abort_task(proxy.socks_task).await;
|
|
}
|
|
}
|
|
|
|
impl Drop for NetworkProxyHandle {
|
|
fn drop(&mut self) {
|
|
if self.completed {
|
|
return;
|
|
}
|
|
let http_task = self.http_task.take();
|
|
let socks_task = self.socks_task.take();
|
|
let environment_proxies = self.environment_proxies.clone();
|
|
tokio::spawn(async move {
|
|
abort_tasks(http_task, socks_task).await;
|
|
abort_environment_proxies(environment_proxies).await;
|
|
});
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::config::NetworkProxySettings;
|
|
use crate::state::network_proxy_state_for_policy;
|
|
use pretty_assertions::assert_eq;
|
|
use std::net::IpAddr;
|
|
use std::net::Ipv4Addr;
|
|
use std::path::Path;
|
|
|
|
#[tokio::test]
|
|
async fn managed_proxy_builder_uses_loopback_ports() {
|
|
let http_listener = StdTcpListener::bind(SocketAddr::from(([127, 0, 0, 1], 0))).unwrap();
|
|
let http_addr = http_listener.local_addr().unwrap();
|
|
let socks_listener = StdTcpListener::bind(SocketAddr::from(([127, 0, 0, 1], 0))).unwrap();
|
|
let socks_addr = socks_listener.local_addr().unwrap();
|
|
drop(http_listener);
|
|
drop(socks_listener);
|
|
|
|
let state = Arc::new(network_proxy_state_for_policy(NetworkProxySettings {
|
|
proxy_url: format!("http://{http_addr}"),
|
|
socks_url: format!("http://{socks_addr}"),
|
|
..NetworkProxySettings::default()
|
|
}));
|
|
let proxy = match NetworkProxy::builder().state(state).build().await {
|
|
Ok(proxy) => proxy,
|
|
Err(err) => {
|
|
if err
|
|
.chain()
|
|
.any(|cause| cause.to_string().contains("Operation not permitted"))
|
|
{
|
|
return;
|
|
}
|
|
panic!("failed to build managed proxy: {err:#}");
|
|
}
|
|
};
|
|
|
|
assert!(proxy.http_addr.ip().is_loopback());
|
|
assert!(proxy.socks_addr.ip().is_loopback());
|
|
#[cfg(target_os = "windows")]
|
|
{
|
|
assert_eq!(proxy.http_addr, http_addr);
|
|
assert_eq!(proxy.socks_addr, socks_addr);
|
|
}
|
|
#[cfg(not(target_os = "windows"))]
|
|
{
|
|
assert_ne!(proxy.http_addr.port(), 0);
|
|
assert_ne!(proxy.socks_addr.port(), 0);
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn non_codex_managed_proxy_builder_uses_configured_ports() {
|
|
let settings = NetworkProxySettings {
|
|
proxy_url: "http://127.0.0.1:43128".to_string(),
|
|
socks_url: "http://127.0.0.1:48081".to_string(),
|
|
..NetworkProxySettings::default()
|
|
};
|
|
let state = Arc::new(network_proxy_state_for_policy(settings));
|
|
let proxy = NetworkProxy::builder()
|
|
.state(state)
|
|
.managed_by_codex(/*managed_by_codex*/ false)
|
|
.build()
|
|
.await
|
|
.unwrap();
|
|
|
|
assert_eq!(
|
|
proxy.http_addr,
|
|
"127.0.0.1:43128".parse::<SocketAddr>().unwrap()
|
|
);
|
|
assert_eq!(
|
|
proxy.socks_addr,
|
|
"127.0.0.1:48081".parse::<SocketAddr>().unwrap()
|
|
);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn prepare_for_environment_keeps_env_and_sandbox_ports_in_sync() -> Result<()> {
|
|
let state = Arc::new(network_proxy_state_for_policy(
|
|
NetworkProxySettings::default(),
|
|
));
|
|
let proxy = NetworkProxy::builder().state(state).build().await?;
|
|
let handle = proxy.run().await?;
|
|
|
|
let base_env = HashMap::from([("PRESERVED".to_string(), "value".to_string())]);
|
|
let local = proxy.prepare_for_optional_environment(base_env.clone(), Some("local"))?;
|
|
let remote = proxy.prepare_for_optional_environment(HashMap::new(), Some("remote"))?;
|
|
|
|
assert_eq!(
|
|
local.env.get("PRESERVED").map(String::as_str),
|
|
Some("value")
|
|
);
|
|
assert_ne!(local.env.get("HTTP_PROXY"), remote.env.get("HTTP_PROXY"));
|
|
assert_ne!(
|
|
local.env.get("HTTP_PROXY"),
|
|
Some(&format!("http://{}", proxy.http_addr()))
|
|
);
|
|
assert_ne!(
|
|
remote.env.get("HTTP_PROXY"),
|
|
Some(&format!("http://{}", proxy.http_addr()))
|
|
);
|
|
for prepared in [&local, &remote] {
|
|
let http_port = prepared
|
|
.env
|
|
.get("HTTP_PROXY")
|
|
.and_then(|value| value.strip_prefix("http://"))
|
|
.and_then(|value| value.parse::<SocketAddr>().ok())
|
|
.map(|addr| addr.port())
|
|
.expect("managed HTTP proxy address");
|
|
let socks_port = prepared
|
|
.env
|
|
.get("ALL_PROXY")
|
|
.and_then(|value| value.strip_prefix("socks5h://"))
|
|
.and_then(|value| value.parse::<SocketAddr>().ok())
|
|
.map(|addr| addr.port())
|
|
.expect("managed SOCKS proxy address");
|
|
let mut expected_ports = vec![http_port, socks_port];
|
|
expected_ports.sort_unstable();
|
|
expected_ports.dedup();
|
|
assert_eq!(
|
|
prepared.sandbox_context,
|
|
ManagedNetworkSandboxContext {
|
|
loopback_ports: expected_ports,
|
|
allow_local_binding: false,
|
|
}
|
|
);
|
|
}
|
|
let mut legacy_env = base_env;
|
|
proxy.apply_to_env_for_environment(&mut legacy_env, "local")?;
|
|
assert_eq!(legacy_env, local.env);
|
|
|
|
handle.shutdown().await?;
|
|
Ok(())
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn managed_proxy_builder_does_not_reserve_socks_listener_when_disabled() {
|
|
let settings = NetworkProxySettings {
|
|
enable_socks5: false,
|
|
proxy_url: "http://127.0.0.1:43128".to_string(),
|
|
socks_url: "http://127.0.0.1:43129".to_string(),
|
|
..NetworkProxySettings::default()
|
|
};
|
|
let state = Arc::new(network_proxy_state_for_policy(settings));
|
|
let proxy = match NetworkProxy::builder().state(state).build().await {
|
|
Ok(proxy) => proxy,
|
|
Err(err) => {
|
|
if err
|
|
.chain()
|
|
.any(|cause| cause.to_string().contains("Operation not permitted"))
|
|
{
|
|
return;
|
|
}
|
|
panic!("failed to build managed proxy: {err:#}");
|
|
}
|
|
};
|
|
|
|
assert!(proxy.http_addr.ip().is_loopback());
|
|
assert_ne!(proxy.http_addr.port(), 0);
|
|
assert_eq!(
|
|
proxy.socks_addr,
|
|
"127.0.0.1:43129".parse::<SocketAddr>().unwrap()
|
|
);
|
|
assert!(
|
|
proxy
|
|
.reserved_listeners
|
|
.as_ref()
|
|
.expect("managed builder should reserve listeners")
|
|
.take_socks()
|
|
.is_none()
|
|
);
|
|
}
|
|
|
|
#[cfg(target_os = "windows")]
|
|
#[test]
|
|
fn windows_managed_loopback_addr_clamps_non_loopback_inputs() {
|
|
assert_eq!(
|
|
windows_managed_loopback_addr("0.0.0.0:3128".parse::<SocketAddr>().unwrap()),
|
|
"127.0.0.1:3128".parse::<SocketAddr>().unwrap()
|
|
);
|
|
assert_eq!(
|
|
windows_managed_loopback_addr("[::]:8081".parse::<SocketAddr>().unwrap()),
|
|
"127.0.0.1:8081".parse::<SocketAddr>().unwrap()
|
|
);
|
|
}
|
|
|
|
#[cfg(target_os = "windows")]
|
|
#[test]
|
|
fn reserve_windows_managed_listeners_falls_back_when_http_port_is_busy() {
|
|
let occupied = StdTcpListener::bind(SocketAddr::from(([127, 0, 0, 1], 0))).unwrap();
|
|
let busy_port = occupied.local_addr().unwrap().port();
|
|
|
|
let reserved = reserve_windows_managed_listeners(
|
|
SocketAddr::from(([127, 0, 0, 1], busy_port)),
|
|
SocketAddr::from(([127, 0, 0, 1], 48081)),
|
|
/*reserve_socks_listener*/ false,
|
|
)
|
|
.unwrap();
|
|
|
|
assert!(reserved.socks_listener.is_none());
|
|
assert!(
|
|
reserved
|
|
.http_listener
|
|
.local_addr()
|
|
.unwrap()
|
|
.ip()
|
|
.is_loopback()
|
|
);
|
|
assert_ne!(
|
|
reserved.http_listener.local_addr().unwrap().port(),
|
|
busy_port
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn proxy_url_env_value_resolves_lowercase_aliases() {
|
|
let mut env = HashMap::new();
|
|
env.insert(
|
|
"http_proxy".to_string(),
|
|
"http://127.0.0.1:3128".to_string(),
|
|
);
|
|
|
|
assert_eq!(
|
|
proxy_url_env_value(&env, "HTTP_PROXY"),
|
|
Some("http://127.0.0.1:3128")
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn has_proxy_url_env_vars_detects_lowercase_aliases() {
|
|
let mut env = HashMap::new();
|
|
env.insert(
|
|
"all_proxy".to_string(),
|
|
"socks5h://127.0.0.1:8081".to_string(),
|
|
);
|
|
|
|
assert_eq!(has_proxy_url_env_vars(&env), true);
|
|
}
|
|
|
|
#[test]
|
|
fn has_proxy_url_env_vars_detects_websocket_proxy_keys() {
|
|
let mut env = HashMap::new();
|
|
env.insert("wss_proxy".to_string(), "http://127.0.0.1:3128".to_string());
|
|
|
|
assert_eq!(has_proxy_url_env_vars(&env), true);
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_sets_common_tool_vars() {
|
|
let mut env = HashMap::new();
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get("HTTP_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("WS_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("WSS_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("npm_config_proxy"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("ALL_PROXY"),
|
|
Some(&"socks5h://127.0.0.1:8081".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("FTP_PROXY"),
|
|
Some(&"socks5h://127.0.0.1:8081".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("NO_PROXY"),
|
|
Some(&DEFAULT_NO_PROXY_VALUE.to_string())
|
|
);
|
|
let no_proxy = env.get("NO_PROXY").expect("NO_PROXY should be set");
|
|
assert!(no_proxy.contains("10.0.0.0/8"));
|
|
assert!(no_proxy.contains("172.16.0.0/12"));
|
|
assert!(no_proxy.contains("192.168.0.0/16"));
|
|
assert!(!no_proxy.contains("169.254.0.0/16"));
|
|
assert_eq!(env.get(PROXY_ACTIVE_ENV_KEY), Some(&"1".to_string()));
|
|
assert_eq!(env.get(ALLOW_LOCAL_BINDING_ENV_KEY), Some(&"0".to_string()));
|
|
assert_eq!(
|
|
env.get(ELECTRON_GET_USE_PROXY_ENV_KEY),
|
|
Some(&"true".to_string())
|
|
);
|
|
assert_eq!(env.get(NODE_USE_ENV_PROXY_ENV_KEY), Some(&"1".to_string()));
|
|
#[cfg(target_os = "macos")]
|
|
assert_eq!(
|
|
env.get(GIT_SSH_COMMAND_ENV_KEY),
|
|
Some(
|
|
&"CODEX_PROXY_GIT_SSH_COMMAND=1 ssh -o ProxyCommand='nc -X 5 -x 127.0.0.1:8081 %h %p'"
|
|
.to_string()
|
|
)
|
|
);
|
|
#[cfg(not(target_os = "macos"))]
|
|
assert_eq!(env.get(GIT_SSH_COMMAND_ENV_KEY), None);
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_sets_only_expected_env_keys() {
|
|
let mut env = HashMap::new();
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
for key in env.keys() {
|
|
let is_managed_git_ssh_key =
|
|
cfg!(target_os = "macos") && key == GIT_SSH_COMMAND_ENV_KEY;
|
|
assert!(
|
|
PROXY_ENV_KEYS.contains(&key.as_str()) || is_managed_git_ssh_key,
|
|
"proxy env writer set unexpected key: {key}"
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_sets_mitm_ca_trust_bundle_vars() {
|
|
let mut env = HashMap::new();
|
|
let mitm_ca_trust_bundle_path = Path::new("/tmp/codex-proxy/ca-bundle.pem");
|
|
let mitm_ca_trust_bundle = crate::certs::ManagedMitmCaTrustBundle {
|
|
path: mitm_ca_trust_bundle_path.to_path_buf(),
|
|
startup_env_values: HashMap::new(),
|
|
};
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
Some(&mitm_ca_trust_bundle),
|
|
);
|
|
|
|
for key in crate::certs::CUSTOM_CA_ENV_KEYS {
|
|
assert_eq!(
|
|
env.get(key),
|
|
Some(&mitm_ca_trust_bundle_path.display().to_string())
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_preserves_command_scoped_mitm_ca_override() {
|
|
let command_ca_bundle_path = "/tmp/command-ca.pem".to_string();
|
|
let mut env = HashMap::from([(
|
|
"REQUESTS_CA_BUNDLE".to_string(),
|
|
command_ca_bundle_path.clone(),
|
|
)]);
|
|
let mitm_ca_trust_bundle_path = Path::new("/tmp/codex-proxy/ca-bundle.pem");
|
|
let mitm_ca_trust_bundle = crate::certs::ManagedMitmCaTrustBundle {
|
|
path: mitm_ca_trust_bundle_path.to_path_buf(),
|
|
startup_env_values: HashMap::new(),
|
|
};
|
|
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
Some(&mitm_ca_trust_bundle),
|
|
);
|
|
|
|
assert_eq!(env.get("REQUESTS_CA_BUNDLE"), Some(&command_ca_bundle_path));
|
|
assert_eq!(
|
|
env.get("SSL_CERT_FILE"),
|
|
Some(&mitm_ca_trust_bundle_path.display().to_string())
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_uses_http_for_all_proxy_without_socks() {
|
|
let mut env = HashMap::new();
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ false,
|
|
/*allow_local_binding*/ true,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get("ALL_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(env.get(ALLOW_LOCAL_BINDING_ENV_KEY), Some(&"1".to_string()));
|
|
}
|
|
|
|
#[test]
|
|
fn apply_proxy_env_overrides_uses_plain_http_proxy_url() {
|
|
let mut env = HashMap::new();
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get("HTTP_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("HTTPS_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("WS_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("WSS_PROXY"),
|
|
Some(&"http://127.0.0.1:3128".to_string())
|
|
);
|
|
assert_eq!(
|
|
env.get("ALL_PROXY"),
|
|
Some(&"socks5h://127.0.0.1:8081".to_string())
|
|
);
|
|
#[cfg(target_os = "macos")]
|
|
assert_eq!(
|
|
env.get(GIT_SSH_COMMAND_ENV_KEY),
|
|
Some(
|
|
&"CODEX_PROXY_GIT_SSH_COMMAND=1 ssh -o ProxyCommand='nc -X 5 -x 127.0.0.1:8081 %h %p'"
|
|
.to_string()
|
|
)
|
|
);
|
|
#[cfg(not(target_os = "macos"))]
|
|
assert_eq!(env.get(GIT_SSH_COMMAND_ENV_KEY), None);
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
#[test]
|
|
fn apply_proxy_env_overrides_preserves_existing_git_ssh_command() {
|
|
let mut env = HashMap::new();
|
|
env.insert(
|
|
GIT_SSH_COMMAND_ENV_KEY.to_string(),
|
|
"ssh -o ProxyCommand='tsh proxy ssh --cluster=dev %r@%h:%p'".to_string(),
|
|
);
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get(GIT_SSH_COMMAND_ENV_KEY),
|
|
Some(&"ssh -o ProxyCommand='tsh proxy ssh --cluster=dev %r@%h:%p'".to_string())
|
|
);
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
#[test]
|
|
fn apply_proxy_env_overrides_preserves_unmarked_git_ssh_command_with_proxy_shape() {
|
|
let mut env = HashMap::new();
|
|
env.insert(
|
|
GIT_SSH_COMMAND_ENV_KEY.to_string(),
|
|
"ssh -o ProxyCommand='nc -X 5 -x 127.0.0.1:8081 %h %p'".to_string(),
|
|
);
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 3128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 48081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get(GIT_SSH_COMMAND_ENV_KEY),
|
|
Some(&"ssh -o ProxyCommand='nc -X 5 -x 127.0.0.1:8081 %h %p'".to_string())
|
|
);
|
|
}
|
|
|
|
#[cfg(target_os = "macos")]
|
|
#[test]
|
|
fn apply_proxy_env_overrides_refreshes_previous_codex_proxy_git_ssh_command() {
|
|
let mut env = HashMap::new();
|
|
env.insert(
|
|
GIT_SSH_COMMAND_ENV_KEY.to_string(),
|
|
codex_proxy_git_ssh_command(SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 8081)),
|
|
);
|
|
|
|
apply_proxy_env_overrides(
|
|
&mut env,
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 43128),
|
|
SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 48081),
|
|
/*socks_enabled*/ true,
|
|
/*allow_local_binding*/ false,
|
|
/*mitm_ca_trust_bundle*/ None,
|
|
);
|
|
|
|
assert_eq!(
|
|
env.get(GIT_SSH_COMMAND_ENV_KEY),
|
|
Some(&codex_proxy_git_ssh_command(SocketAddr::new(
|
|
IpAddr::V4(Ipv4Addr::LOCALHOST),
|
|
48081,
|
|
)))
|
|
);
|
|
}
|
|
}
|