diff --git a/.agents/skills/openshell-cli/SKILL.md b/.agents/skills/openshell-cli/SKILL.md index a36a372df7..b5ef7fd106 100644 --- a/.agents/skills/openshell-cli/SKILL.md +++ b/.agents/skills/openshell-cli/SKILL.md @@ -422,6 +422,8 @@ The `--from` flag accepts a Dockerfile path, a directory containing a Dockerfile Local Dockerfile and directory builds require a local gateway because the CLI builds through the local Docker daemon. Use a registry image reference for remote gateways. Bare community names resolve under `ghcr.io/nvidia/openshell-community/sandboxes` unless `OPENSHELL_COMMUNITY_REGISTRY` overrides the prefix. +For Docker and Podman, local `image` identity mode requires the image to declare a non-root OCI `USER`. Passwd-less images must use an explicit numeric pair such as `USER 1234:1235`. External bind mounts and named volumes require operator-configured `fixed` identity mode. Kubernetes and VM gateways retain their driver-specific identity behavior. + ### Forward ports ```bash diff --git a/architecture/compute-runtimes.md b/architecture/compute-runtimes.md index f211f906d6..b0c1aebaf0 100644 --- a/architecture/compute-runtimes.md +++ b/architecture/compute-runtimes.md @@ -73,15 +73,118 @@ template resource limits. Docker and Podman apply them as runtime limits. Kubernetes mirrors each limit into the matching request. VM accepts the fields but currently ignores them. -Docker and Podman also accept per-sandbox driver-config mounts for existing -runtime-managed named volumes and tmpfs mounts. Podman additionally accepts -image mounts through its image-volume API. User-supplied bind and volume mounts -default to read-only. Direct host bind mounts, and Docker or Podman local-driver -bind-backed named volumes, are available only when explicitly enabled in the -active local driver table of `gateway.toml`. Host bind mounts are an unsafe -operator override because they place gateway-host filesystem state inside the -sandbox and can negate OpenShell workspace isolation and filesystem-policy -controls. Driver-owned supervisor, token, and TLS bind mounts stay reserved. +## Docker and Podman Agent Identity + +By default, Docker and Podman preserve the selected sandbox image's non-root OCI +`USER` for agent processes. This local-container contract does not run the +supervisor as that user: the supervisor starts as root, prepares the isolation +boundary, and then drops privileges for direct and SSH-launched agent children. + +### Image Selection and Startup + +The local drivers bind identity inspection to the root filesystem that the +container engine starts. Docker implements this flow in +`crates/openshell-driver-docker/src/lib.rs`; Podman implements it in +`crates/openshell-driver-podman/src/driver.rs` and +`crates/openshell-driver-podman/src/container.rs`. Supervisor resolution and +persistence live in `crates/openshell-supervisor-process/src/identity.rs`. + +```mermaid +sequenceDiagram + participant Gateway + participant Driver as Docker/Podman driver + participant Engine as Container engine + participant Supervisor + participant Agent as Agent child + + Gateway->>Driver: CreateSandbox(spec) + Driver->>Engine: Resolve or pull requested image + Driver->>Engine: Inspect selected image + Engine-->>Driver: Immutable image ID + OCI Config.User + Driver->>Engine: Create by immutable ID with root supervisor + Engine->>Supervisor: Start in selected image rootfs + Supervisor->>Supervisor: Resolve and persist UID/GID + Supervisor->>Gateway: ConnectSupervisor(resolved identity) + Gateway-->>Supervisor: Persisted identity accepted + Supervisor->>Agent: Clear groups, setgid, setuid, exec +``` + +Docker inspects a local hit or the result of a pull and passes the returned image +ID, rather than the mutable reference, to container creation. Podman pulls +according to policy, inspects the selected result, and creates with +`image_pull_policy = "never"` and the inspected ID. Podman applies the same +inspection and pinning to its supervisor image and requested image mounts. A tag +can therefore move after inspection without changing the rootfs selected for +that sandbox. + +Both drivers insert the identity source, immutable image ID, and required +identity inputs after image and request environment, so the sandbox image cannot +override them. Docker sets the container user to `0`; Podman sets it to `0:0`. +Those values apply only to the supervisor, which needs root for namespace, +mount, network, filesystem, and privilege-drop setup. + +### Identity Modes + +The active Docker or Podman driver configuration selects one identity mode. + +| Mode | Agent identity | Storage boundary | +|---|---|---| +| `image` (default) | Resolve the inspected image's raw OCI `Config.User` to one non-root UID/GID. | Reject creator-selected bind and named-volume mounts. The driver-owned workspace and tmpfs mounts remain available; Podman also permits read-only image mounts pinned to inspected image IDs. | +| `fixed` | Use the operator-configured `fixed_uid` and `fixed_gid`; both must be present and within the process-policy numeric range. | Permit external or shared storage subject to the normal mount and bind-enable checks. | + +Image mode accepts named users and groups from the image's `/etc/passwd` and +`/etc/group`, or an explicit numeric `USER :` without account entries. +A numeric UID without a group requires a matching passwd entry from which to +obtain the primary GID. Missing users or groups and any identity resolving to +UID or GID 0 fail sandbox startup. The supervisor uses bounded, file-only +account parsing rather than NSS and does not create or rewrite image account +files. + +Before the first agent starts, the supervisor stores the resolved identity in +`/var/lib/openshell/agent-identity.json`, bound to the immutable image ID and, +for image mode, the raw OCI user declaration. It reports the same versioned +record to the gateway. The gateway persists the source, image ID, UID, GID, and +empty supplementary-group set and rejects a missing or changed record on later +connections. Restarts reuse the persisted identity rather than resolving a +mutable tag again. + +The gateway marks new Docker and Podman sandboxes as requiring this managed +identity and rejects creator-supplied `process.run_as_user` or +`process.run_as_group`; those policy fields cannot override the runtime's +selection. Existing containers without the creation marker retain their legacy +policy-derived identity and are not reinterpreted during upgrade. + +### External Mount Boundary + +Docker and Podman accept per-sandbox driver-config mounts for existing named +volumes and tmpfs mounts; Podman also accepts OCI image mounts. User-supplied +bind and volume mounts default to read-only, but read-only access does not make +external storage safe for an image-selected principal. Image mode therefore +rejects every creator-selected host bind and named volume even when the global +bind-mount switch is enabled. Operators must select fixed mode to bind external +or shared storage to an operator-controlled UID/GID. + +Fixed mode is necessary but not sufficient for direct host access. Host binds, +including local-driver bind-backed named volumes, still require +`enable_bind_mounts` in the active local driver table of `gateway.toml`. +Requested named volumes must already exist, and the driver inspects them to +detect bind-backed storage. User mounts cannot replace the driver-owned +workspace root, supervisor, token, or TLS paths, and cannot overlap +identity-sensitive account, identity-state, or UID/GID-map paths. + +Host bind mounts remain an unsafe operator override because they place gateway +host filesystem state inside the sandbox and can negate workspace isolation and +filesystem-policy controls. + +### Kubernetes, OpenShift, and VM + +The managed image-identity flow applies only to Docker and Podman. Kubernetes +continues to use an explicitly configured sandbox UID/GID, an OpenShift SCC +namespace-assigned range when available, or its existing numeric defaults. The +VM driver continues to use its configured numeric guest UID/GID, defaulting to +`10001` and using the UID as the GID when no GID is configured. These runtimes +do not derive agent identity from OCI `Config.User` and do not participate in +the Docker/Podman resolved-identity handshake. Kubernetes deployments may set an AppArmor profile on sandbox agent containers through the driver configuration. The Helm chart defaults sandbox agents to diff --git a/architecture/sandbox.md b/architecture/sandbox.md index 4f95e1ef69..dd44fcf8de 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -10,8 +10,8 @@ Each sandbox workload has two trust levels: | Process | Role | |---|---| -| Supervisor | Starts as root inside the workload, prepares isolation, runs the proxy, fetches config, injects credentials, serves the relay socket, and launches child processes. | -| Agent child | Runs as an unprivileged user with filesystem, process, and network restrictions applied. | +| Supervisor | Runs as root inside the workload, prepares isolation, runs the proxy, fetches config, injects credentials, serves the relay socket, and launches child processes. | +| Agent child | Runs as the selected unprivileged identity with filesystem, process, and network restrictions applied. Managed local children use a resolved numeric UID and GID. | The supervisor keeps enough privilege to manage the sandbox, but the agent child loses that privilege before user code runs. On Linux, child setup clears the @@ -21,18 +21,82 @@ container-granted capabilities. This is fail-closed: the supervisor retains aborts unless the bounding set ends up empty. A `setpcap` `EPERM` is tolerated only when the set is already empty; any other outcome fails the spawn. +### Managed local identity + +Docker and Podman select the agent identity; the gateway rejects creator-set +`process.run_as_user` or `process.run_as_group` values for these runtimes. Their +default `image` mode inspects the selected image's immutable ID and OCI +`Config.User`. Operators can instead configure `fixed` mode with both a numeric +UID and GID in the policy identity range. Fixed mode is explicit and is the mode +used when external storage requires an operator-controlled identity; image mode +rejects bind and named-volume mounts that could have incompatible ownership. + +Both local drivers override the image entrypoint and OCI user so the supervisor +starts as `0:0`. They pass the inspected image metadata or fixed IDs through +driver-owned environment entries that template, request, and image environment +values cannot replace. The supervisor then resolves the identity once and +normalizes the process policy to its numeric UID and GID before launching any +agent child. + +Image mode accepts only a non-empty OCI `USER` in `user`, `uid`, `user:group`, +or `uid:gid` form. Named fields and an implicit primary group must resolve from +the image's `/etc/passwd` and `/etc/group`; missing, ambiguous, malformed, or +root-resolving identities fail startup. Resolution parses bounded, regular +account files directly without following symlinks or invoking NSS. OpenShell +does not add accounts or modify the image's passwd or group files. Fixed mode +does not require account-file entries, but still rejects UID or GID 0. + +The first successful resolution is persisted at +`/var/lib/openshell/agent-identity.json` and bound to the immutable image ID, +identity source, and source-specific inputs. A restart fails if those inputs do +not match the persisted record. Managed children receive no supplementary +groups: child setup clears and verifies the group list before `setgid` and +`setuid`, and aborts the launch if it cannot prove the list is empty. + +The gateway persists `SandboxStatus.managed_identity_required = true` when it +creates a Docker- or Podman-managed sandbox. This creation-time marker is the +only authorization for the supervisor to resolve and report a managed local +identity. Unmarked legacy, offline, Kubernetes, and VM sandboxes do not +interpret Docker/Podman source and image metadata. Kubernetes masks and VM +strips those environment keys as defense in depth, so an image cannot opt either +runtime into this protocol. + +Before exposing SSH or launching the direct entrypoint, a marked supervisor +sends the resolved identity in `SupervisorHello`. The gateway validates the +schema version, source, non-empty immutable image ID, non-root UID/GID, and +empty supplementary-group list. It atomically stores the first identity in +`SandboxStatus.resolved_identity`; later sessions must match it exactly. The +supervisor proceeds only after `SessionAccepted`, and fails closed if the +gateway rejects the identity or does not accept it before the startup timeout. +The supervisor also emits an informational OCSF configuration-state event with +the resolved source, image ID, UID, GID, and empty supplementary-group list. + +Filesystem preparation changes ownership only where required. For a +driver-selected identity, it changes the `/sandbox` directory itself but never +recursively traverses image content or nested mounts. Existing policy +`read_write` paths retain their ownership; a missing final directory is created +and owned through descriptor-relative, no-follow operations that reject symlink +races. +The implementation lives in +`crates/openshell-supervisor-process/src/identity.rs` and +`crates/openshell-supervisor-process/src/process.rs`. + ## Startup Flow -1. The compute runtime starts the workload with sandbox identity, callback - endpoint, TLS or secret material, image metadata, and initial command. +1. The compute runtime starts the root supervisor with sandbox identity, + callback endpoint, TLS or secret material, image metadata, and initial + command. 2. The supervisor loads policy and runtime settings from local files or the gateway, depending on mode. -3. It prepares filesystem access, process restrictions, network namespace - routing, trust stores, provider credential resolution, and inference routes. -4. It starts the policy proxy and local SSH server. -5. It opens a supervisor session back to the gateway for connect, exec, file - sync, config polling, and log push. -6. It launches the agent command as the restricted sandbox user. +3. For a marked local sandbox, it resolves and persists the numeric agent + identity. It then prepares filesystem access, process restrictions, network + namespace routing, trust stores, provider credentials, and inference routes. +4. It starts the policy proxy and opens the outbound supervisor session. A + marked sandbox waits for gateway identity acceptance. +5. It starts the local SSH server and launches the agent command. Direct and SSH + children drop to the same resolved identity before executing user code. +6. The supervisor session carries connect, exec, file sync, config polling, and + log traffic for the lifetime of the sandbox. ## Isolation Layers @@ -41,7 +105,7 @@ OpenShell uses overlapping controls rather than a single sandbox primitive: | Layer | Purpose | |---|---| | Filesystem policy | Landlock restricts the paths the agent can read or write. | -| Process policy | The child process runs as a non-root user with reduced privileges. | +| Process policy | The child process runs as a non-root identity without retained capabilities. Managed local children use numeric UID/GID and no supplementary groups. | | Seccomp | Blocks dangerous syscalls, including raw socket paths that bypass the proxy. | | Network namespace | Forces ordinary agent egress through the local CONNECT proxy. | | Policy proxy | Evaluates destination, binary identity, TLS/L7 rules, SSRF checks, and inference interception. | @@ -174,9 +238,10 @@ operator decision. Provider credentials are stored at the gateway and fetched by the supervisor at runtime. The supervisor injects resolved environment variables into the initial -agent process and SSH child processes. Driver-controlled environment variables -override template values so sandbox images cannot spoof identity, callback, or -relay settings. +agent process and SSH child processes. Driver-controlled bootstrap values +override template values so sandbox images cannot spoof callback or relay +settings; managed local identity uses the creation marker and acceptance +protocol described above rather than trusting environment presence alone. Supervisor bootstrap identity is not inherited by agent child processes. When provider token grants mount a SPIFFE Workload API socket, the socket path must diff --git a/crates/openshell-core/src/driver_mounts.rs b/crates/openshell-core/src/driver_mounts.rs index 086d992c37..ee727010ef 100644 --- a/crates/openshell-core/src/driver_mounts.rs +++ b/crates/openshell-core/src/driver_mounts.rs @@ -32,6 +32,15 @@ const RESERVED_MOUNT_TARGETS: &[&str] = &[ "/run/netns", ]; +const IDENTITY_SENSITIVE_MOUNT_TARGETS: &[&str] = &[ + "/etc/passwd", + "/etc/group", + "/var/lib/openshell", + "/proc/self/uid_map", + "/proc/self/gid_map", + "/proc/self/setgroups", +]; + /// Validate a non-empty driver mount source. pub fn validate_mount_source(source: &str, field: &str) -> Result<(), String> { if source.is_empty() { @@ -124,6 +133,15 @@ pub fn validate_container_mount_target(target: &str) -> Result<(), String> { )); } } + for sensitive in IDENTITY_SENSITIVE_MOUNT_TARGETS { + let sensitive = Path::new(sensitive); + if path_is_or_under(path, sensitive) || path_is_or_under(sensitive, path) { + return Err(format!( + "mount target '{target}' overlaps identity-sensitive path '{}'", + sensitive.display() + )); + } + } Ok(()) } @@ -176,6 +194,54 @@ mod tests { validate_container_mount_target("/etc/openshell-tools").unwrap(); } + #[test] + fn container_target_rejects_identity_sensitive_paths_and_descendants() { + for &sensitive in IDENTITY_SENSITIVE_MOUNT_TARGETS { + for target in [sensitive.to_string(), format!("{sensitive}/shadowed")] { + let err = validate_container_mount_target(&target).unwrap_err(); + + assert!( + err.contains("overlaps identity-sensitive path"), + "unexpected error for {target}: {err}" + ); + assert!( + err.contains(sensitive), + "error for {target} does not identify {sensitive}: {err}" + ); + } + } + } + + #[test] + fn container_target_rejects_identity_sensitive_path_ancestors() { + for target in ["/etc", "/var", "/var/lib", "/proc", "/proc/self"] { + let err = validate_container_mount_target(target).unwrap_err(); + + assert!( + err.contains("overlaps identity-sensitive path"), + "unexpected error for {target}: {err}" + ); + } + } + + #[test] + fn container_target_allows_identity_path_prefix_near_misses() { + for target in [ + "/etc-agent", + "/etc/passwd-cache", + "/etc/group.d", + "/var/lib/openshell-data", + "/proc-agent", + "/proc/selfish", + "/proc/self/uid_map-backup", + "/proc/self/gid_map-backup", + "/proc/self/setgroups-backup", + ] { + validate_container_mount_target(target) + .unwrap_or_else(|err| panic!("unexpected error for {target}: {err}")); + } + } + #[test] fn mount_subpath_must_be_relative_without_parent_dirs() { assert!(validate_mount_subpath("project/a").is_ok()); diff --git a/crates/openshell-core/src/grpc_client.rs b/crates/openshell-core/src/grpc_client.rs index 070704fb0a..454729fe67 100644 --- a/crates/openshell-core/src/grpc_client.rs +++ b/crates/openshell-core/src/grpc_client.rs @@ -780,6 +780,8 @@ pub struct SettingsPollResult { pub supervisor_middleware_services: Vec, /// Workspace the sandbox belongs to. pub workspace: String, + /// Whether the gateway requires managed runtime identity for this sandbox. + pub managed_identity_required: bool, } fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult { @@ -795,6 +797,21 @@ fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> Settin provider_env_revision: inner.provider_env_revision, supervisor_middleware_services: inner.supervisor_middleware_services, workspace: inner.workspace, + managed_identity_required: inner.managed_identity_required, + } +} + +#[cfg(test)] +mod settings_snapshot_tests { + use super::*; + + #[test] + fn settings_snapshot_propagates_managed_identity_requirement() { + let snapshot = settings_poll_result(crate::proto::GetSandboxConfigResponse { + managed_identity_required: true, + ..Default::default() + }); + assert!(snapshot.managed_identity_required); } } diff --git a/crates/openshell-core/src/sandbox_env.rs b/crates/openshell-core/src/sandbox_env.rs index f15ce34bb1..816d090047 100644 --- a/crates/openshell-core/src/sandbox_env.rs +++ b/crates/openshell-core/src/sandbox_env.rs @@ -8,6 +8,8 @@ //! supervisor process (which reads them on startup). Using constants here //! prevents typos from producing silently broken sandboxes. +use serde::{Deserialize, Serialize}; + /// Name of the sandbox (used for policy sync and identification). pub const SANDBOX: &str = "OPENSHELL_SANDBOX"; @@ -103,10 +105,10 @@ pub const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET: &str = /// Resolved sandbox UID used to override `run_as_user` when the policy /// specifies a numeric value instead of the hardcoded "sandbox" user name. /// -/// Set by compute drivers (Kubernetes, Docker, VM) from resolved config or -/// cluster autodetection. The supervisor reads this at startup and uses it -/// directly with `setuid()` / `chown()` without requiring an `/etc/passwd` -/// entry in the sandbox image. +/// Set by Kubernetes and VM drivers from resolved config or cluster +/// autodetection, and by local drivers in explicit fixed identity mode. The +/// supervisor reads it at startup and uses it directly with `setuid()` / +/// `chown()` without requiring an `/etc/passwd` entry in the sandbox image. pub const SANDBOX_UID: &str = "OPENSHELL_SANDBOX_UID"; /// Resolved sandbox GID paired with [`SANDBOX_UID`]. @@ -115,6 +117,118 @@ pub const SANDBOX_UID: &str = "OPENSHELL_SANDBOX_UID"; /// supervisor drops privileges to a group other than the UID's primary group. pub const SANDBOX_GID: &str = "OPENSHELL_SANDBOX_GID"; +/// Agent identity selection mode (`image` or `fixed`). +/// +/// Docker and Podman set this protected value after image and request +/// environment variables. When absent, the supervisor preserves the legacy +/// policy-based identity behavior used by existing containers. +pub const IDENTITY_SOURCE: &str = "OPENSHELL_IDENTITY_SOURCE"; + +/// Raw OCI image `Config.User` declaration used by [`IdentitySource::Image`]. +pub const IMAGE_USER: &str = "OPENSHELL_IMAGE_USER"; + +/// Immutable image identifier paired with [`IMAGE_USER`]. +pub const IMAGE_ID: &str = "OPENSHELL_IMAGE_ID"; + +/// Current schema version for resolved agent identity exchange. +pub const RESOLVED_AGENT_IDENTITY_VERSION: u32 = 1; + +/// Source used to select the agent process identity. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum IdentitySource { + /// Resolve the image's OCI `USER` declaration inside the image rootfs. + Image, + /// Use operator-selected numeric [`SANDBOX_UID`] and [`SANDBOX_GID`]. + Fixed, +} + +impl std::str::FromStr for IdentitySource { + type Err = String; + + fn from_str(value: &str) -> Result { + match value { + "image" => Ok(Self::Image), + "fixed" => Ok(Self::Fixed), + _ => Err(format!( + "unsupported identity source '{value}'; expected 'image' or 'fixed'" + )), + } + } +} + +impl std::fmt::Display for IdentitySource { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Image => formatter.write_str("image"), + Self::Fixed => formatter.write_str("fixed"), + } + } +} + +/// Numeric identity selected for agent children and persisted across restarts. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct ResolvedAgentIdentity { + pub uid: u32, + pub gid: u32, + pub presentation_user: String, + pub source: IdentitySource, + #[serde(skip_serializing_if = "Option::is_none")] + pub image_id: Option, +} + +impl From<&ResolvedAgentIdentity> for crate::proto::ResolvedAgentIdentity { + fn from(identity: &ResolvedAgentIdentity) -> Self { + Self { + version: RESOLVED_AGENT_IDENTITY_VERSION, + source: identity.source.to_string(), + image_id: identity.image_id.clone().unwrap_or_default(), + uid: identity.uid, + gid: identity.gid, + supplementary_gids: Vec::new(), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn resolved_identity_proto_conversion_preserves_authoritative_fields() { + let identity = ResolvedAgentIdentity { + uid: 1234, + gid: 2345, + presentation_user: "agent".to_string(), + source: IdentitySource::Image, + image_id: Some("sha256:image".to_string()), + }; + + let proto = crate::proto::ResolvedAgentIdentity::from(&identity); + assert_eq!(proto.version, RESOLVED_AGENT_IDENTITY_VERSION); + assert_eq!(proto.source, "image"); + assert_eq!(proto.image_id, "sha256:image"); + assert_eq!(proto.uid, 1234); + assert_eq!(proto.gid, 2345); + assert!(proto.supplementary_gids.is_empty()); + } + + #[test] + fn fixed_identity_proto_conversion_preserves_image_id() { + let identity = ResolvedAgentIdentity { + uid: 1234, + gid: 2345, + presentation_user: "1234".to_string(), + source: IdentitySource::Fixed, + image_id: Some("sha256:fixed-image".to_string()), + }; + + let proto = crate::proto::ResolvedAgentIdentity::from(&identity); + assert_eq!(proto.source, "fixed"); + assert_eq!(proto.image_id, "sha256:fixed-image"); + } +} + // The corporate upstream-proxy configuration deliberately has no reserved // environment variables: it travels on the supervisor's argv // (`--upstream-proxy` and friends), which a sandbox image cannot forge the diff --git a/crates/openshell-driver-docker/README.md b/crates/openshell-driver-docker/README.md index e17791e747..252b15937b 100644 --- a/crates/openshell-driver-docker/README.md +++ b/crates/openshell-driver-docker/README.md @@ -42,6 +42,39 @@ contract: The agent child process does not retain these supervisor privileges. +## Agent Identity + +`[openshell.drivers.docker].identity_source` selects the identity for agent +children. It accepts `image` or `fixed` and defaults to `image`. Image mode +requires the sandbox image to declare a non-root OCI `USER`; there is no +implicit `10001:10001` fallback. Fixed mode requires both `fixed_uid` and +`fixed_gid`, each in the inclusive range `1000` through `2000000000`. Fixed +fields are invalid in image mode. + +The driver applies the image pull policy, inspects the final image, and carries +its raw OCI `Config.User` and immutable image ID into provisioning. The +container uses the immutable ID as its rootfs image while `user = "0"` keeps +the supervisor privileged. Only agent children drop to the resolved identity. + +Image identity accepts named, numeric, and mixed `user:group` OCI forms. A +named user or group must resolve uniquely from the image's regular +`/etc/passwd` or `/etc/group` file. A numeric UID without a group uses the +matching passwd entry's primary GID. A numeric pair such as `USER 1234:1235` +requires no account entries. OpenShell rejects a missing `USER`, an unknown or +ambiguous name, an accountless numeric UID without a group, and any declaration +that resolves to UID or GID 0. + +Agent children receive `HOME=/sandbox`, no supplementary groups, and the +declared user name in `USER` and `LOGNAME` when one is available. Numeric and +fixed identities use the numeric UID for presentation. OpenShell does not +modify `/etc/passwd` or `/etc/group`. + +The supervisor persists the resolved source, immutable image ID, UID, GID, and +empty supplementary group list. The gateway exposes this record in sandbox +status, and the supervisor emits it as an OCSF configuration-state event. +Restarts reuse the persisted identity instead of resolving a mutable tag or +changed account file again. + ## Driver Config Mounts The gateway forwards the `docker` block from `--driver-config-json` to this @@ -49,11 +82,12 @@ driver. The driver accepts user-supplied `mounts` entries with these Docker mount types: - `bind`: mounts an absolute host path when `[openshell.drivers.docker]` - has `enable_bind_mounts = true`. + has `enable_bind_mounts = true`. It also requires fixed identity mode. - `volume`: mounts an existing Docker named volume. The driver validates that the volume exists before provisioning and never creates or removes it. Docker local-driver volumes created with bind options are treated as host - bind mounts and require `enable_bind_mounts = true`. + bind mounts and require `enable_bind_mounts = true`. All creator-selected + named volumes require fixed identity mode. - `tmpfs`: mounts an in-memory filesystem with optional `options`, `size_bytes`, and `mode`. @@ -62,6 +96,11 @@ paths to sandbox requests. Image mounts are not part of the Docker driver-config schema. The driver still uses internal bind mounts for OpenShell-owned supervisor, token, and TLS material. +Image identity mode rejects creator-selected bind and named-volume mounts, +even when `enable_bind_mounts = true`. It permits `tmpfs` and the driver-owned +per-sandbox workspace volume. Use fixed mode for external or shared storage +that expects an operator-controlled UID and GID. + Docker `bind` mounts accept `source`, `target`, optional `read_only`, and an optional `selinux_label` of `shared` (applies `:z`) or `private` (applies `:Z`) for SELinux-enforcing hosts. Docker `volume` mounts may include @@ -74,6 +113,13 @@ or `/run/netns`. Example named-volume usage: +```toml +[openshell.drivers.docker] +identity_source = "fixed" +fixed_uid = 1000 +fixed_gid = 1000 +``` + ```shell docker volume create openshell-work diff --git a/crates/openshell-driver-docker/src/lib.rs b/crates/openshell-driver-docker/src/lib.rs index 2f89c2229e..7201a9996c 100644 --- a/crates/openshell-driver-docker/src/lib.rs +++ b/crates/openshell-driver-docker/src/lib.rs @@ -9,9 +9,9 @@ use bollard::Docker; use bollard::errors::Error as BollardError; use bollard::models::{ ContainerCreateBody, ContainerSummary, ContainerSummaryStateEnum, CreateImageInfo, - DeviceRequest, EndpointSettings, HostConfig, Mount, MountTmpfsOptions, MountTypeEnum, - MountVolumeOptions, NetworkCreateRequest, NetworkingConfig, ProgressDetail, RestartPolicy, - RestartPolicyNameEnum, SystemInfo, + DeviceRequest, EndpointSettings, HostConfig, ImageInspect, Mount, MountTmpfsOptions, + MountTypeEnum, MountVolumeOptions, NetworkCreateRequest, NetworkingConfig, ProgressDetail, + RestartPolicy, RestartPolicyNameEnum, SystemInfo, }; use bollard::query_parameters::{ CreateContainerOptionsBuilder, CreateImageOptions, DownloadFromContainerOptionsBuilder, @@ -49,6 +49,7 @@ use openshell_core::proto::compute::v1::{ use openshell_core::proto_struct::{ deserialize_optional_non_empty_string_list, struct_to_json_value, }; +use openshell_core::sandbox_env::IdentitySource; use openshell_core::{Config, Error, Result as CoreResult}; use std::collections::{HashMap, HashSet}; use std::io::Read; @@ -78,6 +79,10 @@ const SUPERVISOR_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin const HOST_OPENSHELL_INTERNAL: &str = "host.openshell.internal"; const HOST_DOCKER_INTERNAL: &str = "host.docker.internal"; const DOCKER_NETWORK_DRIVER: &str = "bridge"; +// Keep fixed local-driver identities aligned with the process policy's numeric +// identity range without adding a policy-engine dependency to the driver. +const MIN_FIXED_ID: u32 = 1_000; +const MAX_FIXED_ID: u32 = 2_000_000_000; /// Queried by the Docker driver to decide when a sandbox's supervisor /// relay is live. Implementations return `true` once a sandbox has an @@ -146,6 +151,15 @@ pub struct DockerComputeConfig { /// `template.driver_config`. #[serde(default)] pub enable_bind_mounts: bool, + + /// Source used to select the agent process identity. + pub identity_source: IdentitySource, + + /// Operator-selected UID used when `identity_source = "fixed"`. + pub fixed_uid: Option, + + /// Operator-selected GID used when `identity_source = "fixed"`. + pub fixed_gid: Option, } impl Default for DockerComputeConfig { @@ -166,10 +180,52 @@ impl Default for DockerComputeConfig { ssh_socket_path: "/run/openshell/ssh.sock".to_string(), sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, enable_bind_mounts: false, + identity_source: IdentitySource::Image, + fixed_uid: None, + fixed_gid: None, } } } +fn validate_docker_identity_config( + config: &DockerComputeConfig, +) -> CoreResult { + match config.identity_source { + IdentitySource::Image => { + if config.fixed_uid.is_some() || config.fixed_gid.is_some() { + return Err(Error::config( + "docker fixed_uid and fixed_gid must be omitted when identity_source = 'image'", + )); + } + } + IdentitySource::Fixed => { + let uid = config.fixed_uid.ok_or_else(|| { + Error::config("docker fixed_uid is required when identity_source = 'fixed'") + })?; + let gid = config.fixed_gid.ok_or_else(|| { + Error::config("docker fixed_gid is required when identity_source = 'fixed'") + })?; + validate_fixed_id("fixed_uid", uid)?; + validate_fixed_id("fixed_gid", gid)?; + } + } + + Ok(DockerIdentityConfig { + source: config.identity_source, + fixed_uid: config.fixed_uid, + fixed_gid: config.fixed_gid, + }) +} + +fn validate_fixed_id(field: &str, id: u32) -> CoreResult<()> { + if !(MIN_FIXED_ID..=MAX_FIXED_ID).contains(&id) { + return Err(Error::config(format!( + "docker {field} must be in policy range [{MIN_FIXED_ID}, {MAX_FIXED_ID}], got {id}" + ))); + } + Ok(()) +} + #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct DockerGuestTlsPaths { pub(crate) ca: PathBuf, @@ -195,6 +251,20 @@ struct DockerDriverRuntimeConfig { allow_all_default_gpu: bool, sandbox_pids_limit: i64, enable_bind_mounts: bool, + identity: DockerIdentityConfig, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct DockerIdentityConfig { + source: IdentitySource, + fixed_uid: Option, + fixed_gid: Option, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct DockerImageMetadata { + id: String, + user: Option, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -314,6 +384,9 @@ impl DockerComputeDriver { docker_config: &DockerComputeConfig, supervisor_readiness: Arc, ) -> CoreResult { + // Reject invalid identity settings before connecting to Docker or + // creating driver-managed networks and cache files. + let identity = validate_docker_identity_config(docker_config)?; let socket_path = docker_config .socket_path .clone() @@ -392,6 +465,7 @@ impl DockerComputeDriver { allow_all_default_gpu, sandbox_pids_limit: docker_config.sandbox_pids_limit, enable_bind_mounts: docker_config.enable_bind_mounts, + identity, }, events: broadcast::channel(WATCH_BUFFER).0, pending: Arc::new(Mutex::new(HashMap::new())), @@ -452,7 +526,11 @@ impl DockerComputeDriver { let _ = docker_resource_limits(template)?; let driver_config = DockerSandboxDriverConfig::from_template(template).map_err(Status::invalid_argument)?; - validate_docker_driver_mounts(&driver_config.mounts, config.enable_bind_mounts)?; + validate_docker_driver_mounts( + &driver_config.mounts, + config.enable_bind_mounts, + config.identity.source, + )?; let gpu_requirements = driver_gpu_requirements(spec.resource_requirements.as_ref()); Self::validate_gpu_request(gpu_requirements, config.supports_gpu, &driver_config)?; Ok(ValidatedDockerSandbox { @@ -635,21 +713,16 @@ impl DockerComputeDriver { async fn create_sandbox_inner(&self, sandbox: &DriverSandbox) -> Result<(), Status> { let validated = Self::validated_sandbox(sandbox, &self.config)?; Self::validate_sandbox_auth(sandbox)?; + preflight_container_create_body(sandbox, &self.config, &validated.driver_config)?; self.validate_user_volume_mounts_available(&validated.driver_config) .await?; - let gpu_devices = self + let _ = self .resolve_gpu_cdi_devices( validated.gpu_requirements, &validated.driver_config, CdiGpuDefaultSelector::peek_device_ids, ) .await?; - let _ = build_container_create_body_with_gpu_devices( - sandbox, - &self.config, - &validated.driver_config, - gpu_devices.as_deref(), - )?; if self .find_managed_container_summary(&sandbox.id, &sandbox.name) @@ -710,7 +783,8 @@ impl DockerComputeDriver { DockerProvisioningFailure::new("ContainerCreateFailed", status.message()) })?; let template = validated.template; - self.ensure_image_available(&sandbox.id, &template.image) + let image = self + .ensure_image_available(&sandbox.id, &template.image) .await .map_err(|status| { DockerProvisioningFailure::new("ImagePullFailed", status.message()) @@ -739,6 +813,7 @@ impl DockerComputeDriver { sandbox, &self.config, &validated.driver_config, + &image, gpu_devices.as_deref(), ) .map_err(|status| { @@ -1280,31 +1355,44 @@ impl DockerComputeDriver { })) } - async fn ensure_image_available(&self, sandbox_id: &str, image: &str) -> Result<(), Status> { + async fn ensure_image_available( + &self, + sandbox_id: &str, + image: &str, + ) -> Result { let policy = self.config.image_pull_policy.trim().to_ascii_lowercase(); match policy.as_str() { - "" | "ifnotpresent" => { - if self.docker.inspect_image(image).await.is_ok() { + "" | "ifnotpresent" => match self.docker.inspect_image(image).await { + Ok(inspect) => { + let metadata = docker_image_metadata(image, inspect, self.config.identity)?; self.publish_docker_progress( sandbox_id, "ImagePresent", format!("Docker image \"{image}\" is already present"), HashMap::from([("image_ref".to_string(), image.to_string())]), ); - return Ok(()); + Ok(metadata) + } + Err(err) if is_not_found_error(&err) => { + self.pull_image(sandbox_id, image).await?; + self.inspect_final_image(image).await } - self.pull_image(sandbox_id, image).await + Err(err) => Err(internal_status("inspect Docker image", err)), + }, + "always" => { + self.pull_image(sandbox_id, image).await?; + self.inspect_final_image(image).await } - "always" => self.pull_image(sandbox_id, image).await, "never" => match self.docker.inspect_image(image).await { - Ok(_) => { + Ok(inspect) => { + let metadata = docker_image_metadata(image, inspect, self.config.identity)?; self.publish_docker_progress( sandbox_id, "ImagePresent", format!("Docker image \"{image}\" is already present"), HashMap::from([("image_ref".to_string(), image.to_string())]), ); - Ok(()) + Ok(metadata) } Err(err) if is_not_found_error(&err) => Err(Status::failed_precondition(format!( "docker image '{image}' is not present locally and image_pull_policy=Never" @@ -1317,6 +1405,15 @@ impl DockerComputeDriver { } } + async fn inspect_final_image(&self, image: &str) -> Result { + let inspect = self + .docker + .inspect_image(image) + .await + .map_err(|err| internal_status("inspect Docker image after pull", err))?; + docker_image_metadata(image, inspect, self.config.identity) + } + async fn pull_image(&self, sandbox_id: &str, image: &str) -> Result<(), Status> { self.publish_docker_progress( sandbox_id, @@ -1528,6 +1625,50 @@ impl DockerProvisioningFailure { } } +fn docker_image_metadata( + image: &str, + inspect: ImageInspect, + identity: DockerIdentityConfig, +) -> Result { + let id = inspect + .id + .filter(|id| !id.trim().is_empty()) + .ok_or_else(|| { + Status::failed_precondition(format!( + "docker image '{image}' inspection returned no immutable image ID" + )) + })?; + let metadata = DockerImageMetadata { + id, + user: inspect.config.and_then(|config| config.user), + }; + validate_docker_image_metadata(image, &metadata, identity)?; + Ok(metadata) +} + +fn validate_docker_image_metadata( + image: &str, + metadata: &DockerImageMetadata, + identity: DockerIdentityConfig, +) -> Result<(), Status> { + if metadata.id.trim().is_empty() { + return Err(Status::failed_precondition(format!( + "docker image '{image}' inspection returned no immutable image ID" + ))); + } + if identity.source == IdentitySource::Image + && metadata + .user + .as_ref() + .is_none_or(|user| user.trim().is_empty()) + { + return Err(Status::failed_precondition(format!( + "docker image '{image}' must declare a non-empty OCI Config.User when identity_source = 'image'" + ))); + } + Ok(()) +} + fn sandbox_image(sandbox: &DriverSandbox) -> Option { sandbox .spec @@ -1730,12 +1871,16 @@ fn attach_docker_progress_metadata( #[cfg(test)] fn docker_driver_config( template: &DriverSandboxTemplate, - enable_bind_mounts: bool, + config: &DockerDriverRuntimeConfig, ) -> Result { - let config = + let driver_config = DockerSandboxDriverConfig::from_template(template).map_err(Status::invalid_argument)?; - validate_docker_driver_mounts(&config.mounts, enable_bind_mounts)?; - Ok(config) + validate_docker_driver_mounts( + &driver_config.mounts, + config.enable_bind_mounts, + config.identity.source, + )?; + Ok(driver_config) } /// Collect user-supplied bind mounts as string-format binds. @@ -1866,11 +2011,17 @@ fn docker_mount_from_config(config: &DockerDriverMountConfig) -> Result Result<(), Status> { let mut targets = HashSet::new(); for mount in mounts { let target = match mount { DockerDriverMountConfig::Bind { source, target, .. } => { + if identity_source == IdentitySource::Image { + return Err(Status::failed_precondition( + "docker bind mounts are not allowed when identity_source = 'image'", + )); + } if !enable_bind_mounts { return Err(Status::failed_precondition( "docker bind mounts require enable_bind_mounts = true in [openshell.drivers.docker]", @@ -1886,6 +2037,11 @@ fn validate_docker_driver_mounts( subpath, .. } => { + if identity_source == IdentitySource::Image { + return Err(Status::failed_precondition( + "docker named volume mounts are not allowed when identity_source = 'image'", + )); + } driver_mounts::validate_mount_source(source, "volume source") .map_err(Status::failed_precondition)?; if let Some(subpath) = subpath { @@ -2132,7 +2288,11 @@ fn cleanup_sandbox_token_file_by_id(sandbox_id: &str, config: &DockerDriverRunti } } -fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig) -> Vec { +fn build_environment( + sandbox: &DriverSandbox, + config: &DockerDriverRuntimeConfig, + image: &DockerImageMetadata, +) -> Vec { let mut environment = HashMap::from([ ("HOME".to_string(), "/root".to_string()), ("PATH".to_string(), SUPERVISOR_PATH.to_string()), @@ -2149,6 +2309,7 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig user_env.extend(template.environment.clone()); } user_env.extend(spec.environment.clone()); + user_env.retain(|key, _| !is_protected_identity_environment(key)); environment.extend(user_env.clone()); if !user_env.is_empty() && let Ok(json) = serde_json::to_string(&user_env) @@ -2204,6 +2365,7 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig environment.remove(openshell_core::sandbox_env::SANDBOX_TOKEN); environment.remove(openshell_core::sandbox_env::SANDBOX_TOKEN_FILE); + environment.retain(|key, _| !is_protected_identity_environment(key)); // Gateway-minted sandbox JWT. Keep the raw bearer out of container // metadata; the supervisor reads it from this driver-owned bind mount. @@ -2218,10 +2380,70 @@ fn build_environment(sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig let mut pairs = environment.into_iter().collect::>(); pairs.sort_by(|left, right| left.0.cmp(&right.0)); - pairs + let mut result = pairs .into_iter() .map(|(key, value)| format!("{key}={value}")) - .collect() + .collect::>(); + result.extend(protected_identity_environment(config.identity, image)); + result +} + +fn is_protected_identity_environment(key: &str) -> bool { + matches!( + key, + openshell_core::sandbox_env::IDENTITY_SOURCE + | openshell_core::sandbox_env::IMAGE_USER + | openshell_core::sandbox_env::IMAGE_ID + | openshell_core::sandbox_env::SANDBOX_UID + | openshell_core::sandbox_env::SANDBOX_GID + ) +} + +fn protected_identity_environment( + identity: DockerIdentityConfig, + image: &DockerImageMetadata, +) -> Vec { + let image_user = image.user.as_ref().map_or_else( + || openshell_core::sandbox_env::IMAGE_USER.to_string(), + |user| format!("{}={user}", openshell_core::sandbox_env::IMAGE_USER), + ); + let mut environment = vec![ + format!( + "{}={}", + openshell_core::sandbox_env::IDENTITY_SOURCE, + identity.source + ), + image_user, + // Both identity modes bind persisted identity state to the immutable + // rootfs selected for this container. + format!("{}={}", openshell_core::sandbox_env::IMAGE_ID, image.id), + ]; + + match identity.source { + IdentitySource::Image => { + // Bare Docker environment names remove any values inherited from + // the image while preserving the distinction from an empty value. + environment.push(openshell_core::sandbox_env::SANDBOX_UID.to_string()); + environment.push(openshell_core::sandbox_env::SANDBOX_GID.to_string()); + } + IdentitySource::Fixed => { + let uid = identity + .fixed_uid + .expect("validated fixed Docker identity must include a UID"); + let gid = identity + .fixed_gid + .expect("validated fixed Docker identity must include a GID"); + environment.push(format!( + "{}={uid}", + openshell_core::sandbox_env::SANDBOX_UID + )); + environment.push(format!( + "{}={gid}", + openshell_core::sandbox_env::SANDBOX_GID + )); + } + } + environment } fn docker_cdi_gpu_inventory(info: &SystemInfo) -> CdiGpuInventory { @@ -2264,7 +2486,7 @@ fn build_container_create_body( .as_ref() .and_then(|spec| spec.template.as_ref()) .ok_or_else(|| Status::invalid_argument("sandbox.spec.template is required"))?; - let driver_config = docker_driver_config(template, config.enable_bind_mounts)?; + let driver_config = docker_driver_config(template, config)?; let gpu_requirements = sandbox .spec .as_ref() @@ -2280,13 +2502,42 @@ fn build_container_create_body( } else { None }; - build_container_create_body_with_gpu_devices(sandbox, config, &driver_config, cdi_devices) + let image = DockerImageMetadata { + id: "sha256:test-image".to_string(), + user: Some("sandbox".to_string()), + }; + build_container_create_body_with_gpu_devices( + sandbox, + config, + &driver_config, + &image, + cdi_devices, + ) +} + +fn preflight_container_create_body( + sandbox: &DriverSandbox, + config: &DockerDriverRuntimeConfig, + driver_config: &DockerSandboxDriverConfig, +) -> Result<(), Status> { + let template = sandbox + .spec + .as_ref() + .and_then(|spec| spec.template.as_ref()) + .ok_or_else(|| Status::invalid_argument("sandbox.spec.template is required"))?; + let _ = docker_resource_limits(template)?; + let _ = docker_driver_mounts(driver_config)?; + let _ = docker_driver_bind_strings(driver_config)?; + let _ = build_binds(sandbox, config)?; + let _ = docker_pids_limit(config.sandbox_pids_limit)?; + Ok(()) } fn build_container_create_body_with_gpu_devices( sandbox: &DriverSandbox, config: &DockerDriverRuntimeConfig, driver_config: &DockerSandboxDriverConfig, + image: &DockerImageMetadata, gpu_device_ids: Option<&[String]>, ) -> Result { let spec = sandbox @@ -2297,6 +2548,8 @@ fn build_container_create_body_with_gpu_devices( .template .as_ref() .ok_or_else(|| Status::invalid_argument("sandbox.spec.template is required"))?; + preflight_container_create_body(sandbox, config, driver_config)?; + validate_docker_image_metadata("selected Docker image", image, config.identity)?; let resource_limits = docker_resource_limits(template)?; let user_mounts = docker_driver_mounts(driver_config)?; let user_bind_strings = docker_driver_bind_strings(driver_config)?; @@ -2328,9 +2581,9 @@ fn build_container_create_body_with_gpu_devices( ); Ok(ContainerCreateBody { - image: Some(template.image.clone()), + image: Some(image.id.clone()), user: Some("0".to_string()), - env: Some(build_environment(sandbox, config)), + env: Some(build_environment(sandbox, config, image)), entrypoint: Some(vec![SUPERVISOR_MOUNT_PATH.to_string()]), // Clear the image CMD so Docker does not append inherited args to the // supervisor entrypoint. diff --git a/crates/openshell-driver-docker/src/tests.rs b/crates/openshell-driver-docker/src/tests.rs index 67036b2192..06684bf159 100644 --- a/crates/openshell-driver-docker/src/tests.rs +++ b/crates/openshell-driver-docker/src/tests.rs @@ -115,9 +115,50 @@ fn runtime_config() -> DockerDriverRuntimeConfig { allow_all_default_gpu: false, sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, enable_bind_mounts: false, + identity: DockerIdentityConfig { + source: IdentitySource::Image, + fixed_uid: None, + fixed_gid: None, + }, + } +} + +fn fixed_runtime_config() -> DockerDriverRuntimeConfig { + let mut config = runtime_config(); + config.identity = DockerIdentityConfig { + source: IdentitySource::Fixed, + fixed_uid: Some(1_234), + fixed_gid: Some(1_235), + }; + config +} + +fn image_metadata() -> DockerImageMetadata { + DockerImageMetadata { + id: "sha256:immutable-image-id".to_string(), + user: Some("app:staff".to_string()), + } +} + +fn image_inspect(id: Option<&str>, user: Option<&str>) -> ImageInspect { + ImageInspect { + id: id.map(str::to_string), + config: Some(bollard::models::ImageConfig { + user: user.map(str::to_string), + ..Default::default() + }), + ..Default::default() } } +fn environment_values<'a>(environment: &'a [String], key: &str) -> Vec> { + environment + .iter() + .filter(|entry| entry.split_once('=').map_or(entry.as_str(), |pair| pair.0) == key) + .map(|entry| entry.split_once('=').map(|pair| pair.1)) + .collect() +} + fn json_struct(value: serde_json::Value) -> prost_types::Struct { let serde_json::Value::Object(object) = value else { panic!("expected JSON object"); @@ -521,6 +562,173 @@ fn docker_pids_limit_uses_driver_default_and_allows_runtime_inherit() { fn docker_compute_config_disables_bind_mounts_by_default() { let cfg = DockerComputeConfig::default(); assert!(!cfg.enable_bind_mounts); + assert_eq!(cfg.identity_source, IdentitySource::Image); + assert_eq!(cfg.fixed_uid, None); + assert_eq!(cfg.fixed_gid, None); +} + +#[test] +fn docker_identity_config_accepts_image_and_complete_fixed_modes() { + let image = DockerComputeConfig::default(); + assert_eq!( + validate_docker_identity_config(&image).unwrap(), + DockerIdentityConfig { + source: IdentitySource::Image, + fixed_uid: None, + fixed_gid: None, + } + ); + + let fixed = DockerComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid: Some(MIN_FIXED_ID), + fixed_gid: Some(MAX_FIXED_ID), + ..Default::default() + }; + assert_eq!( + validate_docker_identity_config(&fixed).unwrap(), + DockerIdentityConfig { + source: IdentitySource::Fixed, + fixed_uid: Some(MIN_FIXED_ID), + fixed_gid: Some(MAX_FIXED_ID), + } + ); +} + +#[test] +fn docker_identity_config_rejects_fixed_fields_in_image_mode() { + for (fixed_uid, fixed_gid) in [(Some(1_000), None), (None, Some(1_000))] { + let config = DockerComputeConfig { + fixed_uid, + fixed_gid, + ..Default::default() + }; + let err = validate_docker_identity_config(&config).unwrap_err(); + assert!(err.to_string().contains("must be omitted")); + } +} + +#[test] +fn docker_identity_config_requires_both_fixed_ids() { + for (fixed_uid, fixed_gid, missing) in [ + (None, Some(1_000), "fixed_uid"), + (Some(1_000), None, "fixed_gid"), + ] { + let config = DockerComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid, + fixed_gid, + ..Default::default() + }; + let err = validate_docker_identity_config(&config).unwrap_err(); + assert!(err.to_string().contains(missing)); + assert!(err.to_string().contains("required")); + } +} + +#[test] +fn docker_identity_config_rejects_ids_outside_policy_range() { + for (fixed_uid, fixed_gid, invalid) in [ + (0, MIN_FIXED_ID, 0), + (MIN_FIXED_ID - 1, MIN_FIXED_ID, MIN_FIXED_ID - 1), + (MIN_FIXED_ID, MAX_FIXED_ID + 1, MAX_FIXED_ID + 1), + ] { + let config = DockerComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid: Some(fixed_uid), + fixed_gid: Some(fixed_gid), + ..Default::default() + }; + let err = validate_docker_identity_config(&config).unwrap_err(); + assert!(err.to_string().contains("policy range")); + assert!(err.to_string().contains(&invalid.to_string())); + } +} + +#[test] +fn docker_compute_config_serde_defaults_identity_and_denies_unknown_fields() { + let defaulted: DockerComputeConfig = serde_json::from_value(serde_json::json!({})).unwrap(); + assert_eq!(defaulted.identity_source, IdentitySource::Image); + assert_eq!(defaulted.fixed_uid, None); + assert_eq!(defaulted.fixed_gid, None); + + let fixed: DockerComputeConfig = serde_json::from_value(serde_json::json!({ + "identity_source": "fixed", + "fixed_uid": 1234, + "fixed_gid": 1235 + })) + .unwrap(); + validate_docker_identity_config(&fixed).unwrap(); + + let err = serde_json::from_value::(serde_json::json!({ + "identity_source": "image", + "sandbox_uid": 1234 + })) + .unwrap_err(); + assert!(err.to_string().contains("unknown field")); +} + +#[test] +fn docker_image_metadata_preserves_immutable_id_and_raw_user() { + let metadata = docker_image_metadata( + "example:latest", + image_inspect(Some("sha256:abc123"), Some(" app:staff ")), + runtime_config().identity, + ) + .unwrap(); + + assert_eq!(metadata.id, "sha256:abc123"); + assert_eq!(metadata.user.as_deref(), Some(" app:staff ")); +} + +#[test] +fn docker_image_metadata_requires_nonempty_immutable_id() { + for id in [None, Some(""), Some(" ")] { + let err = docker_image_metadata( + "example:latest", + image_inspect(id, Some("app")), + runtime_config().identity, + ) + .unwrap_err(); + assert!(err.message().contains("immutable image ID")); + } +} + +#[test] +fn docker_image_metadata_requires_user_only_in_image_mode() { + for user in [None, Some(""), Some(" ")] { + let err = docker_image_metadata( + "example:latest", + image_inspect(Some("sha256:abc123"), user), + runtime_config().identity, + ) + .unwrap_err(); + assert!(err.message().contains("Config.User")); + + let metadata = docker_image_metadata( + "example:latest", + image_inspect(Some("sha256:abc123"), user), + fixed_runtime_config().identity, + ) + .unwrap(); + assert_eq!(metadata.user.as_deref(), user); + } +} + +#[test] +fn docker_not_found_classification_does_not_treat_other_inspect_errors_as_missing() { + let not_found = BollardError::DockerResponseServerError { + status_code: 404, + message: "missing".to_string(), + }; + let server_error = BollardError::DockerResponseServerError { + status_code: 500, + message: "daemon failed".to_string(), + }; + + assert!(is_not_found_error(¬_found)); + assert!(!is_not_found_error(&server_error)); + assert!(!is_not_found_error(&BollardError::RequestTimeoutError)); } #[test] @@ -532,7 +740,7 @@ fn container_create_body_sets_driver_owned_pids_limit() { #[test] fn build_environment_sets_docker_tls_paths() { - let env = build_environment(&test_sandbox(), &runtime_config()); + let env = build_environment(&test_sandbox(), &runtime_config(), &image_metadata()); assert!(env.contains(&format!("OPENSHELL_TLS_CA={TLS_CA_MOUNT_PATH}"))); assert!(env.contains(&format!("OPENSHELL_TLS_CERT={TLS_CERT_MOUNT_PATH}"))); assert!(env.contains(&format!("OPENSHELL_TLS_KEY={TLS_KEY_MOUNT_PATH}"))); @@ -553,7 +761,7 @@ fn build_environment_keeps_path_driver_controlled() { .environment .insert("PATH".to_string(), "/malicious/template/bin".to_string()); - let env = build_environment(&sandbox, &runtime_config()); + let env = build_environment(&sandbox, &runtime_config(), &image_metadata()); let path_entries = env .iter() .filter(|entry| entry.starts_with("PATH=")) @@ -579,7 +787,7 @@ fn build_environment_keeps_telemetry_toggle_driver_controlled() { "true".to_string(), ); - let env = build_environment(&sandbox, &runtime_config()); + let env = build_environment(&sandbox, &runtime_config(), &image_metadata()); let telemetry_entries = env .iter() .filter(|entry| { @@ -599,6 +807,198 @@ fn build_environment_keeps_telemetry_toggle_driver_controlled() { ); } +#[test] +fn image_baked_and_request_identity_cannot_override_protected_metadata() { + let protected = [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, + ]; + let mut sandbox = test_sandbox(); + let spec = sandbox.spec.as_mut().unwrap(); + for key in protected { + spec.template + .as_mut() + .unwrap() + .environment + .insert(key.to_string(), "template-spoof".to_string()); + spec.environment + .insert(key.to_string(), "request-spoof".to_string()); + } + let mut inspect = image_inspect(Some("sha256:authoritative-image"), Some("image-app")); + inspect.config.as_mut().unwrap().env = Some( + protected + .iter() + .map(|key| format!("{key}=image-baked-spoof")) + .collect(), + ); + let image = + docker_image_metadata("example:latest", inspect, runtime_config().identity).unwrap(); + let body = build_container_create_body_with_gpu_devices( + &sandbox, + &runtime_config(), + &DockerSandboxDriverConfig::default(), + &image, + None, + ) + .unwrap(); + let env = body.env.expect("final container environment"); + + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IDENTITY_SOURCE), + vec![Some("image")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IMAGE_USER), + vec![Some("image-app")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IMAGE_ID), + vec![Some("sha256:authoritative-image")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_UID), + vec![None] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_GID), + vec![None] + ); + + let user_environment = env + .iter() + .find_map(|entry| { + entry + .strip_prefix(&format!( + "{}=", + openshell_core::sandbox_env::USER_ENVIRONMENT + )) + .map(str::to_string) + }) + .expect("serialized user environment"); + let user_environment: HashMap = + serde_json::from_str(&user_environment).unwrap(); + assert!( + protected + .iter() + .all(|key| !user_environment.contains_key(*key)) + ); +} + +#[test] +fn fixed_identity_environment_injects_one_authoritative_image_id_and_numeric_ids() { + let metadata = DockerImageMetadata { + id: "sha256:fixed-image".to_string(), + user: None, + }; + let mut sandbox = test_sandbox(); + let spec = sandbox.spec.as_mut().unwrap(); + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, + ] { + spec.template + .as_mut() + .unwrap() + .environment + .insert(key.to_string(), "template-spoof".to_string()); + spec.environment + .insert(key.to_string(), "request-spoof".to_string()); + } + let env = build_environment(&sandbox, &fixed_runtime_config(), &metadata); + + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IDENTITY_SOURCE), + vec![Some("fixed")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IMAGE_USER), + vec![None] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IMAGE_ID), + vec![Some("sha256:fixed-image")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_UID), + vec![Some("1234")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_GID), + vec![Some("1235")] + ); +} + +#[test] +fn final_container_body_uses_immutable_image_id_and_root_supervisor() { + let sandbox = test_sandbox(); + let config = runtime_config(); + let driver_config = DockerSandboxDriverConfig::default(); + let image = image_metadata(); + + preflight_container_create_body(&sandbox, &config, &driver_config).unwrap(); + let body = build_container_create_body_with_gpu_devices( + &sandbox, + &config, + &driver_config, + &image, + None, + ) + .unwrap(); + + assert_eq!(body.image.as_deref(), Some("sha256:immutable-image-id")); + assert_eq!(body.user.as_deref(), Some("0")); + assert_ne!( + body.image.as_deref(), + sandbox + .spec + .as_ref() + .and_then(|spec| spec.template.as_ref()) + .map(|template| template.image.as_str()) + ); +} + +#[test] +fn fixed_identity_final_body_pins_image_and_carries_id_for_persistence() { + let image = DockerImageMetadata { + id: "sha256:fixed-image".to_string(), + user: None, + }; + let body = build_container_create_body_with_gpu_devices( + &test_sandbox(), + &fixed_runtime_config(), + &DockerSandboxDriverConfig::default(), + &image, + None, + ) + .unwrap(); + + assert_eq!(body.image.as_deref(), Some("sha256:fixed-image")); + assert_eq!(body.user.as_deref(), Some("0")); + let env = body.env.expect("final container environment"); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IDENTITY_SOURCE), + vec![Some("fixed")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::IMAGE_ID), + vec![Some("sha256:fixed-image")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_UID), + vec![Some("1234")] + ); + assert_eq!( + environment_values(&env, openshell_core::sandbox_env::SANDBOX_GID), + vec![Some("1235")] + ); +} + #[test] fn build_binds_uses_docker_tls_directory() { let binds = build_binds(&test_sandbox(), &runtime_config()).unwrap(); @@ -640,7 +1040,7 @@ fn build_container_create_body_includes_driver_config_mounts() { ] }))); - let body = build_container_create_body(&sandbox, &runtime_config()).unwrap(); + let body = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap(); let mounts = body .host_config .unwrap() @@ -688,7 +1088,7 @@ fn driver_config_defaults_volume_mounts_to_read_only() { }] }))); - let body = build_container_create_body(&sandbox, &runtime_config()).unwrap(); + let body = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap(); let mounts = body .host_config .unwrap() @@ -717,7 +1117,7 @@ fn driver_config_allows_explicit_writable_volume_mounts() { }] }))); - let body = build_container_create_body(&sandbox, &runtime_config()).unwrap(); + let body = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap(); let mounts = body .host_config .unwrap() @@ -751,7 +1151,7 @@ fn driver_config_rejects_duplicate_mount_targets() { ] }))); - let err = build_container_create_body(&sandbox, &runtime_config()).unwrap_err(); + let err = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap_err(); assert_eq!(err.code(), tonic::Code::FailedPrecondition); assert!( @@ -778,12 +1178,92 @@ fn driver_config_rejects_bind_mounts_unless_enabled() { }] }))); - let err = build_container_create_body(&sandbox, &runtime_config()).unwrap_err(); + let err = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap_err(); assert_eq!(err.code(), tonic::Code::FailedPrecondition); assert!(err.message().contains("enable_bind_mounts = true")); } +#[test] +fn image_identity_rejects_bind_mounts_even_when_globally_enabled() { + let bind_src = TempDir::new().unwrap(); + let mut sandbox = test_sandbox(); + sandbox + .spec + .as_mut() + .unwrap() + .template + .as_mut() + .unwrap() + .driver_config = Some(json_struct(serde_json::json!({ + "mounts": [{ + "type": "bind", + "source": bind_src.path().to_str().unwrap(), + "target": "/sandbox/host" + }] + }))); + let mut config = runtime_config(); + config.enable_bind_mounts = true; + + let err = build_container_create_body(&sandbox, &config).unwrap_err(); + assert!(err.message().contains("identity_source = 'image'")); +} + +#[test] +fn image_identity_rejects_named_volumes_but_permits_tmpfs() { + let mut volume_sandbox = test_sandbox(); + volume_sandbox + .spec + .as_mut() + .unwrap() + .template + .as_mut() + .unwrap() + .driver_config = Some(json_struct(serde_json::json!({ + "mounts": [{ + "type": "volume", + "source": "shared-data", + "target": "/sandbox/shared" + }] + }))); + let err = build_container_create_body(&volume_sandbox, &runtime_config()).unwrap_err(); + assert!(err.message().contains("named volume mounts")); + assert!(err.message().contains("identity_source = 'image'")); + + let mut tmpfs_sandbox = test_sandbox(); + tmpfs_sandbox + .spec + .as_mut() + .unwrap() + .template + .as_mut() + .unwrap() + .driver_config = Some(json_struct(serde_json::json!({ + "mounts": [{ + "type": "tmpfs", + "target": "/sandbox/cache" + }] + }))); + let body = build_container_create_body(&tmpfs_sandbox, &runtime_config()).unwrap(); + assert_eq!( + body.host_config + .as_ref() + .and_then(|host| host.mounts.as_ref()) + .and_then(|mounts| mounts.first()) + .and_then(|mount| mount.typ), + Some(MountTypeEnum::TMPFS) + ); + assert!( + body.host_config + .as_ref() + .and_then(|host| host.binds.as_ref()) + .is_some_and(|binds| binds + .iter() + .any(|bind| bind.contains(SUPERVISOR_MOUNT_PATH))), + "driver-owned supervisor bind must remain present in image mode" + ); +} + #[test] fn build_container_create_body_includes_bind_mounts_when_enabled() { let bind_src = TempDir::new().unwrap(); @@ -804,7 +1284,7 @@ fn build_container_create_body_includes_bind_mounts_when_enabled() { "read_only": true }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let body = build_container_create_body(&sandbox, &config).unwrap(); @@ -849,7 +1329,7 @@ fn driver_config_defaults_enabled_bind_mounts_to_read_only() { "target": "/sandbox/host" }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let body = build_container_create_body(&sandbox, &config).unwrap(); @@ -887,7 +1367,7 @@ fn bind_mount_selinux_shared_label() { "selinux_label": "shared" }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let body = build_container_create_body(&sandbox, &config).unwrap(); @@ -925,7 +1405,7 @@ fn bind_mount_selinux_private_label() { "selinux_label": "private" }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let body = build_container_create_body(&sandbox, &config).unwrap(); @@ -962,7 +1442,7 @@ fn bind_mount_without_selinux_label() { "read_only": false }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let body = build_container_create_body(&sandbox, &config).unwrap(); @@ -996,7 +1476,7 @@ fn driver_config_rejects_missing_bind_source() { "target": "/sandbox/data" }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let err = build_container_create_body(&sandbox, &config).unwrap_err(); @@ -1026,7 +1506,7 @@ fn driver_config_rejects_relative_bind_sources_when_enabled() { "target": "/sandbox/host" }] }))); - let mut config = runtime_config(); + let mut config = fixed_runtime_config(); config.enable_bind_mounts = true; let err = build_container_create_body(&sandbox, &config).unwrap_err(); @@ -1056,7 +1536,7 @@ fn driver_config_rejects_image_mounts() { }] }))); - let err = build_container_create_body(&sandbox, &runtime_config()).unwrap_err(); + let err = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap_err(); assert_eq!(err.code(), tonic::Code::FailedPrecondition); assert!(err.message().contains("invalid docker driver_config")); @@ -1080,7 +1560,7 @@ fn driver_config_rejects_reserved_mount_targets() { }] }))); - let err = build_container_create_body(&sandbox, &runtime_config()).unwrap_err(); + let err = build_container_create_body(&sandbox, &fixed_runtime_config()).unwrap_err(); assert_eq!(err.code(), tonic::Code::FailedPrecondition); assert!(err.message().contains("reserved OpenShell path")); @@ -1148,7 +1628,7 @@ fn build_environment_uses_token_file_without_raw_token_env() { "user-provided-token".to_string(), ); - let env = build_environment(&sandbox, &runtime_config()); + let env = build_environment(&sandbox, &runtime_config(), &image_metadata()); assert!(!env.iter().any(|entry| { entry.starts_with(&format!("{}=", openshell_core::sandbox_env::SANDBOX_TOKEN)) @@ -1422,6 +1902,7 @@ fn build_container_create_body_maps_default_gpu_to_selected_cdi_device() { &sandbox, &config, &driver_config, + &image_metadata(), Some(&gpu_devices), ) .unwrap(); @@ -1557,6 +2038,7 @@ fn driver_default_gpu_selection_consumes_distinct_devices_for_creates() { &first_sandbox, &driver.config, &driver_config, + &image_metadata(), Some(&first_devices), ) .unwrap(); @@ -1571,6 +2053,7 @@ fn driver_default_gpu_selection_consumes_distinct_devices_for_creates() { &second_sandbox, &driver.config, &driver_config, + &image_metadata(), Some(&second_devices), ) .unwrap(); diff --git a/crates/openshell-driver-kubernetes/src/driver.rs b/crates/openshell-driver-kubernetes/src/driver.rs index c784f10db9..4fc675e238 100644 --- a/crates/openshell-driver-kubernetes/src/driver.rs +++ b/crates/openshell-driver-kubernetes/src/driver.rs @@ -2889,10 +2889,10 @@ fn build_env_list( provider_spiffe_socket_path: Option<&str>, ) -> Vec { let mut env = existing_env.cloned().unwrap_or_default(); - apply_env_map(&mut env, template_environment); - apply_env_map(&mut env, spec_environment); let mut user_env = template_environment.clone(); user_env.extend(spec_environment.clone()); + remove_local_identity_metadata(&mut user_env); + apply_env_map(&mut env, &user_env); if !user_env.is_empty() && let Ok(json) = serde_json::to_string(&user_env) { @@ -2988,6 +2988,30 @@ fn apply_required_env( socket_path, ); } + tombstone_local_identity_metadata(env); +} + +fn remove_local_identity_metadata(environment: &mut std::collections::HashMap) { + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ] { + environment.remove(key); + } +} + +fn tombstone_local_identity_metadata(env: &mut Vec) { + // Kubernetes cannot remove image Config.Env entries. Explicit empty pod + // values mask them; the supervisor treats these protected tombstones as + // absent and retains the Kubernetes-assigned UID/GID behavior. + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ] { + upsert_env(env, key, ""); + } } fn provider_spiffe_socket_path<'a>(params: &'a SandboxPodParams<'a>) -> Option<&'a str> { @@ -5542,6 +5566,69 @@ mod tests { ); } + #[test] + fn sandbox_template_tombstones_local_identity_metadata() { + let protected = [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ]; + let template = SandboxTemplate { + environment: protected + .into_iter() + .map(|key| (key.to_string(), "image-value".to_string())) + .chain(std::iter::once(( + "SAFE_TEMPLATE".to_string(), + "yes".to_string(), + ))) + .collect(), + ..SandboxTemplate::default() + }; + let spec_environment = protected + .into_iter() + .map(|key| (key.to_string(), "request-value".to_string())) + .chain(std::iter::once(( + "SAFE_SPEC".to_string(), + "yes".to_string(), + ))) + .collect(); + let params = SandboxPodParams { + sandbox_uid: 1500, + sandbox_gid: 1600, + ..SandboxPodParams::default() + }; + + let pod_template = + sandbox_template_to_k8s(&template, false, &spec_environment, false, ¶ms); + let agent = &pod_template["spec"]["containers"][0]; + + for key in protected { + assert_eq!(rendered_env(agent, key), Some("")); + } + assert_eq!( + rendered_env(agent, openshell_core::sandbox_env::SANDBOX_UID), + Some("1500") + ); + assert_eq!( + rendered_env(agent, openshell_core::sandbox_env::SANDBOX_GID), + Some("1600") + ); + + let user_env: std::collections::HashMap = serde_json::from_str( + rendered_env(agent, openshell_core::sandbox_env::USER_ENVIRONMENT) + .expect("safe user environment should be serialized"), + ) + .unwrap(); + for key in protected { + assert!(!user_env.contains_key(key)); + } + assert_eq!( + user_env.get("SAFE_TEMPLATE").map(String::as_str), + Some("yes") + ); + assert_eq!(user_env.get("SAFE_SPEC").map(String::as_str), Some("yes")); + } + #[test] fn node_selector_from_platform_config() { let template = SandboxTemplate { diff --git a/crates/openshell-driver-podman/README.md b/crates/openshell-driver-podman/README.md index d068dfbc8f..8f4006fc59 100644 --- a/crates/openshell-driver-podman/README.md +++ b/crates/openshell-driver-podman/README.md @@ -50,6 +50,41 @@ The container spec in `container.rs` sets these security-critical fields: The restricted agent child does not retain these supervisor privileges. +## Agent Identity + +`[openshell.drivers.podman].identity_source` selects the identity for agent +children. It accepts `image` or `fixed` and defaults to `image`. Image mode +requires the sandbox image to declare a non-root OCI `USER`; there is no +implicit `10001:10001` fallback. Fixed mode requires both `fixed_uid` and +`fixed_gid`, each in the inclusive range `1000` through `2000000000`. Fixed +fields are invalid in image mode. + +The driver applies the image pull policy, inspects the final image, and carries +its raw OCI `Config.User` and immutable image ID into provisioning. The +container uses that immutable ID with create-time pull policy `never`, while +`user = "0:0"` keeps the supervisor privileged. Only agent children drop to +the resolved identity. The driver also resolves the supervisor image and +driver-config image mounts to immutable IDs before container creation. + +Image identity accepts named, numeric, and mixed `user:group` OCI forms. A +named user or group must resolve uniquely from the image's regular +`/etc/passwd` or `/etc/group` file. A numeric UID without a group uses the +matching passwd entry's primary GID. A numeric pair such as `USER 1234:1235` +requires no account entries. OpenShell rejects a missing `USER`, an unknown or +ambiguous name, an accountless numeric UID without a group, and any declaration +that resolves to UID or GID 0. + +Agent children receive `HOME=/sandbox`, no supplementary groups, and the +declared user name in `USER` and `LOGNAME` when one is available. Numeric and +fixed identities use the numeric UID for presentation. OpenShell does not +modify `/etc/passwd` or `/etc/group`. + +The supervisor persists the resolved source, immutable image ID, UID, GID, and +empty supplementary group list. The gateway exposes this record in sandbox +status, and the supervisor emits it as an OCSF configuration-state event. +Restarts reuse the persisted identity instead of resolving a mutable tag or +changed account file again. + ## Driver Config Mounts The gateway forwards the `podman` block from `--driver-config-json` to this @@ -57,20 +92,29 @@ driver. The driver accepts user-supplied `mounts` entries with these Podman mount types: - `bind`: mounts an absolute host path when `[openshell.drivers.podman]` - has `enable_bind_mounts = true`. + has `enable_bind_mounts = true`. It also requires fixed identity mode. - `volume`: mounts an existing Podman named volume. The driver validates that the volume exists before provisioning and never creates or removes it. Podman local-driver volumes created with bind options are treated as host bind - mounts and require `enable_bind_mounts = true`. + mounts and require `enable_bind_mounts = true`. All creator-selected named + volumes require fixed identity mode. - `tmpfs`: mounts an in-memory filesystem with optional `options`, `size_bytes`, and `mode`. - `image`: mounts an OCI image through Podman's image-volume API. The driver - pulls the image during provisioning using the sandbox image pull policy. + pulls and inspects the image during provisioning using the sandbox image pull + policy, then mounts its immutable ID. Image mode requires this mount to be + read-only. Host bind mounts are disabled by default because they expose gateway host paths to sandbox requests. The driver still uses internal bind mounts for configured TLS material; per-sandbox gateway JWTs are delivered through Podman secrets. +Image identity mode rejects creator-selected bind and named-volume mounts, +even when `enable_bind_mounts = true`. It permits `tmpfs`, the driver-owned +per-sandbox workspace volume, and read-only immutable `image` mounts. Use fixed +mode for external or shared storage that expects an operator-controlled UID and +GID. + Podman `bind` mounts accept `source`, `target`, optional `read_only`, and an optional `selinux_label` of `shared` (applies `:z`) or `private` (applies `:Z`) for SELinux-enforcing hosts. User-supplied bind and volume mounts are @@ -83,6 +127,13 @@ the workspace root (`/sandbox`) or overlap OpenShell supervisor files, Example named-volume usage: +```toml +[openshell.drivers.podman] +identity_source = "fixed" +fixed_uid = 1000 +fixed_gid = 1000 +``` + ```shell podman volume create openshell-work @@ -341,6 +392,9 @@ Podman resources after out-of-band container removal or label drift. | `OPENSHELL_SANDBOX_SSH_SOCKET_PATH` | `--sandbox-ssh-socket-path` | `/run/openshell/ssh.sock` | Supervisor Unix socket path in `PodmanComputeConfig`. | | `OPENSHELL_STOP_TIMEOUT` | `--stop-timeout` | `10` | Container stop timeout in seconds. | | `OPENSHELL_SANDBOX_PIDS_LIMIT` | `--sandbox-pids-limit` | `2048` | Podman cgroup PID limit for sandbox containers. Set `0` to inherit Podman's runtime/default PID limit. | +| `OPENSHELL_PODMAN_IDENTITY_SOURCE` | `--identity-source` | `image` | Agent identity source. Accepted values are `image` and `fixed`. | +| `OPENSHELL_PODMAN_FIXED_UID` | `--fixed-uid` | unset | Agent UID for fixed identity mode. Must be set with `--fixed-gid`. | +| `OPENSHELL_PODMAN_FIXED_GID` | `--fixed-gid` | unset | Agent primary GID for fixed identity mode. Must be set with `--fixed-uid`. | | `OPENSHELL_SUPERVISOR_IMAGE` | `--supervisor-image` | `ghcr.io/nvidia/openshell/supervisor:latest` through the gateway, required standalone | OCI image containing the supervisor binary. | | `OPENSHELL_PODMAN_TLS_CA` | `--podman-tls-ca` | unset | Host path to the CA certificate mounted for sandbox mTLS. | | `OPENSHELL_PODMAN_TLS_CERT` | `--podman-tls-cert` | unset | Host path to the client certificate mounted for sandbox mTLS. | diff --git a/crates/openshell-driver-podman/src/client.rs b/crates/openshell-driver-podman/src/client.rs index 59cbda545d..19fa718eb8 100644 --- a/crates/openshell-driver-podman/src/client.rs +++ b/crates/openshell-driver-podman/src/client.rs @@ -206,6 +206,39 @@ pub struct VolumeInspect { pub options: HashMap, } +/// Narrow subset of a Podman image-inspect response used for identity and +/// immutable rootfs selection. +#[derive(Debug, Clone, serde::Deserialize)] +pub struct ImageInspect { + #[serde(rename = "Id", alias = "ID", alias = "id")] + pub id: String, + #[serde(rename = "Config", alias = "config", default)] + pub config: ImageConfig, +} + +#[derive(Debug, Clone, Default, serde::Deserialize)] +pub struct ImageConfig { + #[serde(rename = "User", alias = "user", default)] + pub user: String, +} + +/// Selected immutable image identifier reported by an image pull, when the +/// Podman API supplied one. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct ImagePullResult { + pub id: Option, +} + +#[derive(Debug, serde::Deserialize)] +struct ImagePullReport { + #[serde(default)] + id: String, + #[serde(default)] + images: Vec, + #[serde(default)] + error: String, +} + /// A Podman event from the events stream. #[derive(Debug, Clone, serde::Deserialize)] #[serde(rename_all = "PascalCase")] @@ -644,9 +677,14 @@ impl PodmanClient { /// exists API and the local image store (e.g. `openshell/supervisor:dev` /// vs `localhost/openshell/supervisor:dev`). /// - /// The Podman pull endpoint streams NDJSON progress. We consume the - /// entire stream and check for an `error` field in the final object. - pub async fn pull_image(&self, reference: &str, policy: &str) -> Result<(), PodmanApiError> { + /// The Podman pull endpoint streams NDJSON progress. Every nonempty report + /// is parsed and checked so an error in an intermediate object cannot be + /// hidden by a later progress update. + pub async fn pull_image( + &self, + reference: &str, + policy: &str, + ) -> Result { let path = format!( "/libpod/images/pull?reference={}&policy={}", url_encode(reference), @@ -660,19 +698,54 @@ impl PodmanClient { if !status.is_success() { return Err(error_from_response(status.as_u16(), &bytes)); } - // The response is NDJSON. Check the last line for an error field. - let body = String::from_utf8_lossy(&bytes); - if let Some(last_line) = body.lines().rfind(|l| !l.is_empty()) - && let Ok(obj) = serde_json::from_str::(last_line) - && let Some(err) = obj.get("error").and_then(|v| v.as_str()) - && !err.is_empty() - { - return Err(PodmanApiError::Api { - status: 500, - message: format!("image pull failed: {err}"), - }); + let mut selected_id = None; + for line in bytes.split(|byte| *byte == b'\n') { + if line.iter().all(u8::is_ascii_whitespace) { + continue; + } + let report: ImagePullReport = serde_json::from_slice(line).map_err(|error| { + PodmanApiError::Json(format!( + "invalid image pull report: {error}: {}", + String::from_utf8_lossy(line) + )) + })?; + if !report.error.is_empty() { + return Err(PodmanApiError::Api { + status: 500, + message: format!("image pull failed: {}", report.error), + }); + } + if let Some(id) = std::iter::once(report.id) + .chain(report.images.into_iter().rev()) + .find(|id| !id.trim().is_empty()) + { + selected_id = Some(id); + } } - Ok(()) + Ok(ImagePullResult { id: selected_id }) + } + + /// Inspect an image by reference or immutable ID. + pub async fn inspect_image(&self, image: &str) -> Result { + if image.is_empty() { + return Err(PodmanApiError::InvalidInput( + "image reference must not be empty".to_string(), + )); + } + let encoded = url_encode(image); + let inspect: ImageInspect = self + .request_json( + hyper::Method::GET, + &format!("/libpod/images/{encoded}/json"), + None, + ) + .await?; + if inspect.id.trim().is_empty() { + return Err(PodmanApiError::Json( + "image inspect response has an empty Id".to_string(), + )); + } + Ok(inspect) } // ── System operations ──────────────────────────────────────────────── @@ -903,4 +976,166 @@ mod tests { ); let _ = std::fs::remove_file(socket_path); } + + #[tokio::test] + async fn inspect_image_escapes_reference_and_preserves_raw_user() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "inspect-image", + vec![StubResponse::new( + StatusCode::OK, + r#"{"Id":"sha256:immutable","Config":{"User":" app:staff "}}"#, + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let image = client + .inspect_image("registry.example.com/team/image:latest") + .await + .expect("image inspect should parse"); + + assert_eq!(image.id, "sha256:immutable"); + assert_eq!(image.config.user, " app:staff "); + handle.await.expect("stub task should finish"); + assert_eq!( + request_log + .lock() + .expect("request log lock should not be poisoned") + .as_slice(), + ["GET /v5.0.0/libpod/images/registry.example.com%2Fteam%2Fimage%3Alatest/json"] + ); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn inspect_image_rejects_empty_id() { + let (socket_path, _request_log, handle) = spawn_podman_stub( + "inspect-image-empty-id", + vec![StubResponse::new( + StatusCode::OK, + r#"{"Id":" ","Config":{"User":"app"}}"#, + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let error = client.inspect_image("image:latest").await.unwrap_err(); + assert!(error.to_string().contains("empty Id")); + handle.await.expect("stub task should finish"); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn pull_image_parses_every_report_and_returns_selected_id() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "pull-image-id", + vec![StubResponse::new( + StatusCode::OK, + "{\"images\":[\"sha256:first\"]}\n{\"id\":\"sha256:selected\"}\n", + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let result = client + .pull_image("registry.example.com/team/image:latest", "newer") + .await + .expect("pull reports should parse"); + + assert_eq!(result.id.as_deref(), Some("sha256:selected")); + handle.await.expect("stub task should finish"); + assert_eq!( + request_log + .lock() + .expect("request log lock should not be poisoned") + .as_slice(), + [ + "POST /v5.0.0/libpod/images/pull?reference=registry.example.com%2Fteam%2Fimage%3Alatest&policy=newer" + ] + ); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn pull_image_never_accepts_empty_local_hit_response() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "pull-image-never-empty", + vec![StubResponse::new(StatusCode::OK, "")], + ); + let client = PodmanClient::new(socket_path.clone()); + + let result = client + .pull_image("localhost/image:local", "never") + .await + .expect("a successful local-only resolution may have no pull report"); + + assert_eq!(result.id, None); + handle.await.expect("stub task should finish"); + assert_eq!( + request_log + .lock() + .expect("request log lock should not be poisoned") + .as_slice(), + ["POST /v5.0.0/libpod/images/pull?reference=localhost%2Fimage%3Alocal&policy=never"] + ); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn pull_image_local_hit_report_without_id_returns_none() { + let (socket_path, _request_log, handle) = spawn_podman_stub( + "pull-image-local-hit", + vec![StubResponse::new( + StatusCode::OK, + "{\"stream\":\"Image already exists locally\"}\n", + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let result = client + .pull_image("localhost/image:local", "missing") + .await + .expect("a local hit without an ID should still succeed"); + + assert_eq!(result.id, None); + handle.await.expect("stub task should finish"); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn pull_image_fails_on_intermediate_error_report() { + let (socket_path, _request_log, handle) = spawn_podman_stub( + "pull-image-error", + vec![StubResponse::new( + StatusCode::OK, + "{\"id\":\"sha256:first\"}\n{\"error\":\"registry denied\"}\n{\"id\":\"sha256:last\"}\n", + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let error = client + .pull_image("image:latest", "always") + .await + .unwrap_err(); + assert!(error.to_string().contains("registry denied")); + handle.await.expect("stub task should finish"); + let _ = std::fs::remove_file(socket_path); + } + + #[tokio::test] + async fn pull_image_fails_on_malformed_report() { + let (socket_path, _request_log, handle) = spawn_podman_stub( + "pull-image-malformed", + vec![StubResponse::new( + StatusCode::OK, + "{\"id\":\"sha256:first\"}\nnot-json\n", + )], + ); + let client = PodmanClient::new(socket_path.clone()); + + let error = client + .pull_image("image:latest", "missing") + .await + .unwrap_err(); + assert!(error.to_string().contains("invalid image pull report")); + handle.await.expect("stub task should finish"); + let _ = std::fs::remove_file(socket_path); + } } diff --git a/crates/openshell-driver-podman/src/config.rs b/crates/openshell-driver-podman/src/config.rs index ba193e8e1c..783079a4b4 100644 --- a/crates/openshell-driver-podman/src/config.rs +++ b/crates/openshell-driver-podman/src/config.rs @@ -2,6 +2,7 @@ // SPDX-License-Identifier: Apache-2.0 use openshell_core::config::DEFAULT_STOP_TIMEOUT_SECS; +use openshell_core::sandbox_env::IdentitySource; use std::net::IpAddr; use std::path::PathBuf; use std::str::FromStr; @@ -9,6 +10,9 @@ use std::str::FromStr; /// Default Podman bridge network name. pub const DEFAULT_NETWORK_NAME: &str = "openshell"; pub const MACOS_PODMAN_MACHINE_HOST_GATEWAY_IP: &str = "192.168.127.254"; +// Keep fixed local identities within the process-policy UID/GID range. +const MIN_FIXED_ID: u32 = 1_000; +const MAX_FIXED_ID: u32 = 2_000_000_000; // Re-export the shared default so existing imports inside this crate keep working. pub use openshell_core::config::DEFAULT_SANDBOX_PIDS_LIMIT; @@ -121,6 +125,13 @@ pub struct PodmanComputeConfig { /// /// Set to `0` to leave Podman's runtime/default PID limit unchanged. pub sandbox_pids_limit: i64, + /// Source used to select the agent identity. Image identity is the safe + /// default for isolated local sandboxes. + pub identity_source: IdentitySource, + /// Operator-selected UID used only in fixed identity mode. + pub fixed_uid: Option, + /// Operator-selected GID used only in fixed identity mode. + pub fixed_gid: Option, /// Allow sandbox requests to attach host bind mounts through /// `template.driver_config`. #[serde(default)] @@ -239,6 +250,40 @@ impl PodmanComputeConfig { Ok(()) } + /// Validate the local agent identity selection. + pub fn validate_identity_config(&self) -> Result<(), crate::client::PodmanApiError> { + match self.identity_source { + IdentitySource::Image => { + if self.fixed_uid.is_some() || self.fixed_gid.is_some() { + return Err(crate::client::PodmanApiError::InvalidInput( + "fixed_uid and fixed_gid are only valid when identity_source = 'fixed'" + .to_string(), + )); + } + } + IdentitySource::Fixed => { + let uid = self.fixed_uid.ok_or_else(|| { + crate::client::PodmanApiError::InvalidInput( + "fixed_uid is required when identity_source = 'fixed'".to_string(), + ) + })?; + let gid = self.fixed_gid.ok_or_else(|| { + crate::client::PodmanApiError::InvalidInput( + "fixed_gid is required when identity_source = 'fixed'".to_string(), + ) + })?; + for (field, value) in [("fixed_uid", uid), ("fixed_gid", gid)] { + if !(MIN_FIXED_ID..=MAX_FIXED_ID).contains(&value) { + return Err(crate::client::PodmanApiError::InvalidInput(format!( + "{field} must be in [{MIN_FIXED_ID}, {MAX_FIXED_ID}]" + ))); + } + } + } + } + Ok(()) + } + /// Validate optional corporate proxy configuration. /// /// Shares validation semantics with the in-container supervisor through @@ -372,6 +417,9 @@ impl Default for PodmanComputeConfig { guest_tls_cert: None, guest_tls_key: None, sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, + identity_source: IdentitySource::Image, + fixed_uid: None, + fixed_gid: None, enable_bind_mounts: false, health_check_interval_secs: DEFAULT_HEALTH_CHECK_INTERVAL_SECS, https_proxy: None, @@ -400,6 +448,9 @@ impl std::fmt::Debug for PodmanComputeConfig { .field("guest_tls_cert", &self.guest_tls_cert) .field("guest_tls_key", &self.guest_tls_key) .field("sandbox_pids_limit", &self.sandbox_pids_limit) + .field("identity_source", &self.identity_source) + .field("fixed_uid", &self.fixed_uid) + .field("fixed_gid", &self.fixed_gid) .field("enable_bind_mounts", &self.enable_bind_mounts) .field( "health_check_interval_secs", @@ -472,6 +523,73 @@ mod tests { assert!(err.to_string().contains("sandbox_pids_limit")); } + #[test] + fn identity_defaults_to_image_without_fixed_ids() { + let cfg = PodmanComputeConfig::default(); + assert_eq!(cfg.identity_source, IdentitySource::Image); + assert_eq!(cfg.fixed_uid, None); + assert_eq!(cfg.fixed_gid, None); + assert!(cfg.validate_identity_config().is_ok()); + } + + #[test] + fn image_identity_rejects_fixed_fields() { + for cfg in [ + PodmanComputeConfig { + fixed_uid: Some(10_001), + ..PodmanComputeConfig::default() + }, + PodmanComputeConfig { + fixed_gid: Some(10_001), + ..PodmanComputeConfig::default() + }, + ] { + assert!(cfg.validate_identity_config().is_err()); + } + } + + #[test] + fn fixed_identity_requires_both_ids() { + for (fixed_uid, fixed_gid) in [(None, None), (Some(10_001), None), (None, Some(10_001))] { + let cfg = PodmanComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid, + fixed_gid, + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_identity_config().is_err()); + } + } + + #[test] + fn fixed_identity_accepts_explicit_policy_ids() { + let cfg = PodmanComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid: Some(MIN_FIXED_ID), + fixed_gid: Some(MAX_FIXED_ID), + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_identity_config().is_ok()); + } + + #[test] + fn fixed_identity_rejects_root_and_out_of_policy_ids() { + for (fixed_uid, fixed_gid) in [ + (0, 10_001), + (10_001, 0), + (MIN_FIXED_ID - 1, 10_001), + (10_001, MAX_FIXED_ID + 1), + ] { + let cfg = PodmanComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid: Some(fixed_uid), + fixed_gid: Some(fixed_gid), + ..PodmanComputeConfig::default() + }; + assert!(cfg.validate_identity_config().is_err()); + } + } + // ── Proxy config validation ─────────────────────────────────────── #[test] diff --git a/crates/openshell-driver-podman/src/container.rs b/crates/openshell-driver-podman/src/container.rs index e417358c6e..204cc08c59 100644 --- a/crates/openshell-driver-podman/src/container.rs +++ b/crates/openshell-driver-podman/src/container.rs @@ -10,6 +10,7 @@ use openshell_core::driver_mounts::SelinuxLabel; use openshell_core::gpu::{driver_gpu_requirements, validate_specific_gpu_device_request}; use openshell_core::proto::compute::v1::{DriverSandbox, DriverSandboxTemplate}; use openshell_core::proto_struct::deserialize_optional_non_empty_string_list; +use openshell_core::sandbox_env::IdentitySource; use openshell_core::{driver_mounts, proto_struct}; use serde::Serialize; use serde_json::Value; @@ -282,6 +283,15 @@ struct PodmanUserMounts { mounts: Vec, } +/// Immutable image selections resolved before container creation. +#[derive(Debug, Clone)] +pub struct PodmanContainerImages { + pub sandbox_id: String, + pub sandbox_user: String, + pub supervisor_id: String, + pub image_mount_ids: BTreeMap, +} + #[derive(Serialize)] struct HealthConfig { test: Vec, @@ -396,7 +406,8 @@ fn upstream_proxy_cli_args(config: &PodmanComputeConfig) -> Vec { fn build_env( sandbox: &DriverSandbox, config: &PodmanComputeConfig, - image: &str, + requested_image: &str, + images: &PodmanContainerImages, ) -> BTreeMap { let spec = sandbox.spec.as_ref(); let template = spec.and_then(|s| s.template.as_ref()); @@ -422,6 +433,15 @@ fn build_env( user_env.insert(k.clone(), v.clone()); } } + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, + ] { + user_env.remove(key); + } env.extend(user_env.clone()); if !user_env.is_empty() && let Ok(json) = serde_json::to_string(&user_env) @@ -452,7 +472,10 @@ fn build_env( openshell_core::sandbox_env::SSH_SOCKET_PATH.into(), config.sandbox_ssh_socket_path.clone(), ); - env.insert("OPENSHELL_CONTAINER_IMAGE".into(), image.to_string()); + env.insert( + "OPENSHELL_CONTAINER_IMAGE".into(), + requested_image.to_string(), + ); env.insert( openshell_core::sandbox_env::SANDBOX_COMMAND.into(), "sleep infinity".into(), @@ -494,6 +517,44 @@ fn build_env( ); } + // Identity metadata is driver-owned and inserted last. Explicit create + // environment overrides image-baked ENV values with the same names. + env.insert( + openshell_core::sandbox_env::IDENTITY_SOURCE.into(), + config.identity_source.to_string(), + ); + env.insert( + openshell_core::sandbox_env::IMAGE_ID.into(), + images.sandbox_id.clone(), + ); + match config.identity_source { + IdentitySource::Image => { + env.insert( + openshell_core::sandbox_env::IMAGE_USER.into(), + images.sandbox_user.clone(), + ); + env.remove(openshell_core::sandbox_env::SANDBOX_UID); + env.remove(openshell_core::sandbox_env::SANDBOX_GID); + } + IdentitySource::Fixed => { + env.insert( + openshell_core::sandbox_env::SANDBOX_UID.into(), + config + .fixed_uid + .expect("validated fixed identity must include a UID") + .to_string(), + ); + env.insert( + openshell_core::sandbox_env::SANDBOX_GID.into(), + config + .fixed_gid + .expect("validated fixed identity must include a GID") + .to_string(), + ); + env.remove(openshell_core::sandbox_env::IMAGE_USER); + } + } + env } @@ -555,6 +616,7 @@ fn podman_pids_limit(value: i64) -> Option { pub fn podman_driver_volume_mount_sources( sandbox: &DriverSandbox, enable_bind_mounts: bool, + identity_source: IdentitySource, ) -> Result, String> { let template = sandbox .spec @@ -563,7 +625,7 @@ pub fn podman_driver_volume_mount_sources( let Some(template) = template else { return Ok(Vec::new()); }; - let config = podman_driver_config(template, enable_bind_mounts)?; + let config = podman_driver_config(template, enable_bind_mounts, identity_source)?; Ok(config .mounts .into_iter() @@ -577,6 +639,7 @@ pub fn podman_driver_volume_mount_sources( pub fn podman_driver_image_mount_sources( sandbox: &DriverSandbox, enable_bind_mounts: bool, + identity_source: IdentitySource, ) -> Result, String> { let template = sandbox .spec @@ -585,7 +648,7 @@ pub fn podman_driver_image_mount_sources( let Some(template) = template else { return Ok(Vec::new()); }; - let config = podman_driver_config(template, enable_bind_mounts)?; + let config = podman_driver_config(template, enable_bind_mounts, identity_source)?; Ok(config .mounts .into_iter() @@ -599,6 +662,8 @@ pub fn podman_driver_image_mount_sources( fn podman_user_mounts( sandbox: &DriverSandbox, enable_bind_mounts: bool, + identity_source: IdentitySource, + image_mount_ids: &BTreeMap, ) -> Result { let template = sandbox .spec @@ -607,7 +672,7 @@ fn podman_user_mounts( let Some(template) = template else { return Ok(PodmanUserMounts::default()); }; - let config = podman_driver_config(template, enable_bind_mounts)?; + let config = podman_driver_config(template, enable_bind_mounts, identity_source)?; let mut result = PodmanUserMounts::default(); for mount in config.mounts { match mount { @@ -686,8 +751,11 @@ fn podman_user_mounts( reject_subpath(subpath.as_deref(), "podman image mounts")?; driver_mounts::validate_mount_source(&source, "image source")?; driver_mounts::validate_container_mount_target(&target)?; + let immutable_source = image_mount_ids.get(&source).ok_or_else(|| { + format!("podman image mount source '{source}' was not provisioned") + })?; result.image_volumes.push(ImageVolume { - source, + source: immutable_source.clone(), destination: target, rw: !read_only, }); @@ -700,6 +768,7 @@ fn podman_user_mounts( fn podman_driver_config( template: &DriverSandboxTemplate, enable_bind_mounts: bool, + identity_source: IdentitySource, ) -> Result { let Some(config) = template.driver_config.as_ref() else { return Ok(PodmanSandboxDriverConfig::default()); @@ -707,18 +776,25 @@ fn podman_driver_config( let json = Value::Object(proto_struct::struct_to_json_object(config)); let config: PodmanSandboxDriverConfig = serde_json::from_value(json) .map_err(|err| format!("invalid podman driver_config: {err}"))?; - validate_podman_driver_mounts(&config.mounts, enable_bind_mounts)?; + validate_podman_driver_mounts(&config.mounts, enable_bind_mounts, identity_source)?; Ok(config) } fn validate_podman_driver_mounts( mounts: &[PodmanDriverMountConfig], enable_bind_mounts: bool, + identity_source: IdentitySource, ) -> Result<(), String> { let mut targets = HashSet::new(); for mount in mounts { let target = match mount { PodmanDriverMountConfig::Bind { source, target, .. } => { + if identity_source == IdentitySource::Image { + return Err( + "podman bind mounts are not allowed when identity_source = 'image'; use fixed identity mode for external storage" + .to_string(), + ); + } if !enable_bind_mounts { return Err( "podman bind mounts require enable_bind_mounts = true in [openshell.drivers.podman]" @@ -734,6 +810,12 @@ fn validate_podman_driver_mounts( subpath, .. } => { + if identity_source == IdentitySource::Image { + return Err( + "podman named volume mounts are not allowed when identity_source = 'image'; use fixed identity mode for external storage" + .to_string(), + ); + } driver_mounts::validate_mount_source(source, "volume source")?; reject_subpath(subpath.as_deref(), "podman volume mounts")?; target @@ -752,9 +834,16 @@ fn validate_podman_driver_mounts( PodmanDriverMountConfig::Image { source, target, + read_only, subpath, .. } => { + if identity_source == IdentitySource::Image && !read_only { + return Err( + "podman image mounts must be read-only when identity_source = 'image'" + .to_string(), + ); + } driver_mounts::validate_mount_source(source, "image source")?; reject_subpath(subpath.as_deref(), "podman image mounts")?; target @@ -876,21 +965,68 @@ pub fn try_build_container_spec_with_token( build_container_spec_with_token_and_gpu_devices(sandbox, config, token_secret_name, cdi_devices) } +#[cfg(test)] pub fn build_container_spec_with_token_and_gpu_devices( sandbox: &DriverSandbox, config: &PodmanComputeConfig, token_secret_name: Option<&str>, gpu_device_ids: Option<&[String]>, ) -> Result { - let image = resolve_image(sandbox, config); + let image_mount_ids = podman_driver_image_mount_sources( + sandbox, + config.enable_bind_mounts, + config.identity_source, + ) + .map_err(ComputeDriverError::InvalidArgument)? + .into_iter() + .map(|source| (source.clone(), source)) + .collect(); + let images = PodmanContainerImages { + sandbox_id: resolve_image(sandbox, config).to_string(), + sandbox_user: "sandbox".to_string(), + supervisor_id: config.supervisor_image.clone(), + image_mount_ids, + }; + build_provisioned_container_spec_with_token_and_gpu_devices( + sandbox, + config, + token_secret_name, + gpu_device_ids, + &images, + ) +} + +pub fn build_provisioned_container_spec_with_token_and_gpu_devices( + sandbox: &DriverSandbox, + config: &PodmanComputeConfig, + token_secret_name: Option<&str>, + gpu_device_ids: Option<&[String]>, + images: &PodmanContainerImages, +) -> Result { + config + .validate_identity_config() + .map_err(|error| ComputeDriverError::InvalidArgument(error.to_string()))?; + if config.identity_source == IdentitySource::Image && images.sandbox_user.trim().is_empty() { + return Err(ComputeDriverError::Precondition( + "sandbox image must declare a non-empty OCI Config.User when identity_source = 'image'" + .to_string(), + )); + } + + let requested_image = resolve_image(sandbox, config); let name = container_name(&sandbox.workspace, &sandbox.name, &sandbox.id); let vol = volume_name(&sandbox.id); - let env = build_env(sandbox, config, image); + let env = build_env(sandbox, config, requested_image, images); let labels = build_labels(sandbox); let resource_limits = build_resource_limits(sandbox, config); - let user_mounts = podman_user_mounts(sandbox, config.enable_bind_mounts) - .map_err(ComputeDriverError::InvalidArgument)?; + let user_mounts = podman_user_mounts( + sandbox, + config.enable_bind_mounts, + config.identity_source, + &images.image_mount_ids, + ) + .map_err(ComputeDriverError::InvalidArgument)?; if sandbox .spec .as_ref() @@ -924,7 +1060,7 @@ pub fn build_container_spec_with_token_and_gpu_devices( volumes.extend(user_mounts.volumes); let mut image_volumes = vec![ImageVolume { - source: config.supervisor_image.clone(), + source: images.supervisor_id.clone(), destination: SUPERVISOR_MOUNT_DIR.into(), rw: false, }]; @@ -932,7 +1068,7 @@ pub fn build_container_spec_with_token_and_gpu_devices( let container_spec = ContainerSpec { name, - image: image.to_string(), + image: images.sandbox_id.clone(), labels, env, volumes, @@ -1030,7 +1166,9 @@ pub fn build_container_spec_with_token_and_gpu_devices( // locks itself down. no_new_privileges: true, seccomp_profile_path: "unconfined".into(), - image_pull_policy: config.image_pull_policy.as_str().to_string(), + // All image references were resolved and inspected before this spec + // was built. Never let container creation make a second pull choice. + image_pull_policy: "never".to_string(), healthconfig: HealthConfig { test: vec![ "CMD-SHELL".into(), @@ -1987,6 +2125,315 @@ mod tests { } } + fn fixed_test_config() -> PodmanComputeConfig { + PodmanComputeConfig { + identity_source: IdentitySource::Fixed, + fixed_uid: Some(10_001), + fixed_gid: Some(10_001), + ..test_config() + } + } + + fn sandbox_with_mount(mount: Value) -> DriverSandbox { + use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; + + let mut sandbox = test_sandbox("test-id", "test-name"); + sandbox.spec = Some(DriverSandboxSpec { + template: Some(DriverSandboxTemplate { + driver_config: Some(json_struct(serde_json::json!({"mounts": [mount]}))), + ..Default::default() + }), + ..Default::default() + }); + sandbox + } + + fn provisioned_images(user: &str) -> PodmanContainerImages { + PodmanContainerImages { + sandbox_id: "sha256:sandbox-immutable".to_string(), + sandbox_user: user.to_string(), + supervisor_id: "sha256:supervisor-immutable".to_string(), + image_mount_ids: BTreeMap::new(), + } + } + + #[test] + fn provisioned_container_uses_immutable_images_root_supervisor_and_pull_never() { + let sandbox = test_sandbox("test-id", "test-name"); + let config = test_config(); + let images = provisioned_images("app:staff"); + + let spec = build_provisioned_container_spec_with_token_and_gpu_devices( + &sandbox, &config, None, None, &images, + ) + .unwrap(); + + assert_eq!(spec["image"].as_str(), Some("sha256:sandbox-immutable")); + assert_eq!(spec["user"].as_str(), Some("0:0")); + assert_eq!(spec["image_pull_policy"].as_str(), Some("never")); + assert_eq!( + spec["image_volumes"][0]["source"].as_str(), + Some("sha256:supervisor-immutable") + ); + } + + #[test] + fn image_identity_metadata_overrides_request_spoofing_and_omits_fixed_ids() { + use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; + + let spoofed = std::collections::HashMap::from([ + ( + openshell_core::sandbox_env::IDENTITY_SOURCE.to_string(), + "fixed".to_string(), + ), + ( + openshell_core::sandbox_env::IMAGE_USER.to_string(), + "attacker".to_string(), + ), + ( + openshell_core::sandbox_env::IMAGE_ID.to_string(), + "sha256:attacker".to_string(), + ), + ( + openshell_core::sandbox_env::SANDBOX_UID.to_string(), + "1234".to_string(), + ), + ( + openshell_core::sandbox_env::SANDBOX_GID.to_string(), + "1234".to_string(), + ), + ]); + let mut sandbox = test_sandbox("test-id", "test-name"); + sandbox.spec = Some(DriverSandboxSpec { + environment: spoofed.clone(), + template: Some(DriverSandboxTemplate { + environment: spoofed, + ..Default::default() + }), + ..Default::default() + }); + let images = provisioned_images(" app:staff "); + + let spec = build_provisioned_container_spec_with_token_and_gpu_devices( + &sandbox, + &test_config(), + None, + None, + &images, + ) + .unwrap(); + let env = spec["env"].as_object().unwrap(); + + assert_eq!( + env[openshell_core::sandbox_env::IDENTITY_SOURCE].as_str(), + Some("image") + ); + assert_eq!( + env[openshell_core::sandbox_env::IMAGE_USER].as_str(), + Some(" app:staff ") + ); + assert_eq!( + env[openshell_core::sandbox_env::IMAGE_ID].as_str(), + Some("sha256:sandbox-immutable") + ); + assert!(!env.contains_key(openshell_core::sandbox_env::SANDBOX_UID)); + assert!(!env.contains_key(openshell_core::sandbox_env::SANDBOX_GID)); + if let Some(serialized) = env + .get(openshell_core::sandbox_env::USER_ENVIRONMENT) + .and_then(Value::as_str) + { + let user_env: BTreeMap = serde_json::from_str(serialized).unwrap(); + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, + ] { + assert!(!user_env.contains_key(key)); + } + } + } + + #[test] + fn fixed_identity_injects_explicit_ids_and_immutable_image_id() { + use openshell_core::proto::compute::v1::{DriverSandboxSpec, DriverSandboxTemplate}; + + let spoofed = std::collections::HashMap::from([ + ( + openshell_core::sandbox_env::IDENTITY_SOURCE.to_string(), + "image".to_string(), + ), + ( + openshell_core::sandbox_env::IMAGE_USER.to_string(), + "attacker".to_string(), + ), + ( + openshell_core::sandbox_env::IMAGE_ID.to_string(), + "sha256:attacker".to_string(), + ), + ( + openshell_core::sandbox_env::SANDBOX_UID.to_string(), + "1234".to_string(), + ), + ( + openshell_core::sandbox_env::SANDBOX_GID.to_string(), + "1234".to_string(), + ), + ]); + let mut sandbox = test_sandbox("test-id", "test-name"); + sandbox.spec = Some(DriverSandboxSpec { + environment: spoofed.clone(), + template: Some(DriverSandboxTemplate { + environment: spoofed, + ..Default::default() + }), + ..Default::default() + }); + let config = fixed_test_config(); + let images = provisioned_images(""); + + let spec = build_provisioned_container_spec_with_token_and_gpu_devices( + &sandbox, &config, None, None, &images, + ) + .unwrap(); + let env = spec["env"].as_object().unwrap(); + + assert_eq!( + env[openshell_core::sandbox_env::IDENTITY_SOURCE].as_str(), + Some("fixed") + ); + assert_eq!( + env[openshell_core::sandbox_env::SANDBOX_UID].as_str(), + Some("10001") + ); + assert_eq!( + env[openshell_core::sandbox_env::SANDBOX_GID].as_str(), + Some("10001") + ); + assert_eq!( + env[openshell_core::sandbox_env::IMAGE_ID].as_str(), + Some("sha256:sandbox-immutable") + ); + assert!(!env.contains_key(openshell_core::sandbox_env::IMAGE_USER)); + if let Some(serialized) = env + .get(openshell_core::sandbox_env::USER_ENVIRONMENT) + .and_then(Value::as_str) + { + let user_env: BTreeMap = serde_json::from_str(serialized).unwrap(); + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, + ] { + assert!(!user_env.contains_key(key)); + } + } + } + + #[test] + fn image_identity_rejects_blank_image_user() { + let sandbox = test_sandbox("test-id", "test-name"); + let error = build_provisioned_container_spec_with_token_and_gpu_devices( + &sandbox, + &test_config(), + None, + None, + &provisioned_images(" "), + ) + .unwrap_err(); + + assert!(error.to_string().contains("non-empty OCI Config.User")); + } + + #[test] + fn image_identity_rejects_bind_mount_even_when_enabled() { + let sandbox = sandbox_with_mount(serde_json::json!({ + "type": "bind", + "source": "/host/path", + "target": "/sandbox/host" + })); + let mut config = test_config(); + config.enable_bind_mounts = true; + + let error = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); + assert!(error.to_string().contains("identity_source = 'image'")); + } + + #[test] + fn image_identity_rejects_named_volume_mount() { + let sandbox = sandbox_with_mount(serde_json::json!({ + "type": "volume", + "source": "shared-work", + "target": "/sandbox/work" + })); + + let error = + try_build_container_spec_with_token(&sandbox, &test_config(), None).unwrap_err(); + assert!(error.to_string().contains("named volume mounts")); + } + + #[test] + fn image_identity_allows_tmpfs_mount() { + let sandbox = sandbox_with_mount(serde_json::json!({ + "type": "tmpfs", + "target": "/sandbox/cache" + })); + + let spec = build_container_spec(&sandbox, &test_config()); + assert!(spec["mounts"].as_array().unwrap().iter().any(|mount| { + mount["type"].as_str() == Some("tmpfs") + && mount["destination"].as_str() == Some("/sandbox/cache") + })); + } + + #[test] + fn image_identity_allows_only_read_only_pinned_image_mounts() { + let source = "registry.example.com/tools:latest"; + let sandbox = sandbox_with_mount(serde_json::json!({ + "type": "image", + "source": source, + "target": "/opt/tools", + "read_only": true + })); + let mut images = provisioned_images("app"); + images + .image_mount_ids + .insert(source.to_string(), "sha256:tools-immutable".to_string()); + + let spec = build_provisioned_container_spec_with_token_and_gpu_devices( + &sandbox, + &test_config(), + None, + None, + &images, + ) + .unwrap(); + assert!( + spec["image_volumes"] + .as_array() + .unwrap() + .iter() + .any(|mount| { + mount["source"].as_str() == Some("sha256:tools-immutable") + && mount["destination"].as_str() == Some("/opt/tools") + && mount["rw"].as_bool() == Some(false) + }) + ); + + let writable = sandbox_with_mount(serde_json::json!({ + "type": "image", + "source": source, + "target": "/opt/tools", + "read_only": false + })); + let error = + try_build_container_spec_with_token(&writable, &test_config(), None).unwrap_err(); + assert!(error.to_string().contains("must be read-only")); + } + #[test] fn container_spec_includes_supervisor_image_volume() { let sandbox = test_sandbox("test-id", "test-name"); @@ -2054,7 +2501,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let spec = build_container_spec(&sandbox, &config); let volumes = spec["volumes"] @@ -2121,7 +2568,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let spec = build_container_spec(&sandbox, &config); let volumes = spec["volumes"] @@ -2156,7 +2603,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let spec = build_container_spec(&sandbox, &config); let volumes = spec["volumes"] @@ -2196,7 +2643,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); @@ -2224,7 +2671,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); @@ -2250,7 +2697,7 @@ mod tests { }), ..Default::default() }); - let mut config = test_config(); + let mut config = fixed_test_config(); config.enable_bind_mounts = true; let spec = build_container_spec(&sandbox, &config); @@ -2289,7 +2736,7 @@ mod tests { }), ..Default::default() }); - let mut config = test_config(); + let mut config = fixed_test_config(); config.enable_bind_mounts = true; let spec = build_container_spec(&sandbox, &config); @@ -2328,7 +2775,7 @@ mod tests { }), ..Default::default() }); - let mut config = test_config(); + let mut config = fixed_test_config(); config.enable_bind_mounts = true; let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); @@ -2359,7 +2806,7 @@ mod tests { }), ..Default::default() }); - let mut config = test_config(); + let mut config = fixed_test_config(); config.enable_bind_mounts = true; let spec = build_container_spec(&sandbox, &config); @@ -2398,7 +2845,7 @@ mod tests { }), ..Default::default() }); - let mut config = test_config(); + let mut config = fixed_test_config(); config.enable_bind_mounts = true; let spec = build_container_spec(&sandbox, &config); @@ -2435,7 +2882,7 @@ mod tests { }), ..Default::default() }); - let config = test_config(); + let config = fixed_test_config(); let err = try_build_container_spec_with_token(&sandbox, &config, None).unwrap_err(); diff --git a/crates/openshell-driver-podman/src/driver.rs b/crates/openshell-driver-podman/src/driver.rs index 3878f59836..1cd6ab9944 100644 --- a/crates/openshell-driver-podman/src/driver.rs +++ b/crates/openshell-driver-podman/src/driver.rs @@ -3,7 +3,7 @@ //! Podman compute driver. -use crate::client::{PodmanApiError, PodmanClient, VolumeInspect}; +use crate::client::{ImageInspect, PodmanApiError, PodmanClient, VolumeInspect}; use crate::config::PodmanComputeConfig; use crate::container::{self, LABEL_MANAGED_FILTER, LABEL_SANDBOX_ID, PodmanSandboxDriverConfig}; use crate::watcher::{ @@ -19,6 +19,8 @@ use openshell_core::gpu::{ use openshell_core::proto::compute::v1::{ DriverSandbox, GetCapabilitiesResponse, GpuResourceRequirements, }; +use openshell_core::sandbox_env::IdentitySource; +use std::collections::BTreeMap; use std::path::{Path, PathBuf}; use std::sync::Arc; use std::time::Duration; @@ -261,6 +263,7 @@ impl PodmanComputeDriver { // get a clear error instead of a silent fallback to plaintext HTTP. config.validate_tls_config()?; config.validate_runtime_limits()?; + config.validate_identity_config()?; config.validate_host_gateway_ip()?; config.validate_proxy_config()?; @@ -412,6 +415,9 @@ impl PodmanComputeDriver { &self, sandbox: &'a DriverSandbox, ) -> Result, ComputeDriverError> { + self.config + .validate_identity_config() + .map_err(|error| ComputeDriverError::Precondition(error.to_string()))?; let gpu_requirements = sandbox .spec .as_ref() @@ -489,9 +495,12 @@ impl PodmanComputeDriver { &self, sandbox: &DriverSandbox, ) -> Result<(), ComputeDriverError> { - let volumes = - container::podman_driver_volume_mount_sources(sandbox, self.config.enable_bind_mounts) - .map_err(ComputeDriverError::Precondition)?; + let volumes = container::podman_driver_volume_mount_sources( + sandbox, + self.config.enable_bind_mounts, + self.config.identity_source, + ) + .map_err(ComputeDriverError::Precondition)?; for volume in volumes { match self.client.inspect_volume(&volume).await { Ok(volume_info) => { @@ -513,6 +522,23 @@ impl PodmanComputeDriver { Ok(()) } + async fn provision_image( + &self, + reference: &str, + pull_policy: &str, + ) -> Result { + let pull = self + .client + .pull_image(reference, pull_policy) + .await + .map_err(ComputeDriverError::from)?; + let selected = pull.id.as_deref().unwrap_or(reference); + self.client + .inspect_image(selected) + .await + .map_err(ComputeDriverError::from) + } + /// Create a sandbox container. pub async fn create_sandbox(&self, sandbox: &DriverSandbox) -> Result<(), ComputeDriverError> { if sandbox.name.is_empty() { @@ -551,10 +577,9 @@ impl PodmanComputeDriver { policy = supervisor_pull_policy, "Ensuring supervisor image" ); - self.client - .pull_image(&self.config.supervisor_image, supervisor_pull_policy) - .await - .map_err(ComputeDriverError::from)?; + let supervisor_image = self + .provision_image(&self.config.supervisor_image, supervisor_pull_policy) + .await?; // 1b. Pull the sandbox image if needed (Podman does not pull on create). let image = container::resolve_image(sandbox, &self.config); @@ -567,22 +592,38 @@ impl PodmanComputeDriver { } let pull_policy = self.config.image_pull_policy.as_str(); info!(image = %image, policy = %pull_policy, "Ensuring sandbox image"); - self.client - .pull_image(image, pull_policy) - .await - .map_err(ComputeDriverError::from)?; + let sandbox_image = self.provision_image(image, pull_policy).await?; + if self.config.identity_source == IdentitySource::Image + && sandbox_image.config.user.trim().is_empty() + { + return Err(ComputeDriverError::Precondition(format!( + "sandbox image '{image}' must declare a non-empty OCI Config.User when identity_source = 'image'" + ))); + } - for image in - container::podman_driver_image_mount_sources(sandbox, self.config.enable_bind_mounts) - .map_err(ComputeDriverError::Precondition)? + let mut image_mount_ids = BTreeMap::new(); + for image in container::podman_driver_image_mount_sources( + sandbox, + self.config.enable_bind_mounts, + self.config.identity_source, + ) + .map_err(ComputeDriverError::Precondition)? { + if image_mount_ids.contains_key(&image) { + continue; + } info!(image = %image, policy = %pull_policy, "Ensuring image mount source"); - self.client - .pull_image(&image, pull_policy) - .await - .map_err(ComputeDriverError::from)?; + let inspected = self.provision_image(&image, pull_policy).await?; + image_mount_ids.insert(image, inspected.id); } + let images = container::PodmanContainerImages { + sandbox_id: sandbox_image.id, + sandbox_user: sandbox_image.config.user, + supervisor_id: supervisor_image.id, + image_mount_ids, + }; + // 2. Create workspace volume and per-sandbox token secret. if let Err(e) = self.client.create_volume(&vol_name).await { return Err(ComputeDriverError::from(e)); @@ -630,11 +671,12 @@ impl PodmanComputeDriver { return Err(e); } }; - let spec = match container::build_container_spec_with_token_and_gpu_devices( + let spec = match container::build_provisioned_container_spec_with_token_and_gpu_devices( sandbox, &self.config, token_secret_name.as_deref(), gpu_devices.as_deref(), + &images, ) { Ok(spec) => spec, Err(e) => { @@ -1377,6 +1419,18 @@ mod tests { PodmanComputeDriver::for_tests(config) } + fn test_driver_fixed(socket_path: PathBuf) -> PodmanComputeDriver { + let config = PodmanComputeConfig { + socket_path: Some(socket_path), + stop_timeout_secs: 10, + identity_source: IdentitySource::Fixed, + fixed_uid: Some(10_001), + fixed_gid: Some(10_001), + ..PodmanComputeConfig::default() + }; + PodmanComputeDriver::for_tests(config) + } + fn test_driver_with_config(config: PodmanComputeConfig) -> PodmanComputeDriver { PodmanComputeDriver::for_tests(config) } @@ -1416,6 +1470,117 @@ mod tests { format!("/v5.0.0{path}") } + #[tokio::test] + async fn provision_image_inspects_pull_selected_id() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "provision-selected-id", + vec![ + StubResponse::new(StatusCode::OK, "{\"id\":\"sha256:selected\"}\n"), + StubResponse::new( + StatusCode::OK, + r#"{"Id":"sha256:selected","Config":{"User":"app"}}"#, + ), + ], + ); + let driver = test_driver(socket_path.clone()); + + let image = driver + .provision_image("registry.example.com/team/image:latest", "newer") + .await + .unwrap(); + + assert_eq!(image.id, "sha256:selected"); + assert_eq!(image.config.user, "app"); + handle.await.expect("stub task should finish"); + assert_eq!( + request_log + .lock() + .expect("request log lock should not be poisoned") + .as_slice(), + [ + "POST /v5.0.0/libpod/images/pull?reference=registry.example.com%2Fteam%2Fimage%3Alatest&policy=newer", + "GET /v5.0.0/libpod/images/sha256%3Aselected/json", + ] + ); + let _ = fs::remove_file(socket_path); + } + + #[tokio::test] + async fn provision_image_without_pull_id_inspects_original_reference() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "provision-reference-fallback", + vec![ + StubResponse::new(StatusCode::OK, ""), + StubResponse::new( + StatusCode::OK, + r#"{"Id":"sha256:local","Config":{"User":"app"}}"#, + ), + ], + ); + let driver = test_driver(socket_path.clone()); + + let image = driver + .provision_image("localhost/team/image:local", "never") + .await + .unwrap(); + + assert_eq!(image.id, "sha256:local"); + handle.await.expect("stub task should finish"); + assert_eq!( + request_log + .lock() + .expect("request log lock should not be poisoned") + .as_slice(), + [ + "POST /v5.0.0/libpod/images/pull?reference=localhost%2Fteam%2Fimage%3Alocal&policy=never", + "GET /v5.0.0/libpod/images/localhost%2Fteam%2Fimage%3Alocal/json", + ] + ); + let _ = fs::remove_file(socket_path); + } + + #[tokio::test] + async fn image_identity_missing_user_fails_before_volume_or_secret_creation() { + let (socket_path, request_log, handle) = spawn_podman_stub( + "missing-image-user", + vec![ + StubResponse::new(StatusCode::OK, "{\"id\":\"sha256:supervisor\"}\n"), + StubResponse::new( + StatusCode::OK, + r#"{"Id":"sha256:supervisor","Config":{"User":"0:0"}}"#, + ), + StubResponse::new(StatusCode::OK, "{\"id\":\"sha256:sandbox\"}\n"), + StubResponse::new( + StatusCode::OK, + r#"{"Id":"sha256:sandbox","Config":{"User":" "}}"#, + ), + ], + ); + let driver = test_driver(socket_path.clone()); + let mut sandbox = plain_sandbox("sandbox-no-user", "demo"); + sandbox.spec = Some(DriverSandboxSpec { + sandbox_token: "must-not-be-staged".to_string(), + ..Default::default() + }); + + let error = driver.create_sandbox(&sandbox).await.unwrap_err(); + + assert!(error.to_string().contains("non-empty OCI Config.User")); + handle.await.expect("stub task should finish"); + let requests = request_log + .lock() + .expect("request log lock should not be poisoned") + .clone(); + assert_eq!(requests.len(), 4); + assert!( + requests + .iter() + .all(|request| !request.contains("/volumes/") && !request.contains("/secrets/")), + "identity failure must happen before volume or secret creation: {requests:?}" + ); + let _ = fs::remove_file(socket_path); + } + #[test] fn podman_local_volume_with_bind_option_is_bind_backed() { let volume = VolumeInspect { @@ -1475,7 +1640,7 @@ mod tests { r#"{"Name":"work-bind","Driver":"local","Options":{"type":"none","o":"rw,bind","device":"/srv/work"}}"#, )], ); - let driver = test_driver(socket_path.clone()); + let driver = test_driver_fixed(socket_path.clone()); let sandbox = sandbox_with_volume_mount("work-bind"); let err = driver @@ -1512,7 +1677,7 @@ mod tests { r#"{"Name":"work-rbind","Driver":"local","Options":{"type":"none","o":"rw,rbind","device":"/srv/work"}}"#, )], ); - let driver = test_driver(socket_path.clone()); + let driver = test_driver_fixed(socket_path.clone()); let sandbox = sandbox_with_volume_mount("work-rbind"); let err = driver @@ -1552,6 +1717,9 @@ mod tests { let config = PodmanComputeConfig { socket_path: Some(socket_path.clone()), enable_bind_mounts: true, + identity_source: IdentitySource::Fixed, + fixed_uid: Some(10_001), + fixed_gid: Some(10_001), ..PodmanComputeConfig::default() }; let driver = test_driver_with_config(config); @@ -1617,6 +1785,9 @@ mod tests { stop_timeout_secs: 10, proxy_auth_file: Some(auth_file.to_string_lossy().into_owned()), proxy_auth_allow_insecure: Some(true), + identity_source: IdentitySource::Fixed, + fixed_uid: Some(10_001), + fixed_gid: Some(10_001), ..PodmanComputeConfig::default() } } @@ -1652,7 +1823,9 @@ mod tests { "create-container-fail", vec![ StubResponse::new(StatusCode::OK, "{}"), // pull supervisor image + StubResponse::new(StatusCode::OK, r#"{"Id":"sha256:supervisor","Config":{}}"#), // inspect supervisor image StubResponse::new(StatusCode::OK, "{}"), // pull sandbox image + StubResponse::new(StatusCode::OK, r#"{"Id":"sha256:sandbox","Config":{}}"#), // inspect sandbox image StubResponse::new(StatusCode::CREATED, "{}"), // create volume StubResponse::new(StatusCode::CREATED, "{}"), // create proxy-auth secret StubResponse::new(StatusCode::INTERNAL_SERVER_ERROR, r#"{"message":"boom"}"#), // create container @@ -1690,7 +1863,9 @@ mod tests { "create-start-fail", vec![ StubResponse::new(StatusCode::OK, "{}"), // pull supervisor image + StubResponse::new(StatusCode::OK, r#"{"Id":"sha256:supervisor","Config":{}}"#), // inspect supervisor image StubResponse::new(StatusCode::OK, "{}"), // pull sandbox image + StubResponse::new(StatusCode::OK, r#"{"Id":"sha256:sandbox","Config":{}}"#), // inspect sandbox image StubResponse::new(StatusCode::CREATED, "{}"), // create volume StubResponse::new(StatusCode::CREATED, "{}"), // create proxy-auth secret StubResponse::new(StatusCode::CREATED, "{}"), // create container diff --git a/crates/openshell-driver-podman/src/main.rs b/crates/openshell-driver-podman/src/main.rs index 35191e4cda..7f80961e47 100644 --- a/crates/openshell-driver-podman/src/main.rs +++ b/crates/openshell-driver-podman/src/main.rs @@ -11,6 +11,7 @@ use tracing_subscriber::EnvFilter; use openshell_core::VERSION; use openshell_core::config::DEFAULT_STOP_TIMEOUT_SECS; use openshell_core::proto::compute::v1::compute_driver_server::ComputeDriverServer; +use openshell_core::sandbox_env::IdentitySource; use openshell_driver_podman::config::{ DEFAULT_NETWORK_NAME, DEFAULT_SANDBOX_PIDS_LIMIT, ImagePullPolicy, }; @@ -88,6 +89,23 @@ struct Args { )] sandbox_pids_limit: i64, + /// Source for the agent identity: the image's OCI USER or an explicit + /// operator-controlled numeric identity. + #[arg( + long, + env = "OPENSHELL_PODMAN_IDENTITY_SOURCE", + default_value_t = IdentitySource::Image + )] + identity_source: IdentitySource, + + /// Numeric UID for fixed identity mode. + #[arg(long, env = "OPENSHELL_PODMAN_FIXED_UID")] + fixed_uid: Option, + + /// Numeric GID for fixed identity mode. + #[arg(long, env = "OPENSHELL_PODMAN_FIXED_GID")] + fixed_gid: Option, + /// OCI image containing the openshell-sandbox supervisor binary. #[arg(long, env = "OPENSHELL_SUPERVISOR_IMAGE")] supervisor_image: Option, @@ -163,6 +181,9 @@ async fn main() -> Result<()> { guest_tls_cert: args.podman_tls_cert, guest_tls_key: args.podman_tls_key, sandbox_pids_limit: args.sandbox_pids_limit, + identity_source: args.identity_source, + fixed_uid: args.fixed_uid, + fixed_gid: args.fixed_gid, https_proxy: args.sandbox_https_proxy, no_proxy: args.sandbox_no_proxy, proxy_auth_file: args.sandbox_proxy_auth_file, diff --git a/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh b/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh index 14dbc0466b..f29428e77f 100644 --- a/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh +++ b/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh @@ -815,6 +815,10 @@ run_openshell_init_dropins rewrite_openshell_endpoint_if_needed +# Local container drivers own these identity selectors. VM sandboxes retain +# their platform-assigned sandbox UID/GID and must never inherit image metadata. +unset OPENSHELL_IDENTITY_SOURCE OPENSHELL_IMAGE_USER OPENSHELL_IMAGE_ID + # Log supervisor connectivity state for debugging stuck-in-Provisioning issues if [ -n "${OPENSHELL_ENDPOINT:-}" ]; then _ep_parsed="$(parse_endpoint "$OPENSHELL_ENDPOINT" 2>/dev/null || true)" diff --git a/crates/openshell-driver-vm/src/driver.rs b/crates/openshell-driver-vm/src/driver.rs index 7af0ddc389..7f295e8bb7 100644 --- a/crates/openshell-driver-vm/src/driver.rs +++ b/crates/openshell-driver-vm/src/driver.rs @@ -3998,6 +3998,13 @@ fn merged_environment(sandbox: &Sandbox) -> HashMap { if let Some(spec) = sandbox.spec.as_ref() { environment.extend(spec.environment.clone()); } + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ] { + environment.remove(key); + } environment } @@ -5980,6 +5987,69 @@ mod tests { ))); } + #[test] + fn build_guest_environment_removes_local_identity_metadata() { + let protected = [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ]; + let sandbox = Sandbox { + spec: Some(SandboxSpec { + environment: protected + .into_iter() + .map(|key| (key.to_string(), "request-value".to_string())) + .chain(std::iter::once(( + "SAFE_SPEC".to_string(), + "yes".to_string(), + ))) + .collect(), + template: Some(SandboxTemplate { + environment: protected + .into_iter() + .map(|key| (key.to_string(), "image-value".to_string())) + .chain(std::iter::once(( + "SAFE_TEMPLATE".to_string(), + "yes".to_string(), + ))) + .collect(), + ..SandboxTemplate::default() + }), + ..SandboxSpec::default() + }), + ..Sandbox::default() + }; + + let env = build_guest_environment(&sandbox, &VmDriverConfig::default(), None); + + for key in protected { + assert!( + !env.iter() + .any(|value| value.starts_with(&format!("{key}="))) + ); + } + let user_env = env + .iter() + .find_map(|value| { + value + .strip_prefix(&format!( + "{}=", + openshell_core::sandbox_env::USER_ENVIRONMENT + )) + .map(str::to_string) + }) + .expect("safe user environment should be serialized"); + let user_env: HashMap = serde_json::from_str(&user_env).unwrap(); + for key in protected { + assert!(!user_env.contains_key(key)); + } + assert_eq!( + user_env.get("SAFE_TEMPLATE").map(String::as_str), + Some("yes") + ); + assert_eq!(user_env.get("SAFE_SPEC").map(String::as_str), Some("yes")); + } + #[test] fn build_guest_environment_uses_token_file_without_raw_token_env() { let config = VmDriverConfig { diff --git a/crates/openshell-driver-vm/src/runtime.rs b/crates/openshell-driver-vm/src/runtime.rs index f6020af829..f43311e087 100644 --- a/crates/openshell-driver-vm/src/runtime.rs +++ b/crates/openshell-driver-vm/src/runtime.rs @@ -273,6 +273,7 @@ fn write_guest_env_file(overlay_disk: &Path, env_vars: &[String]) -> Result<(), fn qemu_guest_env_vars(config: &VmLaunchConfig, dns_server: Option) -> Vec { let mut env_vars = config.env.clone(); + env_vars.retain(|value| !is_local_identity_metadata(value)); if let Some(ip) = &config.guest_ip && let Some(host_ip) = &config.host_ip @@ -302,6 +303,16 @@ fn shell_escape(s: &str) -> String { .replace('\r', "\\r") } +fn is_local_identity_metadata(value: &str) -> bool { + let key = value.split_once('=').map_or(value, |(key, _)| key); + matches!( + key, + openshell_core::sandbox_env::IDENTITY_SOURCE + | openshell_core::sandbox_env::IMAGE_USER + | openshell_core::sandbox_env::IMAGE_ID + ) +} + fn build_kernel_cmdline(config: &VmLaunchConfig) -> String { let mut parts = vec![ "console=ttyS0".to_string(), @@ -897,7 +908,7 @@ fn libkrun_guest_env(config: &VmLaunchConfig) -> Vec { // executable. OpenShell's guest init is itself an init process and ends // by exec'ing the supervisor, so ask libkrun to exec it directly. Keep // this driver-owned setting authoritative over sandbox image/user env. - env.retain(|value| !value.starts_with("KRUN_INIT_PID1=")); + env.retain(|value| !value.starts_with("KRUN_INIT_PID1=") && !is_local_identity_metadata(value)); env.push(KRUN_INIT_PID1_ENV.to_string()); env } @@ -1477,6 +1488,42 @@ mod tests { assert_eq!(pid_one_settings[0], KRUN_INIT_PID1_ENV); } + #[test] + fn guest_backends_remove_local_identity_metadata() { + let mut config = qemu_config(); + config.env.extend([ + format!("{}=image", openshell_core::sandbox_env::IDENTITY_SOURCE), + format!("{}=sandbox", openshell_core::sandbox_env::IMAGE_USER), + format!("{}=sha256:malicious", openshell_core::sandbox_env::IMAGE_ID), + format!("{}=1500", openshell_core::sandbox_env::SANDBOX_UID), + format!("{}=1600", openshell_core::sandbox_env::SANDBOX_GID), + ]); + + for env in [ + qemu_guest_env_vars(&config, None), + libkrun_guest_env(&config), + ] { + for key in [ + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + ] { + assert!( + !env.iter() + .any(|value| value.starts_with(&format!("{key}="))) + ); + } + assert!(env.contains(&format!( + "{}=1500", + openshell_core::sandbox_env::SANDBOX_UID + ))); + assert!(env.contains(&format!( + "{}=1600", + openshell_core::sandbox_env::SANDBOX_GID + ))); + } + } + #[test] fn libkrun_guest_env_keeps_defaults_when_no_env_is_configured() { let mut config = qemu_config(); diff --git a/crates/openshell-sandbox/src/lib.rs b/crates/openshell-sandbox/src/lib.rs index 3a5c7b70ae..43f351c946 100644 --- a/crates/openshell-sandbox/src/lib.rs +++ b/crates/openshell-sandbox/src/lib.rs @@ -71,6 +71,24 @@ const SIDECAR_CA_BUNDLE: &str = "ca-bundle.pem"; const SIDECAR_PROCESS_PROXY_ADDR: &str = "127.0.0.1:3128"; const SIDECAR_READY_TIMEOUT_SECS: u64 = 120; +fn resolve_identity_if_required( + managed_identity_required: bool, + resolve: impl FnOnce() -> Result>, +) -> Result> { + if !managed_identity_required { + return Ok(None); + } + resolve()?.map_or_else( + || { + Err(miette::miette!( + "gateway requires managed agent identity, but protected {} metadata is missing", + openshell_core::sandbox_env::IDENTITY_SOURCE + )) + }, + |identity| Ok(Some(identity)), + ) +} + /// Run a command in the sandbox. /// /// # Errors @@ -160,6 +178,7 @@ pub async fn run_sandbox( middleware_registry_status, loaded_policy_origin, initial_agent_proposals_enabled, + managed_identity_required, ) = if let Some(bootstrap) = sidecar_bootstrap.as_ref() { let (policy, opa_engine, retained_proto, loaded_policy_origin) = load_policy_from_sidecar_bootstrap(bootstrap)?; @@ -170,6 +189,7 @@ pub async fn run_sandbox( MiddlewareRegistryStatus::Synchronized, loaded_policy_origin, bootstrap.agent_proposals_enabled, + false, ) } else { load_policy( @@ -182,37 +202,100 @@ pub async fn run_sandbox( .await? }; + // Only the gateway-persisted creation marker authorizes protected local + // identity metadata. Unmarked legacy/offline sandboxes deliberately ignore + // image-baked OPENSHELL_IDENTITY_* variables. + #[cfg(unix)] + let resolved_agent_identity = { + let resolved = resolve_identity_if_required(managed_identity_required, || { + openshell_supervisor_process::identity::resolve_and_persist_agent_identity() + })?; + if let Some(identity) = resolved.as_ref() + && identity.source == openshell_core::sandbox_env::IdentitySource::Fixed + && identity.image_id.as_deref().is_none_or(str::is_empty) + { + return Err(miette::miette!( + "managed fixed identity is missing its immutable image ID" + )); + } + resolved + }; + #[cfg(not(unix))] + let resolved_agent_identity: Option = { + if managed_identity_required { + return Err(miette::miette!( + "gateway requires managed agent identity, which is unsupported on this platform" + )); + } + None + }; + + let driver_identity_resolved = if let Some(identity) = resolved_agent_identity.as_ref() { + policy.process.run_as_user = Some(identity.uid.to_string()); + policy.process.run_as_group = Some(identity.gid.to_string()); + ocsf_emit!( + ConfigStateChangeBuilder::new(ocsf_ctx()) + .severity(SeverityId::Informational) + .status(StatusId::Success) + .state(StateId::Enabled, "resolved") + .unmapped( + "resolved_agent_identity", + serde_json::json!({ + "version": openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + "source": identity.source.to_string(), + "image_id": identity.image_id.as_deref(), + "uid": identity.uid, + "gid": identity.gid, + "supplementary_gids": [], + }), + ) + .message(format!( + "Resolved agent identity [source:{} uid:{} gid:{} image_id:{}]", + identity.source, + identity.uid, + identity.gid, + identity.image_id.as_deref().unwrap_or("none") + )) + .build() + ); + true + } else { + false + }; + // Override the policy's process identity with the driver-resolved UID/GID // from the pod environment. The policy defaults to the name "sandbox" which // resolves via /etc/passwd, but the driver may have chosen a different // numeric UID (e.g. from OpenShift SCC annotations). // Validate overrides against the same rules as the policy layer to prevent // env-injected values (e.g. GID 0) from bypassing policy restrictions. - if let Ok(uid) = std::env::var(openshell_core::sandbox_env::SANDBOX_UID) - && !uid.is_empty() - { - if !openshell_policy::is_valid_sandbox_identity(&uid) { - return Err(miette::miette!( - "OPENSHELL_SANDBOX_UID contains invalid sandbox identity '{uid}'; \ - expected 'sandbox' or a numeric UID in range [{}, {}]", - openshell_policy::MIN_SANDBOX_UID, - openshell_policy::MAX_SANDBOX_UID, - )); + if !driver_identity_resolved { + if let Ok(uid) = std::env::var(openshell_core::sandbox_env::SANDBOX_UID) + && !uid.is_empty() + { + if !openshell_policy::is_valid_sandbox_identity(&uid) { + return Err(miette::miette!( + "OPENSHELL_SANDBOX_UID contains invalid sandbox identity '{uid}'; \ + expected 'sandbox' or a numeric UID in range [{}, {}]", + openshell_policy::MIN_SANDBOX_UID, + openshell_policy::MAX_SANDBOX_UID, + )); + } + policy.process.run_as_user = Some(uid); } - policy.process.run_as_user = Some(uid); - } - if let Ok(gid) = std::env::var(openshell_core::sandbox_env::SANDBOX_GID) - && !gid.is_empty() - { - if !openshell_policy::is_valid_sandbox_identity(&gid) { - return Err(miette::miette!( - "OPENSHELL_SANDBOX_GID contains invalid sandbox identity '{gid}'; \ - expected 'sandbox' or a numeric GID in range [{}, {}]", - openshell_policy::MIN_SANDBOX_UID, - openshell_policy::MAX_SANDBOX_UID, - )); + if let Ok(gid) = std::env::var(openshell_core::sandbox_env::SANDBOX_GID) + && !gid.is_empty() + { + if !openshell_policy::is_valid_sandbox_identity(&gid) { + return Err(miette::miette!( + "OPENSHELL_SANDBOX_GID contains invalid sandbox identity '{gid}'; \ + expected 'sandbox' or a numeric GID in range [{}, {}]", + openshell_policy::MIN_SANDBOX_UID, + openshell_policy::MAX_SANDBOX_UID, + )); + } + policy.process.run_as_group = Some(gid); } - policy.process.run_as_group = Some(gid); } #[cfg_attr(not(target_os = "linux"), allow(unused_mut))] @@ -713,6 +796,8 @@ pub async fn run_sandbox( provider_env, ca_file_paths, agent_proposals.clone(), + managed_identity_required, + resolved_agent_identity, #[cfg(target_os = "linux")] netns.as_ref(), #[cfg(target_os = "linux")] @@ -984,12 +1069,13 @@ fn spawn_sidecar_entrypoint_handler( ); continue; }; - openshell_supervisor_process::supervisor_session::spawn( + let _ = openshell_supervisor_process::supervisor_session::spawn( endpoint.clone(), id.clone(), trusted_ssh_socket_path.clone(), None, Some(supervisor_pid), + None, ); session_started = true; info!("sidecar supervisor session task spawned"); @@ -1849,6 +1935,7 @@ async fn load_policy( MiddlewareRegistryStatus, LoadedPolicyOrigin, bool, + bool, )> { // File mode: load OPA engine from rego rules + YAML data (dev override) if let (Some(policy_file), Some(data_file)) = (&policy_rules, &policy_data) { @@ -1898,6 +1985,7 @@ async fn load_policy( MiddlewareRegistryStatus::Synchronized, LoadedPolicyOrigin::LocalOverride, false, + false, )); } @@ -2080,6 +2168,7 @@ async fn load_policy( revision: loaded_policy_revision, }, agent_proposals_enabled_from_settings(&snapshot.settings), + snapshot.managed_identity_required, )); } @@ -3448,6 +3537,15 @@ filesystem_policy: assert!(matches!(local_policy.network.mode, NetworkMode::Proxy)); } + #[test] + fn unmarked_sandbox_does_not_consult_protected_identity_metadata() { + let resolved = resolve_identity_if_required(false, || -> Result> { + panic!("unmarked sandbox must not inspect identity metadata") + }) + .unwrap(); + assert!(resolved.is_none()); + } + // ---- Initial policy acknowledgement tests ---- fn proto_policy_fixture() -> openshell_core::proto::SandboxPolicy { @@ -3470,6 +3568,7 @@ filesystem_policy: provider_env_revision: 0, supervisor_middleware_services: Vec::new(), workspace: String::new(), + managed_identity_required: false, } } diff --git a/crates/openshell-server/src/compute/mod.rs b/crates/openshell-server/src/compute/mod.rs index cf6b17c7e9..8c7a39c62f 100644 --- a/crates/openshell-server/src/compute/mod.rs +++ b/crates/openshell-server/src/compute/mod.rs @@ -2431,11 +2431,21 @@ fn public_status_from_driver( .collect(), phase: phase as i32, current_policy_version, + resolved_identity: None, + managed_identity_required: false, } } fn apply_driver_snapshot(sandbox: &mut Sandbox, incoming: &DriverSandbox, session_connected: bool) { let old_phase = SandboxPhase::try_from(sandbox.phase()).unwrap_or(SandboxPhase::Unknown); + let resolved_identity = sandbox + .status + .as_ref() + .and_then(|status| status.resolved_identity.clone()); + let managed_identity_required = sandbox + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required); let mut phase = incoming .status .as_ref() @@ -2463,6 +2473,10 @@ fn apply_driver_snapshot(sandbox: &mut Sandbox, incoming: &DriverSandbox, sessio { status.sandbox_name.clone_from(sandbox_name); } + if let Some(status) = status.as_mut() { + status.resolved_identity = resolved_identity; + status.managed_identity_required = managed_identity_required; + } if old_phase != phase { info!( @@ -2742,11 +2756,20 @@ impl ComputeDriver for NoopTestDriver { } #[cfg(test)] -pub async fn new_test_runtime(store: Arc) -> ComputeRuntime { +pub fn new_test_runtime(store: Arc) -> ComputeRuntime { + new_test_runtime_with_driver_kind(store, None) +} + +#[cfg(test)] +pub fn new_test_runtime_with_driver_kind( + store: Arc, + driver_kind: Option, +) -> ComputeRuntime { + let driver_name = driver_kind.map_or_else(|| "test".to_string(), |kind| kind.as_str().into()); ComputeRuntime { driver: Arc::new(NoopTestDriver), driver_info: ComputeDriverInfoSnapshot { - name: "test".to_string(), + name: driver_name, driver_name: "test".to_string(), driver_version: "test".to_string(), }, @@ -3574,6 +3597,41 @@ mod tests { } } + #[test] + fn driver_snapshot_preserves_persisted_resolved_identity() { + let identity = openshell_core::proto::ResolvedAgentIdentity { + version: openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + source: "image".to_string(), + image_id: "sha256:image".to_string(), + uid: 1234, + gid: 2345, + supplementary_gids: Vec::new(), + }; + let mut sandbox = sandbox_record("sb-identity", "identity", SandboxPhase::Provisioning); + sandbox.status.as_mut().unwrap().resolved_identity = Some(identity.clone()); + sandbox.status.as_mut().unwrap().managed_identity_required = true; + + apply_driver_snapshot( + &mut sandbox, + &ready_driver_sandbox("sb-identity", "identity"), + true, + ); + + assert_eq!( + sandbox + .status + .as_ref() + .and_then(|status| status.resolved_identity.as_ref()), + Some(&identity) + ); + assert!( + sandbox + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required) + ); + } + #[test] fn derive_phase_returns_unknown_without_status() { assert_eq!(derive_phase(None), SandboxPhase::Unknown); diff --git a/crates/openshell-server/src/grpc/auth_rpc.rs b/crates/openshell-server/src/grpc/auth_rpc.rs index f4cb6a872a..6fab01365d 100644 --- a/crates/openshell-server/src/grpc/auth_rpc.rs +++ b/crates/openshell-server/src/grpc/auth_rpc.rs @@ -167,7 +167,7 @@ mod tests { .await .unwrap(), ); - let compute = new_test_runtime(store.clone()).await; + let compute = new_test_runtime(store.clone()); let mut state = ServerState::new( Config::new(None).with_database_url("sqlite::memory:?cache=shared"), store, @@ -346,7 +346,7 @@ mod tests { .await .unwrap(), ); - let compute = new_test_runtime(store.clone()).await; + let compute = new_test_runtime(store.clone()); let state = Arc::new(ServerState::new( Config::new(None).with_database_url("sqlite::memory:?cache=shared"), store, diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index ecd96ea90d..53751c7b45 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -805,7 +805,7 @@ pub mod test_support { .unwrap(), ); crate::ensure_default_workspace(&store).await.unwrap(); - let compute = new_test_runtime(store.clone()).await; + let compute = new_test_runtime(store.clone()); Arc::new(ServerState::new( Config::new(None).with_database_url("sqlite::memory:?cache=shared"), store, diff --git a/crates/openshell-server/src/grpc/policy.rs b/crates/openshell-server/src/grpc/policy.rs index 53124261e6..75ea43d163 100644 --- a/crates/openshell-server/src/grpc/policy.rs +++ b/crates/openshell-server/src/grpc/policy.rs @@ -1447,6 +1447,14 @@ pub(super) async fn handle_get_sandbox_config( provider_env_revision, supervisor_middleware_services, workspace, + resolved_identity: sandbox + .status + .as_ref() + .and_then(|status| status.resolved_identity.clone()), + managed_identity_required: sandbox + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required), })) } @@ -4828,7 +4836,7 @@ mod tests { #[tokio::test] async fn same_sandbox_get_sandbox_config_allowed() { - use openshell_core::proto::{SandboxPhase, SandboxSpec}; + use openshell_core::proto::{ResolvedAgentIdentity, SandboxPhase, SandboxSpec}; let state = test_server_state().await; let mut sandbox = Sandbox { metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { @@ -4848,6 +4856,16 @@ mod tests { ..Default::default() }; sandbox.set_phase(SandboxPhase::Provisioning as i32); + let identity = ResolvedAgentIdentity { + version: openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + source: "image".to_string(), + image_id: "sha256:image".to_string(), + uid: 1234, + gid: 2345, + supplementary_gids: Vec::new(), + }; + sandbox.status.as_mut().unwrap().resolved_identity = Some(identity.clone()); + sandbox.status.as_mut().unwrap().managed_identity_required = true; state.store.put_message(&sandbox).await.unwrap(); let req = with_sandbox( Request::new(GetSandboxConfigRequest { @@ -4855,9 +4873,12 @@ mod tests { }), "sb-self", ); - handle_get_sandbox_config(&state, req) + let response = handle_get_sandbox_config(&state, req) .await - .expect("matching principal must be allowed"); + .expect("matching principal must be allowed") + .into_inner(); + assert_eq!(response.resolved_identity, Some(identity)); + assert!(response.managed_identity_required); } #[tokio::test] diff --git a/crates/openshell-server/src/grpc/sandbox.rs b/crates/openshell-server/src/grpc/sandbox.rs index 57bf421b88..792b34fd4b 100644 --- a/crates/openshell-server/src/grpc/sandbox.rs +++ b/crates/openshell-server/src/grpc/sandbox.rs @@ -55,6 +55,39 @@ use crate::persistence::current_time_ms; const TCP_FORWARD_CHUNK_SIZE: usize = 64 * 1024; +fn managed_local_driver(driver_kind: Option) -> bool { + matches!( + driver_kind, + Some(openshell_core::ComputeDriverKind::Docker | openshell_core::ComputeDriverKind::Podman) + ) +} + +fn validate_creator_process_identity( + driver_kind: Option, + policy: Option<&openshell_core::proto::SandboxPolicy>, +) -> Result<(), Status> { + if !managed_local_driver(driver_kind) { + return Ok(()); + } + let Some(process) = policy.and_then(|policy| policy.process.as_ref()) else { + return Ok(()); + }; + if !process.run_as_user.is_empty() || !process.run_as_group.is_empty() { + return Err(Status::invalid_argument( + "process.run_as_user and process.run_as_group are managed by the Docker/Podman runtime and must be omitted", + )); + } + Ok(()) +} + +fn initialize_sandbox_status(sandbox: &mut Sandbox, managed_identity_required: bool) { + sandbox.set_phase(SandboxPhase::Provisioning as i32); + sandbox + .status + .get_or_insert_default() + .managed_identity_required = managed_identity_required; +} + fn generate_routable_name() -> String { let name = petname::petname(2, "-").unwrap_or_else(generate_name); let mut truncated = &name[..name.len().min(MAX_ROUTABLE_NAME_LEN)]; @@ -135,6 +168,8 @@ async fn handle_create_sandbox_inner( // Validate field sizes before any I/O (fail fast on oversized payloads). validate_sandbox_spec(&request.name, &spec)?; + let managed_identity_required = managed_local_driver(state.compute.driver_kind()); + validate_creator_process_identity(state.compute.driver_kind(), spec.policy.as_ref())?; // Validate labels (keys and values must meet Kubernetes requirements). for (key, value) in &request.labels { @@ -204,7 +239,7 @@ async fn handle_create_sandbox_inner( spec: Some(spec), status: None, }; - sandbox.set_phase(SandboxPhase::Provisioning as i32); + initialize_sandbox_status(&mut sandbox, managed_identity_required); // Ensure metadata is valid (defense in depth - should always be true for server-constructed metadata) super::validation::validate_object_metadata(sandbox.metadata.as_ref(), "sandbox")?; @@ -2109,6 +2144,43 @@ mod tests { use crate::grpc::test_support::test_server_state; use openshell_core::proto::datamodel::v1::ObjectMeta; + async fn test_server_state_with_driver_kind( + driver_kind: Option, + ) -> Arc { + let store = Arc::new( + crate::persistence::Store::connect("sqlite::memory:?cache=shared") + .await + .unwrap(), + ); + crate::ensure_default_workspace(&store).await.unwrap(); + let compute = crate::compute::new_test_runtime_with_driver_kind(store.clone(), driver_kind); + Arc::new(ServerState::new( + openshell_core::Config::new(None).with_database_url("sqlite::memory:?cache=shared"), + store, + compute, + crate::sandbox_index::SandboxIndex::new(), + crate::sandbox_watch::SandboxWatchBus::new(), + crate::tracing_bus::TracingLogBus::new(), + Arc::new(crate::supervisor_session::SupervisorSessionRegistry::new()), + None, + )) + } + + fn with_sandbox_principal(mut request: Request, sandbox_id: &str) -> Request { + request + .extensions_mut() + .insert(crate::auth::principal::Principal::Sandbox( + crate::auth::principal::SandboxPrincipal { + sandbox_id: sandbox_id.to_string(), + source: crate::auth::principal::SandboxIdentitySource::BootstrapJwt { + issuer: "openshell-gateway:test".to_string(), + }, + trust_domain: None, + }, + )); + request + } + // ---- shell_escape ---- #[test] @@ -2402,6 +2474,143 @@ mod tests { } } + #[test] + fn managed_local_create_rejects_creator_process_identity_before_normalization() { + let policy = openshell_core::proto::SandboxPolicy { + process: Some(openshell_core::proto::ProcessPolicy { + run_as_user: "agent".to_string(), + run_as_group: String::new(), + }), + ..Default::default() + }; + + for driver in [ + openshell_core::ComputeDriverKind::Docker, + openshell_core::ComputeDriverKind::Podman, + ] { + let error = validate_creator_process_identity(Some(driver), Some(&policy)) + .expect_err("managed local create must reject creator identity"); + assert_eq!(error.code(), tonic::Code::InvalidArgument); + } + } + + #[test] + fn creator_process_identity_is_preserved_outside_managed_local_create() { + let policy = openshell_core::proto::SandboxPolicy { + process: Some(openshell_core::proto::ProcessPolicy { + run_as_user: "sandbox".to_string(), + run_as_group: "sandbox".to_string(), + }), + ..Default::default() + }; + + for driver in [ + Some(openshell_core::ComputeDriverKind::Kubernetes), + Some(openshell_core::ComputeDriverKind::Vm), + None, + ] { + validate_creator_process_identity(driver, Some(&policy)) + .expect("non-local and custom drivers retain creator identity"); + } + } + + #[test] + fn managed_local_create_allows_omitted_process_identity() { + let policy = openshell_core::proto::SandboxPolicy { + process: Some(openshell_core::proto::ProcessPolicy::default()), + ..Default::default() + }; + + validate_creator_process_identity( + Some(openshell_core::ComputeDriverKind::Docker), + Some(&policy), + ) + .expect("normalization must receive an omitted identity"); + } + + #[test] + fn create_status_persists_managed_local_identity_requirement() { + let mut managed = Sandbox::default(); + initialize_sandbox_status(&mut managed, true); + let status = managed.status.unwrap(); + assert_eq!(status.phase, SandboxPhase::Provisioning as i32); + assert!(status.managed_identity_required); + + let mut legacy = Sandbox::default(); + initialize_sandbox_status(&mut legacy, false); + assert!(!legacy.status.unwrap().managed_identity_required); + } + + #[tokio::test] + async fn create_and_config_managed_identity_marker_follow_driver_kind_matrix() { + let cases = [ + (Some(openshell_core::ComputeDriverKind::Docker), true), + (Some(openshell_core::ComputeDriverKind::Podman), true), + (Some(openshell_core::ComputeDriverKind::Kubernetes), false), + (Some(openshell_core::ComputeDriverKind::Vm), false), + (None, false), + ]; + + for (driver_kind, expected) in cases { + let state = test_server_state_with_driver_kind(driver_kind).await; + let name = driver_kind.map_or("custom", openshell_core::ComputeDriverKind::as_str); + let created = handle_create_sandbox( + &state, + Request::new(CreateSandboxRequest { + name: format!("id-{name}"), + spec: Some(openshell_core::proto::SandboxSpec::default()), + ..Default::default() + }), + ) + .await + .expect("test driver create must succeed") + .into_inner() + .sandbox + .expect("create response must contain sandbox"); + + assert_eq!( + created + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required), + expected, + "unexpected create marker for {name}" + ); + let sandbox_id = created.object_id().to_string(); + let stored = state + .store + .get_message::(&sandbox_id) + .await + .unwrap() + .unwrap(); + assert_eq!( + stored + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required), + expected, + "unexpected persisted marker for {name}" + ); + + let config = crate::grpc::policy::handle_get_sandbox_config( + &state, + with_sandbox_principal( + Request::new(openshell_core::proto::GetSandboxConfigRequest { + sandbox_id: sandbox_id.clone(), + }), + &sandbox_id, + ), + ) + .await + .expect("sandbox config fetch must succeed") + .into_inner(); + assert_eq!( + config.managed_identity_required, expected, + "unexpected config marker for {name}" + ); + } + } + fn test_provider(name: &str, provider_type: &str) -> Provider { test_provider_with_credential_key(name, provider_type, "TOKEN") } diff --git a/crates/openshell-server/src/lib.rs b/crates/openshell-server/src/lib.rs index a75534d397..1e569081fd 100644 --- a/crates/openshell-server/src/lib.rs +++ b/crates/openshell-server/src/lib.rs @@ -1078,7 +1078,7 @@ mod tests { .await .expect("failed to create test store"), ); - let compute = crate::compute::new_test_runtime(store.clone()).await; + let compute = crate::compute::new_test_runtime(store.clone()); Arc::new(ServerState::new( Config::new(None) .with_database_url("sqlite::memory:?cache=shared") diff --git a/crates/openshell-server/src/supervisor_session.rs b/crates/openshell-server/src/supervisor_session.rs index a839791ad4..dfc97aa108 100644 --- a/crates/openshell-server/src/supervisor_session.rs +++ b/crates/openshell-server/src/supervisor_session.rs @@ -13,8 +13,9 @@ use tracing::{info, warn}; use uuid::Uuid; use openshell_core::proto::{ - GatewayMessage, RelayFrame, RelayInit, RelayOpen, Sandbox, SessionAccepted, SshRelayTarget, - SupervisorMessage, gateway_message, relay_open, supervisor_message, + GatewayMessage, RelayFrame, RelayInit, RelayOpen, ResolvedAgentIdentity, Sandbox, + SessionAccepted, SshRelayTarget, SupervisorMessage, gateway_message, relay_open, + supervisor_message, }; use crate::ServerState; @@ -22,6 +23,7 @@ use crate::auth::principal::Principal; const HEARTBEAT_INTERVAL_SECS: u32 = 15; const RELAY_PENDING_TIMEOUT: Duration = Duration::from_secs(10); +const IDENTITY_CAS_RETRY_LIMIT: usize = 4; /// Initial backoff between session-availability polls in `wait_for_session`. const SESSION_WAIT_INITIAL_BACKOFF: Duration = Duration::from_millis(100); /// Maximum backoff between session-availability polls in `wait_for_session`. @@ -437,17 +439,126 @@ pub fn spawn_relay_reaper(state: Arc, interval: Duration) { async fn require_persisted_sandbox( store: &Arc, sandbox_id: &str, -) -> Result<(), Status> { +) -> Result { let sandbox = store .get_message::(sandbox_id) .await .map_err(|err| Status::internal(format!("failed to load sandbox: {err}")))?; - if sandbox.is_none() { - return Err(Status::not_found("sandbox not found")); + sandbox.ok_or_else(|| Status::not_found("sandbox not found")) +} + +fn managed_identity_required(sandbox: &Sandbox) -> bool { + sandbox + .status + .as_ref() + .is_some_and(|status| status.managed_identity_required) +} + +fn validate_resolved_identity(identity: &ResolvedAgentIdentity) -> Result<(), Status> { + if identity.version != openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION { + return Err(Status::invalid_argument(format!( + "unsupported resolved identity version {}; expected {}", + identity.version, + openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION + ))); + } + if identity.uid == 0 || identity.gid == 0 { + return Err(Status::invalid_argument( + "resolved identity UID and GID must be nonzero", + )); + } + if !identity.supplementary_gids.is_empty() { + return Err(Status::invalid_argument( + "managed resolved identity must not contain supplementary groups", + )); + } + match identity.source.as_str() { + "image" | "fixed" if identity.image_id.is_empty() => Err(Status::invalid_argument( + "managed resolved identity requires an immutable image ID", + )), + "image" | "fixed" => Ok(()), + _ => Err(Status::invalid_argument( + "resolved identity source must be 'image' or 'fixed'", + )), + } +} + +async fn validate_and_persist_supervisor_identity( + state: &Arc, + sandbox_id: &str, + incoming: Option<&ResolvedAgentIdentity>, +) -> Result<(), Status> { + let _guard = state.compute.sandbox_sync_guard().await; + for _ in 0..IDENTITY_CAS_RETRY_LIMIT { + let sandbox = require_persisted_sandbox(&state.store, sandbox_id).await?; + let identity_required = managed_identity_required(&sandbox); + let stored = sandbox + .status + .as_ref() + .and_then(|status| status.resolved_identity.as_ref()); + + if !identity_required { + if incoming.is_some() { + return Err(Status::failed_precondition( + "sandbox does not permit a managed resolved identity", + )); + } + if stored.is_some() { + return Err(Status::failed_precondition( + "sandbox has a persisted identity without the managed identity requirement", + )); + } + return Ok(()); + } + + let Some(identity) = incoming else { + return Err(Status::failed_precondition( + "managed sandbox supervisor hello requires a resolved identity", + )); + }; + validate_resolved_identity(identity)?; + + if let Some(stored) = stored { + if incoming != Some(stored) { + return Err(Status::failed_precondition( + "supervisor resolved identity does not match the persisted identity", + )); + } + return Ok(()); + } + + let expected_resource_version = sandbox + .metadata + .as_ref() + .map_or(0, |metadata| metadata.resource_version); + let identity = identity.clone(); + match state + .store + .update_message_cas::(sandbox_id, expected_resource_version, |sandbox| { + sandbox.status.get_or_insert_default().resolved_identity = Some(identity.clone()); + }) + .await + { + Ok(updated) => { + state.sandbox_index.update_from_sandbox(&updated); + state.sandbox_watch_bus.notify(sandbox_id); + return Ok(()); + } + Err(crate::persistence::PersistenceError::Conflict { .. }) => { + tokio::task::yield_now().await; + } + Err(error) => { + return Err(Status::internal(format!( + "failed to persist resolved identity: {error}" + ))); + } + } } - Ok(()) + Err(Status::aborted( + "resolved identity persistence conflicted with concurrent sandbox updates", + )) } // --------------------------------------------------------------------------- @@ -598,7 +709,8 @@ pub async fn handle_connect_supervisor( if let Some(principal) = principal.as_ref() { crate::auth::guard::ensure_sandbox_principal_scope(principal, &sandbox_id)?; } - require_persisted_sandbox(&state.store, &sandbox_id).await?; + validate_and_persist_supervisor_identity(state, &sandbox_id, hello.resolved_identity.as_ref()) + .await?; let session_id = Uuid::new_v4().to_string(); info!( @@ -849,6 +961,35 @@ mod tests { } } + fn image_identity() -> ResolvedAgentIdentity { + ResolvedAgentIdentity { + version: openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + source: "image".to_string(), + image_id: "sha256:image".to_string(), + uid: 1234, + gid: 2345, + supplementary_gids: Vec::new(), + } + } + + fn fixed_identity() -> ResolvedAgentIdentity { + ResolvedAgentIdentity { + version: openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + source: "fixed".to_string(), + image_id: "sha256:fixed-image".to_string(), + uid: 1234, + gid: 2345, + supplementary_gids: Vec::new(), + } + } + + fn managed_sandbox_record(id: &str, name: &str) -> Sandbox { + let mut sandbox = sandbox_record(id, name); + sandbox.set_phase(openshell_core::proto::SandboxPhase::Provisioning as i32); + sandbox.status.as_mut().unwrap().managed_identity_required = true; + sandbox + } + fn pending_relay( sandbox_id: &str, relay_tx: RelayStreamSender, @@ -1287,6 +1428,181 @@ mod tests { .expect("persisted sandbox should be accepted"); } + #[test] + fn resolved_identity_validation_enforces_managed_contract() { + let valid = image_identity(); + validate_resolved_identity(&valid).unwrap(); + + let invalid = [ + ResolvedAgentIdentity { + version: 2, + ..valid.clone() + }, + ResolvedAgentIdentity { + source: "unknown".to_string(), + ..valid.clone() + }, + ResolvedAgentIdentity { + uid: 0, + ..valid.clone() + }, + ResolvedAgentIdentity { + gid: 0, + ..valid.clone() + }, + ResolvedAgentIdentity { + supplementary_gids: vec![3456], + ..valid.clone() + }, + ResolvedAgentIdentity { + image_id: String::new(), + ..valid.clone() + }, + ResolvedAgentIdentity { + source: "fixed".to_string(), + image_id: String::new(), + ..valid.clone() + }, + ResolvedAgentIdentity { + source: "fixed-unknown".to_string(), + ..valid + }, + ]; + + for identity in invalid { + let error = validate_resolved_identity(&identity) + .expect_err("invalid identity must be rejected"); + assert_eq!(error.code(), tonic::Code::InvalidArgument); + } + + validate_resolved_identity(&ResolvedAgentIdentity { + version: openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION, + source: "fixed".to_string(), + image_id: "sha256:fixed-image".to_string(), + uid: 1234, + gid: 2345, + supplementary_gids: Vec::new(), + }) + .expect("fixed identity carries its immutable image ID"); + } + + #[tokio::test] + async fn first_managed_hello_persists_identity_and_restart_requires_exact_equality() { + let state = crate::grpc::test_support::test_server_state().await; + let sandbox = managed_sandbox_record("sbx-identity", "identity"); + state.store.put_message(&sandbox).await.unwrap(); + + let identity = image_identity(); + validate_and_persist_supervisor_identity(&state, "sbx-identity", Some(&identity)) + .await + .expect("first hello must persist before acceptance"); + + let stored = state + .store + .get_message::("sbx-identity") + .await + .unwrap() + .unwrap(); + assert_eq!( + stored + .status + .as_ref() + .and_then(|status| status.resolved_identity.as_ref()), + Some(&identity) + ); + + validate_and_persist_supervisor_identity(&state, "sbx-identity", Some(&identity)) + .await + .expect("identical restart identity must be accepted"); + + let mut changed = identity.clone(); + changed.uid += 1; + let error = + validate_and_persist_supervisor_identity(&state, "sbx-identity", Some(&changed)) + .await + .expect_err("changed restart identity must be rejected"); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + + let error = validate_and_persist_supervisor_identity(&state, "sbx-identity", None) + .await + .expect_err("restart must report persisted identity"); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + } + + #[tokio::test] + async fn fixed_identity_restart_compares_immutable_image_id() { + let state = crate::grpc::test_support::test_server_state().await; + state + .store + .put_message(&managed_sandbox_record("sbx-fixed", "fixed")) + .await + .unwrap(); + + let identity = fixed_identity(); + validate_and_persist_supervisor_identity(&state, "sbx-fixed", Some(&identity)) + .await + .expect("fixed identity with image ID must persist"); + + let mut changed = identity; + changed.image_id = "sha256:other-image".to_string(); + let error = validate_and_persist_supervisor_identity(&state, "sbx-fixed", Some(&changed)) + .await + .expect_err("fixed identity image ID must be restart-stable"); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + } + + #[tokio::test] + async fn unmarked_kubernetes_vm_and_legacy_sandboxes_reject_supplied_identity() { + let state = crate::grpc::test_support::test_server_state().await; + let identity = image_identity(); + for (id, name) in [ + ("sbx-kubernetes", "kubernetes"), + ("sbx-vm", "vm"), + ("sbx-legacy", "legacy"), + ] { + state + .store + .put_message(&sandbox_record(id, name)) + .await + .unwrap(); + validate_and_persist_supervisor_identity(&state, id, None) + .await + .expect("unmarked sandbox may omit identity"); + let error = validate_and_persist_supervisor_identity(&state, id, Some(&identity)) + .await + .expect_err("unmarked sandbox must reject supplied identity"); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + let stored = state + .store + .get_message::(id) + .await + .unwrap() + .unwrap(); + assert!( + stored + .status + .as_ref() + .and_then(|status| status.resolved_identity.as_ref()) + .is_none(), + "rejected identity must not be persisted for {name}" + ); + } + } + + #[tokio::test] + async fn marked_sandbox_rejects_missing_identity() { + let state = crate::grpc::test_support::test_server_state().await; + state + .store + .put_message(&managed_sandbox_record("sbx-managed", "managed")) + .await + .unwrap(); + let error = validate_and_persist_supervisor_identity(&state, "sbx-managed", None) + .await + .expect_err("new managed sandbox must report identity"); + assert_eq!(error.code(), tonic::Code::FailedPrecondition); + } + // ---- claim_relay: expiry, drop, wiring ---- #[test] diff --git a/crates/openshell-supervisor-process/Cargo.toml b/crates/openshell-supervisor-process/Cargo.toml index 3c4be356f1..5ed003740d 100644 --- a/crates/openshell-supervisor-process/Cargo.toml +++ b/crates/openshell-supervisor-process/Cargo.toml @@ -33,7 +33,7 @@ uuid = { workspace = true } [target.'cfg(unix)'.dependencies] libc = "0.2" -rustix = { workspace = true } +rustix = { workspace = true, features = ["fs", "thread"] } [target.'cfg(target_os = "linux")'.dependencies] capctl = "0.2.4" diff --git a/crates/openshell-supervisor-process/src/child_env.rs b/crates/openshell-supervisor-process/src/child_env.rs index 32eecbee35..b49a7b4082 100644 --- a/crates/openshell-supervisor-process/src/child_env.rs +++ b/crates/openshell-supervisor-process/src/child_env.rs @@ -3,8 +3,45 @@ use std::path::Path; +use openshell_core::policy::SandboxPolicy; + const LOCAL_NO_PROXY: &str = "127.0.0.1,localhost,::1"; +/// Identity environment shared by direct and SSH-launched agent children. +/// +/// Image and fixed identities are normalized into numeric policy fields before +/// process preparation, while the separately resolved presentation name is +/// retained in supervisor memory and never exposed as protected metadata. +pub fn identity_env_vars(policy: &SandboxPolicy) -> [(&'static str, String); 3] { + #[cfg(unix)] + let presentation_user = crate::identity::resolved_presentation_user(); + #[cfg(not(unix))] + let presentation_user = None; + identity_env_vars_with_presentation(policy, presentation_user) +} + +fn identity_env_vars_with_presentation( + policy: &SandboxPolicy, + presentation_user: Option<&str>, +) -> [(&'static str, String); 3] { + let user = presentation_user + .filter(|user| !user.is_empty()) + .or_else(|| { + policy + .process + .run_as_user + .as_deref() + .filter(|user| !user.is_empty()) + }) + .unwrap_or("sandbox") + .to_string(); + [ + ("HOME", "/sandbox".to_string()), + ("USER", user.clone()), + ("LOGNAME", user), + ] +} + pub fn proxy_env_vars(proxy_url: &str) -> [(&'static str, String); 9] { [ ("ALL_PROXY", proxy_url.to_owned()), @@ -45,6 +82,52 @@ mod tests { use std::process::Command; use std::process::Stdio; + #[test] + fn identity_env_uses_sandbox_home_and_numeric_presentation() { + let policy = SandboxPolicy { + version: 1, + filesystem: openshell_core::policy::FilesystemPolicy::default(), + network: openshell_core::policy::NetworkPolicy::default(), + landlock: openshell_core::policy::LandlockPolicy::default(), + process: openshell_core::policy::ProcessPolicy { + run_as_user: Some("1234".into()), + run_as_group: Some("2345".into()), + }, + }; + + assert_eq!( + identity_env_vars(&policy), + [ + ("HOME", "/sandbox".to_string()), + ("USER", "1234".to_string()), + ("LOGNAME", "1234".to_string()), + ] + ); + } + + #[test] + fn identity_env_preserves_resolved_presentation_user() { + let policy = SandboxPolicy { + version: 1, + filesystem: openshell_core::policy::FilesystemPolicy::default(), + network: openshell_core::policy::NetworkPolicy::default(), + landlock: openshell_core::policy::LandlockPolicy::default(), + process: openshell_core::policy::ProcessPolicy { + run_as_user: Some("1234".into()), + run_as_group: Some("2345".into()), + }, + }; + + assert_eq!( + identity_env_vars_with_presentation(&policy, Some("app")), + [ + ("HOME", "/sandbox".to_string()), + ("USER", "app".to_string()), + ("LOGNAME", "app".to_string()), + ] + ); + } + #[test] fn apply_proxy_env_includes_node_proxy_opt_in_and_local_bypass() { let mut cmd = Command::new("/usr/bin/env"); diff --git a/crates/openshell-supervisor-process/src/identity.rs b/crates/openshell-supervisor-process/src/identity.rs new file mode 100644 index 0000000000..e0fb67828b --- /dev/null +++ b/crates/openshell-supervisor-process/src/identity.rs @@ -0,0 +1,1641 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Safe image identity resolution and creation-time persistence. + +use miette::{Context, IntoDiagnostic, Result}; +use openshell_core::sandbox_env::{IdentitySource, ResolvedAgentIdentity}; +use rustix::fs::{AtFlags, Mode, OFlags, RenameFlags}; +use std::ffi::{OsStr, OsString}; +use std::fs::File; +use std::io::{Read, Write}; +use std::os::fd::OwnedFd; +use std::os::unix::fs::MetadataExt; +#[cfg(any(test, target_os = "linux"))] +use std::path::PathBuf; +use std::path::{Component, Path}; +use std::sync::OnceLock; + +const PASSWD_PATH: &str = "/etc/passwd"; +const GROUP_PATH: &str = "/etc/group"; +pub const IDENTITY_STATE_PATH: &str = "/var/lib/openshell/agent-identity.json"; +const IDENTITY_STATE_VERSION: u32 = 1; +const MAX_ACCOUNT_FILE_SIZE: u64 = 1024 * 1024; +const MAX_ACCOUNT_LINE_SIZE: usize = 8 * 1024; +const MAX_ACCOUNT_FIELD_SIZE: usize = 1024; +const MAX_STATE_FILE_SIZE: u64 = 16 * 1024; +#[cfg(target_os = "linux")] +const MAX_ID_MAP_FILE_SIZE: u64 = 64 * 1024; + +static RESOLVED_IDENTITY: OnceLock = OnceLock::new(); + +#[derive(Debug, Clone, PartialEq, Eq)] +enum IdentityRequest { + Image { + image_user: String, + image_id: String, + }, + Fixed { + uid: u32, + gid: u32, + image_id: String, + }, +} + +impl IdentityRequest { + fn from_env() -> Result> { + let Some(source) = std::env::var_os(openshell_core::sandbox_env::IDENTITY_SOURCE) else { + return Ok(None); + }; + let source = source + .into_string() + .map_err(|_| miette::miette!("OPENSHELL_IDENTITY_SOURCE is not valid UTF-8"))? + .parse::() + .map_err(|error| miette::miette!(error))?; + + match source { + IdentitySource::Image => { + let image_user = required_env(openshell_core::sandbox_env::IMAGE_USER)?; + let image_id = required_env(openshell_core::sandbox_env::IMAGE_ID)?; + validate_metadata_value(openshell_core::sandbox_env::IMAGE_ID, &image_id)?; + Ok(Some(Self::Image { + image_user, + image_id, + })) + } + IdentitySource::Fixed => { + let image_id = required_env(openshell_core::sandbox_env::IMAGE_ID)?; + validate_metadata_value(openshell_core::sandbox_env::IMAGE_ID, &image_id)?; + let uid = parse_fixed_id( + openshell_core::sandbox_env::SANDBOX_UID, + &required_env(openshell_core::sandbox_env::SANDBOX_UID)?, + )?; + let gid = parse_fixed_id( + openshell_core::sandbox_env::SANDBOX_GID, + &required_env(openshell_core::sandbox_env::SANDBOX_GID)?, + )?; + Ok(Some(Self::Fixed { uid, gid, image_id })) + } + } + } + + const fn source(&self) -> IdentitySource { + match self { + Self::Image { .. } => IdentitySource::Image, + Self::Fixed { .. } => IdentitySource::Fixed, + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct PersistedIdentity { + version: u32, + identity: ResolvedAgentIdentity, + image_user: Option, +} + +#[derive(Debug, Clone)] +struct PasswdEntry { + name: String, + uid: u32, + gid: u32, +} + +#[derive(Debug, Clone)] +struct GroupEntry { + name: String, + gid: u32, +} + +/// Resolve or reload the protected agent identity metadata. +/// +/// `None` means `OPENSHELL_IDENTITY_SOURCE` was absent and callers must retain +/// the legacy policy identity behavior. +pub fn resolve_and_persist_agent_identity() -> Result> { + let request = IdentityRequest::from_env()?; + let identity = resolve_and_persist_at( + request.as_ref(), + Path::new(PASSWD_PATH), + Path::new(GROUP_PATH), + Path::new(IDENTITY_STATE_PATH), + true, + |identity| validate_user_namespace_mappings(identity.uid, identity.gid), + )?; + if let Some(identity) = identity.as_ref() { + let installed = RESOLVED_IDENTITY.get_or_init(|| identity.clone()); + if installed != identity { + return Err(miette::miette!( + "resolved agent identity changed after initialization" + )); + } + } + Ok(identity) +} + +/// Presentation name selected during image/fixed identity resolution. +#[must_use] +pub fn resolved_presentation_user() -> Option<&'static str> { + RESOLVED_IDENTITY + .get() + .map(|identity| identity.presentation_user.as_str()) +} + +/// Resolve an OCI `USER` declaration using alternate account-file paths. +/// +/// This helper never invokes libc NSS and is suitable for focused tests or +/// callers resolving an unpacked rootfs. +pub fn resolve_image_identity_at( + image_user: &str, + image_id: &str, + passwd_path: &Path, + group_path: &Path, +) -> Result { + validate_metadata_value(openshell_core::sandbox_env::IMAGE_ID, image_id)?; + let (user, group) = parse_oci_user(image_user)?; + + let (uid, primary_gid, presentation_user) = if let Some(uid) = parse_numeric_id(user, "UID")? { + let primary_gid = if group.is_none() { + let entries = read_passwd(passwd_path)?; + Some(unique_passwd_by_uid(&entries, uid)?.ok_or_else(|| { + miette::miette!( + "OCI USER '{image_user}' uses numeric UID {uid} without an explicit group, but /etc/passwd has no matching entry" + ) + })?.gid) + } else { + None + }; + (uid, primary_gid, uid.to_string()) + } else { + if user == "root" { + return Err(miette::miette!("OCI USER must not select root")); + } + let entries = read_passwd(passwd_path)?; + let entry = unique_passwd_by_name(&entries, user)?.ok_or_else(|| { + miette::miette!("OCI USER name '{user}' was not found in /etc/passwd") + })?; + (entry.uid, Some(entry.gid), user.to_string()) + }; + + let gid = match group { + Some(group) => { + if let Some(gid) = parse_numeric_id(group, "GID")? { + gid + } else { + if group == "root" { + return Err(miette::miette!("OCI USER group must not select root")); + } + let entries = read_group(group_path)?; + unique_group_by_name(&entries, group)? + .ok_or_else(|| { + miette::miette!("OCI USER group '{group}' was not found in /etc/group") + })? + .gid + } + } + None => primary_gid.ok_or_else(|| { + miette::miette!("OCI USER '{image_user}' did not resolve a primary GID") + })?, + }; + + reject_root_identity(uid, gid, image_user)?; + Ok(ResolvedAgentIdentity { + uid, + gid, + presentation_user, + source: IdentitySource::Image, + image_id: Some(image_id.to_string()), + }) +} + +fn resolve_and_persist_at( + request: Option<&IdentityRequest>, + passwd_path: &Path, + group_path: &Path, + state_path: &Path, + require_root_owner: bool, + validate_mapping: impl Fn(&ResolvedAgentIdentity) -> Result<()>, +) -> Result> { + let Some(request) = request else { + return Ok(None); + }; + + if let Some(state) = load_state(state_path, require_root_owner)? { + validate_persisted(&state, request)?; + validate_mapping(&state.identity)?; + return Ok(Some(state.identity)); + } + + let (identity, image_user) = match request { + IdentityRequest::Image { + image_user, + image_id, + } => ( + resolve_image_identity_at(image_user, image_id, passwd_path, group_path)?, + Some(image_user.clone()), + ), + IdentityRequest::Fixed { uid, gid, image_id } => { + reject_root_identity(*uid, *gid, "fixed identity")?; + ( + ResolvedAgentIdentity { + uid: *uid, + gid: *gid, + presentation_user: uid.to_string(), + source: IdentitySource::Fixed, + image_id: Some(image_id.clone()), + }, + None, + ) + } + }; + let state = PersistedIdentity { + version: IDENTITY_STATE_VERSION, + identity: identity.clone(), + image_user, + }; + validate_mapping(&identity)?; + persist_state(state_path, &state, require_root_owner)?; + Ok(Some(identity)) +} + +fn validate_persisted(state: &PersistedIdentity, request: &IdentityRequest) -> Result<()> { + if state.version != IDENTITY_STATE_VERSION { + return Err(miette::miette!( + "unsupported agent identity state version {}; expected {}", + state.version, + IDENTITY_STATE_VERSION + )); + } + if state.identity.source != request.source() { + return Err(miette::miette!( + "persisted identity source '{}' does not match requested source '{}'", + state.identity.source, + request.source() + )); + } + reject_root_identity(state.identity.uid, state.identity.gid, "persisted identity")?; + + match request { + IdentityRequest::Image { + image_user, + image_id, + } => { + if state.identity.image_id.as_deref() != Some(image_id) + || state.image_user.as_deref() != Some(image_user) + { + return Err(miette::miette!( + "persisted image identity metadata does not match OPENSHELL_IMAGE_ID/OPENSHELL_IMAGE_USER" + )); + } + } + IdentityRequest::Fixed { uid, gid, image_id } => { + if state.identity.uid != *uid + || state.identity.gid != *gid + || state.identity.image_id.as_deref() != Some(image_id) + || state.image_user.is_some() + { + return Err(miette::miette!( + "persisted fixed identity does not match OPENSHELL_SANDBOX_UID/OPENSHELL_SANDBOX_GID/OPENSHELL_IMAGE_ID" + )); + } + } + } + Ok(()) +} + +fn required_env(name: &str) -> Result { + let value = std::env::var(name).map_err(|_| miette::miette!("{name} is required"))?; + if value.is_empty() { + return Err(miette::miette!("{name} must not be empty")); + } + Ok(value) +} + +fn validate_metadata_value(name: &str, value: &str) -> Result<()> { + if value.is_empty() + || value.len() > MAX_ACCOUNT_FIELD_SIZE + || value.trim() != value + || value.chars().any(char::is_control) + { + return Err(miette::miette!("{name} contains an invalid value")); + } + Ok(()) +} + +fn parse_fixed_id(name: &str, value: &str) -> Result { + if !openshell_policy::is_valid_sandbox_identity(value) || value == "sandbox" { + return Err(miette::miette!( + "{name} must be a numeric identity in range [{}, {}]", + openshell_policy::MIN_SANDBOX_UID, + openshell_policy::MAX_SANDBOX_UID + )); + } + value + .parse::() + .into_diagnostic() + .wrap_err_with(|| format!("invalid {name}")) +} + +fn parse_oci_user(value: &str) -> Result<(&str, Option<&str>)> { + if value.is_empty() { + return Err(miette::miette!( + "OCI USER is empty; declare a non-root user" + )); + } + if value.trim() != value || value.chars().any(char::is_whitespace) { + return Err(miette::miette!("OCI USER must not contain whitespace")); + } + let mut fields = value.split(':'); + let user = fields.next().unwrap_or_default(); + let group = fields.next(); + if user.is_empty() || group == Some("") || fields.next().is_some() { + return Err(miette::miette!( + "malformed OCI USER '{value}'; expected user or user:group" + )); + } + validate_identity_token(user, "user")?; + if let Some(group) = group { + validate_identity_token(group, "group")?; + } + Ok((user, group)) +} + +fn validate_identity_token(value: &str, kind: &str) -> Result<()> { + if value.len() > MAX_ACCOUNT_FIELD_SIZE + || value + .bytes() + .any(|byte| byte == b'\0' || byte == b'\n' || byte == b'\r') + { + return Err(miette::miette!("OCI USER {kind} field is invalid")); + } + Ok(()) +} + +fn parse_numeric_id(value: &str, kind: &str) -> Result> { + if !value.bytes().all(|byte| byte.is_ascii_digit()) { + return Ok(None); + } + let id = value + .parse::() + .into_diagnostic() + .wrap_err_with(|| format!("OCI USER {kind} is outside the supported range"))?; + if id == u32::MAX { + return Err(miette::miette!( + "OCI USER {kind} must not use the reserved value {}", + u32::MAX + )); + } + Ok(Some(id)) +} + +fn reject_root_identity(uid: u32, gid: u32, declaration: &str) -> Result<()> { + if uid == 0 || gid == 0 { + return Err(miette::miette!( + "{declaration} resolves to prohibited root identity {uid}:{gid}" + )); + } + Ok(()) +} + +/// One extent from Linux `/proc/*/{uid,gid}_map`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct IdMapRange { + pub inside_start: u64, + pub outside_start: u64, + pub length: u64, +} + +/// Parse a Linux user-namespace ID map with overflow checks. +pub fn parse_id_map(contents: &str, map_name: &str) -> Result> { + let mut ranges = Vec::new(); + for (index, line) in contents.lines().enumerate() { + if line.trim().is_empty() { + continue; + } + let fields: Vec<_> = line.split_ascii_whitespace().collect(); + if fields.len() != 3 { + return Err(miette::miette!( + "malformed {map_name} at line {}: expected three integer fields", + index + 1 + )); + } + let parse_field = |field: &str| -> Result { + field.parse::().into_diagnostic().wrap_err_with(|| { + format!( + "malformed {map_name} at line {}: invalid integer", + index + 1 + ) + }) + }; + let range = IdMapRange { + inside_start: parse_field(fields[0])?, + outside_start: parse_field(fields[1])?, + length: parse_field(fields[2])?, + }; + if range.length == 0 + || range.inside_start.checked_add(range.length).is_none() + || range.outside_start.checked_add(range.length).is_none() + { + return Err(miette::miette!( + "malformed {map_name} at line {}: zero-length or overflowing range", + index + 1 + )); + } + ranges.push(range); + } + if ranges.is_empty() { + return Err(miette::miette!("{map_name} contains no mappings")); + } + Ok(ranges) +} + +/// Verify that a namespace-visible ID is covered by a parsed map. +pub fn validate_id_is_mapped(id: u32, ranges: &[IdMapRange], map_name: &str) -> Result<()> { + let id = u64::from(id); + if ranges.iter().any(|range| { + range + .inside_start + .checked_add(range.length) + .is_some_and(|end| id >= range.inside_start && id < end) + }) { + return Ok(()); + } + Err(miette::miette!( + "identity ID {id} is not mapped by {map_name}" + )) +} + +/// Validate UID/GID mappings using alternate proc-map paths. +#[cfg(target_os = "linux")] +pub fn validate_user_namespace_mappings_at( + uid: u32, + gid: u32, + uid_map_path: &Path, + gid_map_path: &Path, +) -> Result<()> { + let uid_map = read_utf8_bounded_regular_file(uid_map_path, MAX_ID_MAP_FILE_SIZE)?; + let gid_map = read_utf8_bounded_regular_file(gid_map_path, MAX_ID_MAP_FILE_SIZE)?; + validate_id_is_mapped(uid, &parse_id_map(&uid_map, "uid_map")?, "uid_map")?; + validate_id_is_mapped(gid, &parse_id_map(&gid_map, "gid_map")?, "gid_map") +} + +/// Validate UID/GID mappings for the current Linux user namespace. +#[cfg(target_os = "linux")] +pub fn validate_user_namespace_mappings(uid: u32, gid: u32) -> Result<()> { + let proc_dir = PathBuf::from(format!("/proc/{}", std::process::id())); + validate_user_namespace_mappings_at( + uid, + gid, + &proc_dir.join("uid_map"), + &proc_dir.join("gid_map"), + ) +} + +/// Non-Linux platforms retain their existing identity behavior. +#[cfg(not(target_os = "linux"))] +pub fn validate_user_namespace_mappings(_uid: u32, _gid: u32) -> Result<()> { + Ok(()) +} + +fn read_passwd(path: &Path) -> Result> { + parse_account_lines(path, 7, |fields, line| { + let uid = parse_account_id(fields[2], "UID", path, line)?; + let gid = parse_account_id(fields[3], "GID", path, line)?; + Ok(PasswdEntry { + name: fields[0].to_string(), + uid, + gid, + }) + }) +} + +fn read_group(path: &Path) -> Result> { + parse_account_lines(path, 4, |fields, line| { + let gid = parse_account_id(fields[2], "GID", path, line)?; + Ok(GroupEntry { + name: fields[0].to_string(), + gid, + }) + }) +} + +fn parse_account_lines( + path: &Path, + expected_fields: usize, + mut parse: impl FnMut(&[&str], usize) -> Result, +) -> Result> { + let bytes = read_bounded_regular_file(path, MAX_ACCOUNT_FILE_SIZE)?; + let contents = std::str::from_utf8(&bytes) + .into_diagnostic() + .wrap_err_with(|| format!("account file {} is not valid UTF-8", path.display()))?; + let mut entries = Vec::new(); + for (index, raw_line) in contents.split('\n').enumerate() { + let line_number = index + 1; + let line = raw_line.strip_suffix('\r').unwrap_or(raw_line); + if line.is_empty() || line.starts_with('#') { + continue; + } + if line.len() > MAX_ACCOUNT_LINE_SIZE { + return Err(miette::miette!( + "account file {} line {line_number} exceeds {} bytes", + path.display(), + MAX_ACCOUNT_LINE_SIZE + )); + } + let fields: Vec<&str> = line.split(':').collect(); + if fields.len() != expected_fields + || fields[0].is_empty() + || fields + .iter() + .any(|field| field.len() > MAX_ACCOUNT_FIELD_SIZE) + { + return Err(miette::miette!( + "malformed account entry in {} at line {line_number}", + path.display() + )); + } + entries.push(parse(&fields, line_number)?); + } + Ok(entries) +} + +fn parse_account_id(value: &str, kind: &str, path: &Path, line: usize) -> Result { + if value.is_empty() || !value.bytes().all(|byte| byte.is_ascii_digit()) { + return Err(miette::miette!( + "malformed {kind} in {} at line {line}", + path.display() + )); + } + let value = value + .parse::() + .into_diagnostic() + .wrap_err_with(|| format!("malformed {kind} in {} at line {line}", path.display()))?; + if value == u32::MAX { + return Err(miette::miette!( + "reserved {kind} value in {} at line {line}", + path.display() + )); + } + Ok(value) +} + +fn unique_passwd_by_name<'a>( + entries: &'a [PasswdEntry], + name: &str, +) -> Result> { + let matches: Vec<_> = entries.iter().filter(|entry| entry.name == name).collect(); + let Some(entry) = unique_match(matches, "passwd name", name)? else { + return Ok(None); + }; + reject_duplicate_passwd_uid(entries, entry.uid)?; + Ok(Some(entry)) +} + +fn unique_passwd_by_uid(entries: &[PasswdEntry], uid: u32) -> Result> { + let matches: Vec<_> = entries.iter().filter(|entry| entry.uid == uid).collect(); + unique_match(matches, "passwd UID", &uid.to_string()) +} + +fn reject_duplicate_passwd_uid(entries: &[PasswdEntry], uid: u32) -> Result<()> { + if entries.iter().filter(|entry| entry.uid == uid).count() > 1 { + return Err(miette::miette!("duplicate passwd UID {uid}")); + } + Ok(()) +} + +fn unique_group_by_name<'a>( + entries: &'a [GroupEntry], + name: &str, +) -> Result> { + let matches: Vec<_> = entries.iter().filter(|entry| entry.name == name).collect(); + let Some(entry) = unique_match(matches, "group name", name)? else { + return Ok(None); + }; + if entries + .iter() + .filter(|candidate| candidate.gid == entry.gid) + .count() + > 1 + { + return Err(miette::miette!("duplicate group GID {}", entry.gid)); + } + Ok(Some(entry)) +} + +fn unique_match<'a, T>(matches: Vec<&'a T>, kind: &str, value: &str) -> Result> { + match matches.as_slice() { + [] => Ok(None), + [entry] => Ok(Some(*entry)), + _ => Err(miette::miette!("duplicate {kind} '{value}'")), + } +} + +fn read_bounded_regular_file(path: &Path, max_size: u64) -> Result> { + let file = open_regular_file_no_follow(path)?; + let metadata = file.metadata().into_diagnostic()?; + if metadata.len() > max_size { + return Err(miette::miette!( + "{} exceeds the {} byte size limit", + path.display(), + max_size + )); + } + let mut bytes = Vec::with_capacity(usize::try_from(metadata.len()).unwrap_or(0)); + file.take(max_size + 1) + .read_to_end(&mut bytes) + .into_diagnostic()?; + if u64::try_from(bytes.len()).unwrap_or(u64::MAX) > max_size { + return Err(miette::miette!( + "{} exceeds the {} byte size limit", + path.display(), + max_size + )); + } + Ok(bytes) +} + +#[cfg(target_os = "linux")] +fn read_utf8_bounded_regular_file(path: &Path, max_size: u64) -> Result { + String::from_utf8(read_bounded_regular_file(path, max_size)?) + .into_diagnostic() + .wrap_err_with(|| format!("{} is not valid UTF-8", path.display())) +} + +fn load_state(path: &Path, require_root_owner: bool) -> Result> { + let Some((parent, name)) = open_parent_directory(path, false, Mode::empty())? else { + return Ok(None); + }; + let state = load_state_from_parent(&parent, &name, path, require_root_owner)?; + if state.is_some() { + validate_state_directory(&parent, path, require_root_owner)?; + } + Ok(state) +} + +fn load_state_from_parent( + parent: &OwnedFd, + name: &OsStr, + display_path: &Path, + require_root_owner: bool, +) -> Result> { + let fd = match rustix::fs::openat( + parent, + name, + OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::NONBLOCK, + Mode::empty(), + ) { + Ok(fd) => fd, + Err(rustix::io::Errno::NOENT) => return Ok(None), + Err(error) => { + return Err(miette::miette!( + "failed to open identity state {} without following symlinks: {error}", + display_path.display() + )); + } + }; + let file = File::from(fd); + validate_state_metadata( + display_path, + &file.metadata().into_diagnostic()?, + require_root_owner, + )?; + let mut bytes = Vec::new(); + file.take(MAX_STATE_FILE_SIZE + 1) + .read_to_end(&mut bytes) + .into_diagnostic()?; + if u64::try_from(bytes.len()).unwrap_or(u64::MAX) > MAX_STATE_FILE_SIZE { + return Err(miette::miette!("agent identity state file is oversized")); + } + let state = decode_state(&bytes)?; + Ok(Some(state)) +} + +fn persist_state(path: &Path, state: &PersistedIdentity, require_root_owner: bool) -> Result<()> { + let (parent, name) = open_parent_directory(path, true, Mode::from_raw_mode(0o700))? + .ok_or_else(|| miette::miette!("identity state path parent is unavailable"))?; + prepare_state_directory(&parent, path, require_root_owner)?; + let bytes = encode_state(state)?; + if u64::try_from(bytes.len()).unwrap_or(u64::MAX) > MAX_STATE_FILE_SIZE { + return Err(miette::miette!("agent identity state exceeds size limit")); + } + let temp_name = OsString::from(format!(".agent-identity-{}.tmp", uuid::Uuid::new_v4())); + let temp_fd = rustix::fs::openat( + &parent, + &temp_name, + OFlags::WRONLY | OFlags::CREATE | OFlags::EXCL | OFlags::CLOEXEC | OFlags::NOFOLLOW, + Mode::from_raw_mode(0o600), + ) + .map_err(|error| { + miette::miette!("failed to create exclusive identity state temp file: {error}") + })?; + let mut file = File::from(temp_fd); + let write_result = (|| -> Result<()> { + file.write_all(&bytes).into_diagnostic()?; + file.write_all(b"\n").into_diagnostic()?; + rustix::fs::fsync(&file).into_diagnostic()?; + validate_state_metadata( + path, + &file.metadata().into_diagnostic()?, + require_root_owner, + ) + })(); + drop(file); + if let Err(error) = write_result { + let _ = rustix::fs::unlinkat(&parent, &temp_name, AtFlags::empty()); + return Err(error); + } + + match publish_state_no_replace(&parent, &temp_name, &name) { + Ok(()) => { + rustix::fs::fsync(&parent).into_diagnostic()?; + Ok(()) + } + Err(rustix::io::Errno::EXIST) => { + let _ = rustix::fs::unlinkat(&parent, &temp_name, AtFlags::empty()); + let existing = load_state_from_parent(&parent, &name, path, require_root_owner)? + .ok_or_else(|| miette::miette!("identity state disappeared during publication"))?; + if existing != *state { + return Err(miette::miette!( + "concurrent identity state does not match resolved identity" + )); + } + rustix::fs::fsync(&parent).into_diagnostic()?; + Ok(()) + } + Err(error) => { + let _ = rustix::fs::unlinkat(&parent, &temp_name, AtFlags::empty()); + Err(miette::miette!( + "failed to publish identity state atomically: {error}" + )) + } + } +} + +#[cfg(any(target_os = "linux", target_vendor = "apple"))] +fn publish_state_no_replace( + parent: &OwnedFd, + temp_name: &OsStr, + state_name: &OsStr, +) -> rustix::io::Result<()> { + rustix::fs::renameat_with( + parent, + temp_name, + parent, + state_name, + RenameFlags::NOREPLACE, + ) +} + +#[cfg(not(any(target_os = "linux", target_vendor = "apple")))] +fn publish_state_no_replace( + parent: &OwnedFd, + temp_name: &OsStr, + state_name: &OsStr, +) -> rustix::io::Result<()> { + rustix::fs::linkat(parent, temp_name, parent, state_name, AtFlags::empty())?; + rustix::fs::unlinkat(parent, temp_name, AtFlags::empty()) +} + +fn encode_state(state: &PersistedIdentity) -> Result> { + serde_json::to_vec(&serde_json::json!({ + "version": state.version, + "identity": state.identity, + "image_user": state.image_user, + })) + .into_diagnostic() +} + +fn decode_state(bytes: &[u8]) -> Result { + let mut value: serde_json::Value = serde_json::from_slice(bytes) + .into_diagnostic() + .wrap_err("failed to parse persisted agent identity")?; + let object = value + .as_object_mut() + .ok_or_else(|| miette::miette!("persisted agent identity must be a JSON object"))?; + let version = object + .remove("version") + .and_then(|value| value.as_u64()) + .and_then(|value| u32::try_from(value).ok()) + .ok_or_else(|| miette::miette!("persisted agent identity has an invalid version"))?; + let identity = serde_json::from_value( + object + .remove("identity") + .ok_or_else(|| miette::miette!("persisted agent identity is missing identity"))?, + ) + .into_diagnostic() + .wrap_err("persisted agent identity has an invalid identity record")?; + let image_user = match object.remove("image_user") { + None | Some(serde_json::Value::Null) => None, + Some(serde_json::Value::String(value)) => Some(value), + Some(_) => { + return Err(miette::miette!( + "persisted agent identity has invalid image_user metadata" + )); + } + }; + Ok(PersistedIdentity { + version, + identity, + image_user, + }) +} + +fn prepare_state_directory( + directory: &OwnedFd, + state_path: &Path, + require_root_owner: bool, +) -> Result<()> { + let stat = rustix::fs::fstat(directory).into_diagnostic()?; + if require_root_owner && stat.st_uid != 0 { + return Err(miette::miette!( + "identity state directory {} must be root-owned", + state_path.parent().unwrap_or(state_path).display() + )); + } + rustix::fs::fchmod(directory, Mode::from_raw_mode(0o700)).into_diagnostic()?; + Ok(()) +} + +fn validate_state_directory( + directory: &OwnedFd, + state_path: &Path, + require_root_owner: bool, +) -> Result<()> { + let stat = rustix::fs::fstat(directory).into_diagnostic()?; + if require_root_owner && stat.st_uid != 0 { + return Err(miette::miette!( + "identity state directory {} must be root-owned", + state_path.parent().unwrap_or(state_path).display() + )); + } + if stat.st_mode & 0o777 != 0o700 { + return Err(miette::miette!( + "identity state directory {} must have mode 0700", + state_path.parent().unwrap_or(state_path).display() + )); + } + Ok(()) +} + +fn validate_state_metadata( + path: &Path, + metadata: &std::fs::Metadata, + require_root_owner: bool, +) -> Result<()> { + if !metadata.file_type().is_file() { + return Err(miette::miette!( + "identity state {} must be a regular file", + path.display() + )); + } + if metadata.mode() & 0o777 != 0o600 { + return Err(miette::miette!( + "identity state {} must have mode 0600", + path.display() + )); + } + if require_root_owner && metadata.uid() != 0 { + return Err(miette::miette!( + "identity state {} must be root-owned", + path.display() + )); + } + Ok(()) +} + +fn open_regular_file_no_follow(path: &Path) -> Result { + let (parent, name) = open_parent_directory(path, false, Mode::empty())? + .ok_or_else(|| miette::miette!("parent of {} does not exist", path.display()))?; + let fd = rustix::fs::openat( + &parent, + &name, + OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::NONBLOCK, + Mode::empty(), + ) + .map_err(|error| { + miette::miette!( + "failed to open {} without following symlinks: {error}", + path.display() + ) + })?; + let file = File::from(fd); + if !file.metadata().into_diagnostic()?.file_type().is_file() { + return Err(miette::miette!("{} must be a regular file", path.display())); + } + Ok(file) +} + +/// Prepare a read-write path and own a newly created final directory through +/// its already-verified descriptor. Existing paths retain their ownership. +pub(crate) fn prepare_read_write_path_owned( + path: &Path, + uid: Option, + gid: Option, +) -> Result { + prepare_read_write_path_owned_with(path, uid, gid, fchown_fd) +} + +fn prepare_read_write_path_owned_with( + path: &Path, + uid: Option, + gid: Option, + own: impl FnOnce(&OwnedFd, Option, Option) -> Result<()>, +) -> Result { + let (parent, name) = open_parent_directory(path, true, Mode::from_raw_mode(0o777))? + .ok_or_else(|| miette::miette!("directory parent is unavailable: {}", path.display()))?; + match rustix::fs::openat( + &parent, + &name, + OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::NONBLOCK, + Mode::empty(), + ) { + Ok(_) => Ok(false), + Err(rustix::io::Errno::NOENT) => { + let created = match rustix::fs::mkdirat(&parent, &name, Mode::from_raw_mode(0o777)) { + Ok(()) => true, + Err(rustix::io::Errno::EXIST) => false, + Err(error) => { + return Err(miette::miette!( + "failed to create directory {} safely: {error}", + path.display() + )); + } + }; + let directory = rustix::fs::openat( + &parent, + &name, + OFlags::RDONLY | OFlags::DIRECTORY | OFlags::CLOEXEC | OFlags::NOFOLLOW, + Mode::empty(), + ) + .map_err(|error| { + miette::miette!( + "failed to verify created directory {} without following symlinks: {error}", + path.display() + ) + })?; + if created { + own(&directory, uid, gid)?; + } + Ok(created) + } + Err(error) => Err(miette::miette!( + "refusing unsafe directory path {}: {error}", + path.display() + )), + } +} + +/// Own one existing directory through a descriptor-relative, no-follow open. +pub(crate) fn chown_directory_no_follow( + path: &Path, + uid: Option, + gid: Option, +) -> Result { + chown_directory_no_follow_with(path, uid, gid, fchown_fd) +} + +fn chown_directory_no_follow_with( + path: &Path, + uid: Option, + gid: Option, + own: impl FnOnce(&OwnedFd, Option, Option) -> Result<()>, +) -> Result { + let Some((parent, name)) = open_parent_directory(path, false, Mode::empty())? else { + return Ok(false); + }; + let directory = match rustix::fs::openat( + &parent, + &name, + OFlags::RDONLY | OFlags::DIRECTORY | OFlags::CLOEXEC | OFlags::NOFOLLOW, + Mode::empty(), + ) { + Ok(directory) => directory, + Err(rustix::io::Errno::NOENT) => return Ok(false), + Err(error) => { + return Err(miette::miette!( + "failed to open directory {} without following symlinks: {error}", + path.display() + )); + } + }; + own(&directory, uid, gid)?; + Ok(true) +} + +fn fchown_fd(fd: &OwnedFd, uid: Option, gid: Option) -> Result<()> { + rustix::fs::fchown( + fd, + uid.map(rustix::fs::Uid::from_raw), + gid.map(rustix::fs::Gid::from_raw), + ) + .into_diagnostic() +} + +fn open_parent_directory( + path: &Path, + create: bool, + create_mode: Mode, +) -> Result> { + let name = path + .file_name() + .ok_or_else(|| miette::miette!("path has no final component: {}", path.display()))? + .to_os_string(); + let parent = path.parent().unwrap_or_else(|| Path::new(".")); + Ok(open_directory_path(parent, create, create_mode)?.map(|directory| (directory, name))) +} + +fn open_directory_path(path: &Path, create: bool, create_mode: Mode) -> Result> { + let start = if path.is_absolute() { "/" } else { "." }; + let mut directory: OwnedFd = File::open(start).into_diagnostic()?.into(); + for component in path.components() { + let name = match component { + Component::RootDir | Component::CurDir => continue, + Component::Normal(name) => name, + Component::ParentDir | Component::Prefix(_) => { + return Err(miette::miette!( + "path contains an unsafe component: {}", + path.display() + )); + } + }; + let flags = OFlags::RDONLY | OFlags::DIRECTORY | OFlags::CLOEXEC | OFlags::NOFOLLOW; + directory = match rustix::fs::openat(&directory, name, flags, Mode::empty()) { + Ok(next) => next, + Err(rustix::io::Errno::NOENT) if !create => return Ok(None), + Err(rustix::io::Errno::NOENT) => { + match rustix::fs::mkdirat(&directory, name, create_mode) { + Ok(()) | Err(rustix::io::Errno::EXIST) => {} + Err(error) => { + return Err(miette::miette!( + "failed to create directory component '{}' in {}: {error}", + name.to_string_lossy(), + path.display() + )); + } + } + rustix::fs::openat(&directory, name, flags, Mode::empty()).map_err(|error| { + miette::miette!( + "failed to verify directory component '{}' in {} without following symlinks: {error}", + name.to_string_lossy(), + path.display() + ) + })? + } + Err(error) => { + return Err(miette::miette!( + "refusing path {} with symlink or non-directory ancestor '{}': {error}", + path.display(), + name.to_string_lossy() + )); + } + }; + } + Ok(Some(directory)) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::os::unix::fs::PermissionsExt; + use std::os::unix::fs::symlink; + + fn account_files(passwd: &str, group: &str) -> (tempfile::TempDir, PathBuf, PathBuf) { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let passwd_path = root.join("passwd"); + let group_path = root.join("group"); + std::fs::write(&passwd_path, passwd).unwrap(); + std::fs::write(&group_path, group).unwrap(); + (dir, passwd_path, group_path) + } + + #[test] + fn resolves_all_oci_user_forms() { + let (_dir, passwd, group) = + account_files("app:x:1234:1235::/home/app:/bin/sh\n", "staff:x:2345:\n"); + let cases = [ + ("app", 1234, 1235, "app"), + ("app:staff", 1234, 2345, "app"), + ("app:2345", 1234, 2345, "app"), + ("1234:staff", 1234, 2345, "1234"), + ("1234", 1234, 1235, "1234"), + ("1234:2345", 1234, 2345, "1234"), + ]; + for (declaration, uid, gid, presentation) in cases { + let identity = + resolve_image_identity_at(declaration, "sha256:test", &passwd, &group).unwrap(); + assert_eq!((identity.uid, identity.gid), (uid, gid), "{declaration}"); + assert_eq!(identity.presentation_user, presentation, "{declaration}"); + } + } + + #[test] + fn numeric_pair_does_not_require_account_files() { + let missing = Path::new("/definitely/missing/account-file"); + let identity = + resolve_image_identity_at("1234:2345", "sha256:test", missing, missing).unwrap(); + assert_eq!((identity.uid, identity.gid), (1234, 2345)); + } + + #[test] + fn rejects_empty_malformed_root_and_accountless_uid() { + let (_dir, passwd, group) = account_files( + "root:x:0:0::/root:/bin/sh\nrootish:x:0:1000::/:/bin/sh\n", + "root:x:0:\n", + ); + for declaration in [ + "", + ":", + "app:", + "app:staff:extra", + " app", + "root", + "rootish", + "1234", + ] { + assert!( + resolve_image_identity_at(declaration, "sha256:test", &passwd, &group).is_err(), + "{declaration:?} should fail" + ); + } + assert!(resolve_image_identity_at("1234:0", "sha256:test", &passwd, &group).is_err()); + } + + #[test] + fn rejects_malformed_and_duplicate_matching_account_entries() { + let (_dir, passwd, group) = account_files( + "app:x:1234:1235::/home/app:/bin/sh\nmalformed\n", + "staff:x:2345:\n", + ); + assert!(resolve_image_identity_at("app", "sha256:test", &passwd, &group).is_err()); + + std::fs::write( + &passwd, + "app:x:1234:1235::/home/app:/bin/sh\nother:x:1234:1235::/home/other:/bin/sh\n", + ) + .unwrap(); + assert!(resolve_image_identity_at("app", "sha256:test", &passwd, &group).is_err()); + + std::fs::write(&passwd, "app:x:1234:1235::/home/app:/bin/sh\n").unwrap(); + std::fs::write(&group, "staff:x:2345:\nother:x:2345:\n").unwrap(); + assert!(resolve_image_identity_at("app:staff", "sha256:test", &passwd, &group).is_err()); + } + + #[test] + fn rejects_symlink_non_regular_and_oversized_account_files() { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let real = root.join("real-passwd"); + let link = root.join("passwd-link"); + std::fs::write(&real, "app:x:1234:1235::/:/bin/sh\n").unwrap(); + symlink(&real, &link).unwrap(); + assert!(resolve_image_identity_at("app", "sha256:test", &link, &real).is_err()); + assert!(resolve_image_identity_at("app", "sha256:test", &root, &real).is_err()); + + let oversized = root.join("oversized"); + std::fs::write( + &oversized, + vec![b'x'; usize::try_from(MAX_ACCOUNT_FILE_SIZE + 1).unwrap()], + ) + .unwrap(); + assert!(resolve_image_identity_at("app", "sha256:test", &oversized, &real).is_err()); + + let long_line = root.join("long-line"); + let mut contents = "app:x:1234:1235::/home/app:/bin/sh".to_string(); + contents.push_str(&"x".repeat(MAX_ACCOUNT_LINE_SIZE)); + std::fs::write(&long_line, contents).unwrap(); + assert!(resolve_image_identity_at("app", "sha256:test", &long_line, &real).is_err()); + } + + #[test] + fn rejects_account_file_with_symlink_ancestor() { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let real_dir = root.join("real-etc"); + let linked_dir = root.join("linked-etc"); + std::fs::create_dir(&real_dir).unwrap(); + std::fs::write( + real_dir.join("passwd"), + "app:x:1234:1235::/home/app:/bin/sh\n", + ) + .unwrap(); + symlink(&real_dir, &linked_dir).unwrap(); + + assert!( + resolve_image_identity_at( + "app:1235", + "sha256:test", + &linked_dir.join("passwd"), + &real_dir.join("group"), + ) + .is_err() + ); + } + + #[test] + fn image_resolution_does_not_modify_account_files() { + let passwd_contents = "app:x:1234:1235::/home/app:/bin/sh\n"; + let group_contents = "staff:x:2345:\n"; + let (_dir, passwd, group) = account_files(passwd_contents, group_contents); + + resolve_image_identity_at("app:staff", "sha256:test", &passwd, &group).unwrap(); + + assert_eq!(std::fs::read_to_string(passwd).unwrap(), passwd_contents); + assert_eq!(std::fs::read_to_string(group).unwrap(), group_contents); + } + + #[test] + fn persisted_identity_is_reused_and_metadata_must_match() { + let (dir, passwd, group) = + account_files("app:x:1234:1235::/home/app:/bin/sh\n", "staff:x:2345:\n"); + let state = dir + .path() + .canonicalize() + .unwrap() + .join("state/agent-identity.json"); + let request = IdentityRequest::Image { + image_user: "app".into(), + image_id: "sha256:first".into(), + }; + let identity = + resolve_and_persist_at(Some(&request), &passwd, &group, &state, false, |_| Ok(())) + .unwrap() + .unwrap(); + assert_eq!((identity.uid, identity.gid), (1234, 1235)); + assert_eq!(std::fs::metadata(&state).unwrap().mode() & 0o777, 0o600); + + std::fs::write(&passwd, "app:x:9999:9999::/home/app:/bin/sh\n").unwrap(); + let restarted = + resolve_and_persist_at(Some(&request), &passwd, &group, &state, false, |_| Ok(())) + .unwrap() + .unwrap(); + assert_eq!(restarted, identity); + + let changed = IdentityRequest::Image { + image_user: "app".into(), + image_id: "sha256:changed".into(), + }; + assert!( + resolve_and_persist_at(Some(&changed), &passwd, &group, &state, false, |_| Ok(()),) + .is_err() + ); + } + + #[test] + fn missing_identity_source_preserves_legacy_behavior_without_state_access() { + let inaccessible = Path::new("/definitely/missing/legacy-state"); + assert_eq!( + resolve_and_persist_at( + None, + inaccessible, + inaccessible, + inaccessible, + true, + |_| Ok(()), + ) + .unwrap(), + None + ); + } + + #[test] + fn fixed_identity_is_persisted_and_checked_on_restart() { + let dir = tempfile::tempdir().unwrap(); + let state = dir + .path() + .canonicalize() + .unwrap() + .join("state/agent-identity.json"); + let request = IdentityRequest::Fixed { + uid: 4321, + gid: 5432, + image_id: "sha256:fixed".into(), + }; + let identity = resolve_and_persist_at( + Some(&request), + Path::new("/missing-passwd"), + Path::new("/missing-group"), + &state, + false, + |_| Ok(()), + ) + .unwrap() + .unwrap(); + assert_eq!(identity.source, IdentitySource::Fixed); + assert_eq!((identity.uid, identity.gid), (4321, 5432)); + assert_eq!(identity.image_id.as_deref(), Some("sha256:fixed")); + + let changed = IdentityRequest::Fixed { + uid: 4321, + gid: 9999, + image_id: "sha256:fixed".into(), + }; + assert!( + resolve_and_persist_at( + Some(&changed), + Path::new("/missing-passwd"), + Path::new("/missing-group"), + &state, + false, + |_| Ok(()), + ) + .is_err() + ); + + let changed_image = IdentityRequest::Fixed { + uid: 4321, + gid: 5432, + image_id: "sha256:changed".into(), + }; + assert!( + resolve_and_persist_at( + Some(&changed_image), + Path::new("/missing-passwd"), + Path::new("/missing-group"), + &state, + false, + |_| Ok(()), + ) + .is_err() + ); + } + + #[test] + fn id_map_parser_handles_full_map_gaps_and_overflow() { + let full = parse_id_map("0 0 4294967295\n", "uid_map").unwrap(); + validate_id_is_mapped(0, &full, "uid_map").unwrap(); + validate_id_is_mapped(u32::MAX - 1, &full, "uid_map").unwrap(); + let error = validate_id_is_mapped(u32::MAX, &full, "uid_map").unwrap_err(); + assert!(error.to_string().contains("4294967295")); + assert!(error.to_string().contains("uid_map")); + + let segmented = parse_id_map("0 100000 1000\n2000 200000 1000\n", "gid_map").unwrap(); + validate_id_is_mapped(2500, &segmented, "gid_map").unwrap(); + let error = validate_id_is_mapped(1500, &segmented, "gid_map").unwrap_err(); + assert!(error.to_string().contains("1500")); + assert!(error.to_string().contains("gid_map")); + + assert!(parse_id_map("18446744073709551615 0 2\n", "uid_map").is_err()); + assert!(parse_id_map("0 0 0\n", "uid_map").is_err()); + assert!(parse_id_map("0 0\n", "uid_map").is_err()); + } + + #[cfg(target_os = "linux")] + #[test] + fn user_namespace_mapping_validation_accepts_alternate_paths() { + let dir = tempfile::tempdir().unwrap(); + let uid_map = dir.path().join("uid_map"); + let gid_map = dir.path().join("gid_map"); + std::fs::write(&uid_map, "0 100000 65536\n").unwrap(); + std::fs::write(&gid_map, "0 200000 65536\n").unwrap(); + + validate_user_namespace_mappings_at(1234, 2345, &uid_map, &gid_map).unwrap(); + let error = + validate_user_namespace_mappings_at(70_000, 2345, &uid_map, &gid_map).unwrap_err(); + assert!(error.to_string().contains("70000")); + assert!(error.to_string().contains("uid_map")); + } + + #[test] + fn mapping_failure_prevents_state_persistence() { + let dir = tempfile::tempdir().unwrap(); + let state = dir + .path() + .canonicalize() + .unwrap() + .join("state/agent-identity.json"); + let request = IdentityRequest::Fixed { + uid: 4321, + gid: 5432, + image_id: "sha256:fixed".into(), + }; + let error = resolve_and_persist_at( + Some(&request), + Path::new("/missing-passwd"), + Path::new("/missing-group"), + &state, + false, + |_| Err(miette::miette!("identity ID 4321 is not mapped by uid_map")), + ) + .unwrap_err(); + assert!(error.to_string().contains("uid_map")); + assert!(!state.exists()); + } + + #[test] + fn state_publication_is_no_replace_and_ignores_partial_temp() { + let dir = tempfile::tempdir().unwrap(); + let state_path = dir + .path() + .canonicalize() + .unwrap() + .join("state/agent-identity.json"); + let state = PersistedIdentity { + version: IDENTITY_STATE_VERSION, + identity: ResolvedAgentIdentity { + uid: 4321, + gid: 5432, + presentation_user: "app".into(), + source: IdentitySource::Image, + image_id: Some("sha256:image".into()), + }, + image_user: Some("app".into()), + }; + std::fs::create_dir_all(state_path.parent().unwrap()).unwrap(); + std::fs::write( + state_path + .parent() + .unwrap() + .join(".agent-identity-partial.tmp"), + b"partial", + ) + .unwrap(); + + persist_state(&state_path, &state, false).unwrap(); + persist_state(&state_path, &state, false).unwrap(); + assert_eq!(load_state(&state_path, false).unwrap(), Some(state.clone())); + + let mut conflicting = state; + conflicting.identity.uid = 9999; + assert!(persist_state(&state_path, &conflicting, false).is_err()); + assert_eq!( + load_state(&state_path, false) + .unwrap() + .unwrap() + .identity + .uid, + 4321 + ); + } + + #[test] + fn state_path_rejects_symlink_ancestor() { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let real = root.join("real"); + let linked = root.join("linked"); + std::fs::create_dir(&real).unwrap(); + symlink(&real, &linked).unwrap(); + let state_path = linked.join("nested/agent-identity.json"); + let state = PersistedIdentity { + version: IDENTITY_STATE_VERSION, + identity: ResolvedAgentIdentity { + uid: 4321, + gid: 5432, + presentation_user: "4321".into(), + source: IdentitySource::Fixed, + image_id: Some("sha256:fixed".into()), + }, + image_user: None, + }; + + assert!(persist_state(&state_path, &state, false).is_err()); + assert!(!real.join("nested/agent-identity.json").exists()); + } + + #[test] + fn persisted_state_rejects_insecure_parent_mode() { + let dir = tempfile::tempdir().unwrap(); + let state_path = dir + .path() + .canonicalize() + .unwrap() + .join("state/agent-identity.json"); + let state = PersistedIdentity { + version: IDENTITY_STATE_VERSION, + identity: ResolvedAgentIdentity { + uid: 4321, + gid: 5432, + presentation_user: "4321".into(), + source: IdentitySource::Fixed, + image_id: Some("sha256:fixed".into()), + }, + image_user: None, + }; + persist_state(&state_path, &state, false).unwrap(); + std::fs::set_permissions( + state_path.parent().unwrap(), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + + assert!(load_state(&state_path, false).is_err()); + } + + #[test] + fn new_read_write_path_ownership_uses_open_descriptor_after_substitution() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let requested = root.join("requested"); + let moved = root.join("opened-directory"); + let substitution_target = root.join("substitution-target"); + std::fs::create_dir(&substitution_target).unwrap(); + let hook_called = std::cell::Cell::new(false); + + let created = prepare_read_write_path_owned_with( + &requested, + Some(1234), + Some(2345), + |fd, uid, gid| { + let opened_inode = rustix::fs::fstat(fd).into_diagnostic()?.st_ino; + std::fs::rename(&requested, &moved).into_diagnostic()?; + symlink(&substitution_target, &requested).into_diagnostic()?; + + assert_eq!(opened_inode, std::fs::metadata(&moved).unwrap().ino()); + assert_ne!(opened_inode, std::fs::metadata(&requested).unwrap().ino()); + assert_eq!(uid, Some(1234)); + assert_eq!(gid, Some(2345)); + hook_called.set(true); + Ok(()) + }, + ) + .unwrap(); + + assert!(created); + assert!(hook_called.get()); + assert!( + std::fs::symlink_metadata(requested) + .unwrap() + .file_type() + .is_symlink() + ); + } + + #[test] + fn existing_read_write_path_never_invokes_ownership_hook() { + let dir = tempfile::tempdir().unwrap(); + let existing = dir.path().canonicalize().unwrap().join("existing"); + std::fs::create_dir(&existing).unwrap(); + + let created = prepare_read_write_path_owned_with( + &existing, + Some(1234), + Some(2345), + |_fd, _uid, _gid| panic!("existing path ownership must be preserved"), + ) + .unwrap(); + + assert!(!created); + } + + #[test] + fn sandbox_root_ownership_uses_descriptor_and_does_not_descend() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let temp_root = dir.path().canonicalize().unwrap(); + let sandbox = temp_root.join("sandbox"); + let moved = temp_root.join("opened-sandbox"); + let substitution_target = temp_root.join("substitution-target"); + std::fs::create_dir(&sandbox).unwrap(); + std::fs::write(sandbox.join("child"), b"unchanged").unwrap(); + std::fs::create_dir(&substitution_target).unwrap(); + let hook_count = std::cell::Cell::new(0); + + let opened = + chown_directory_no_follow_with(&sandbox, Some(1234), Some(2345), |fd, uid, gid| { + let opened_inode = rustix::fs::fstat(fd).into_diagnostic()?.st_ino; + std::fs::rename(&sandbox, &moved).into_diagnostic()?; + symlink(&substitution_target, &sandbox).into_diagnostic()?; + + assert_eq!(opened_inode, std::fs::metadata(&moved).unwrap().ino()); + assert_ne!(opened_inode, std::fs::metadata(&sandbox).unwrap().ino()); + assert_eq!(uid, Some(1234)); + assert_eq!(gid, Some(2345)); + hook_count.set(hook_count.get() + 1); + Ok(()) + }) + .unwrap(); + + assert!(opened); + assert_eq!(hook_count.get(), 1); + assert_eq!(std::fs::read(moved.join("child")).unwrap(), b"unchanged"); + } + + #[test] + fn sandbox_root_ownership_rejects_symlink_ancestor() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let temp_root = dir.path().canonicalize().unwrap(); + let real = temp_root.join("real"); + let linked = temp_root.join("linked"); + std::fs::create_dir(&real).unwrap(); + std::fs::create_dir(real.join("sandbox")).unwrap(); + symlink(&real, &linked).unwrap(); + + let error = chown_directory_no_follow_with( + &linked.join("sandbox"), + Some(1234), + Some(2345), + |_fd, _uid, _gid| panic!("symlink ancestor must be rejected before ownership"), + ) + .unwrap_err(); + assert!( + error + .to_string() + .contains("symlink or non-directory ancestor") + ); + } +} diff --git a/crates/openshell-supervisor-process/src/lib.rs b/crates/openshell-supervisor-process/src/lib.rs index 842b62f9df..ca93230929 100644 --- a/crates/openshell-supervisor-process/src/lib.rs +++ b/crates/openshell-supervisor-process/src/lib.rs @@ -10,6 +10,8 @@ pub mod child_env; pub mod debug_rpc; +#[cfg(unix)] +pub mod identity; pub mod log_push; pub mod managed_children; pub mod process; diff --git a/crates/openshell-supervisor-process/src/process.rs b/crates/openshell-supervisor-process/src/process.rs index 322dcadd86..dbaba7678f 100644 --- a/crates/openshell-supervisor-process/src/process.rs +++ b/crates/openshell-supervisor-process/src/process.rs @@ -78,6 +78,11 @@ const SUPERVISOR_ONLY_ENV_VARS: &[&str] = &[ openshell_core::sandbox_env::TLS_CERT, openshell_core::sandbox_env::TLS_KEY, openshell_core::sandbox_env::PROVIDER_SPIFFE_WORKLOAD_API_SOCKET, + openshell_core::sandbox_env::IDENTITY_SOURCE, + openshell_core::sandbox_env::IMAGE_USER, + openshell_core::sandbox_env::IMAGE_ID, + openshell_core::sandbox_env::SANDBOX_UID, + openshell_core::sandbox_env::SANDBOX_GID, ]; pub fn is_supervisor_only_env_var(key: &str) -> bool { @@ -599,6 +604,12 @@ impl ProcessHandle { } } + // Apply identity last so image, request, and provider environment + // cannot replace the supervisor-selected child identity presentation. + for (key, value) in child_env::identity_env_vars(policy) { + cmd.env(key, value); + } + // Probe Landlock availability and emit OCSF logs from the parent // process where the tracing subscriber is functional. The child's // pre_exec context cannot reliably emit structured logs. @@ -740,6 +751,10 @@ impl ProcessHandle { } } + for (key, value) in child_env::identity_env_vars(policy) { + cmd.env(key, value); + } + // Set up process group for signal handling (non-interactive mode only). // In interactive mode, we inherit the parent's process group to maintain // proper terminal control for shells and interactive programs. @@ -869,7 +884,17 @@ pub fn validate_sandbox_user(policy: &SandboxPolicy) -> Result<()> { let identity = policy.process.run_as_user.as_deref().unwrap_or("sandbox"); // Numeric UID — no passwd entry required; kernel resolves directly. - if openshell_policy::is_valid_sandbox_identity(identity) && identity.parse::().is_ok() { + if let Ok(uid) = identity.parse::() { + let driver_resolved = + std::env::var_os(openshell_core::sandbox_env::IDENTITY_SOURCE).is_some(); + if uid == 0 { + return Err(miette::miette!("sandbox UID must not be 0")); + } + if !driver_resolved && !openshell_policy::is_valid_sandbox_identity(identity) { + return Err(miette::miette!( + "numeric sandbox UID {uid} is outside the allowed policy range [{MIN_SANDBOX_UID}, {MAX_SANDBOX_UID}]" + )); + } openshell_ocsf::ocsf_emit!( openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) .severity(openshell_ocsf::SeverityId::Informational) @@ -943,7 +968,17 @@ pub fn validate_sandbox_user(policy: &SandboxPolicy) -> Result<()> { pub fn validate_sandbox_group(policy: &SandboxPolicy) -> Result<()> { let identity = policy.process.run_as_group.as_deref().unwrap_or("sandbox"); - if openshell_policy::is_valid_sandbox_identity(identity) && identity.parse::().is_ok() { + if let Ok(gid) = identity.parse::() { + let driver_resolved = + std::env::var_os(openshell_core::sandbox_env::IDENTITY_SOURCE).is_some(); + if gid == 0 { + return Err(miette::miette!("sandbox GID must not be 0")); + } + if !driver_resolved && !openshell_policy::is_valid_sandbox_identity(identity) { + return Err(miette::miette!( + "numeric sandbox GID {gid} is outside the allowed policy range [{MIN_SANDBOX_UID}, {MAX_SANDBOX_UID}]" + )); + } openshell_ocsf::ocsf_emit!( openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) .severity(openshell_ocsf::SeverityId::Informational) @@ -1009,241 +1044,27 @@ pub use openshell_policy::{MAX_SANDBOX_UID, MIN_SANDBOX_UID}; /// Prepare a `read_write` path for the sandboxed process. /// -/// Returns `true` when the path was created by the supervisor and therefore -/// still needs to be chowned to the sandbox user/group. Existing paths keep -/// their image-defined ownership. +/// Returns `true` when the path was created and descriptor-owned by the +/// supervisor. Existing paths keep their image-defined ownership. #[cfg(unix)] -fn prepare_read_write_path(path: &Path) -> Result { - // SECURITY: use symlink_metadata (lstat) to inspect each path *before* - // calling chown. chown follows symlinks, so a malicious container image - // could place a symlink (e.g. /sandbox -> /etc/shadow) to trick the - // root supervisor into transferring ownership of arbitrary files. - // The TOCTOU window between lstat and chown is not exploitable because - // no untrusted process is running yet (the child has not been forked). - if let Ok(meta) = std::fs::symlink_metadata(path) { - if meta.file_type().is_symlink() { - return Err(miette::miette!( - "read_write path '{}' is a symlink — refusing to chown (potential privilege escalation)", - path.display() - )); - } - - debug!( - path = %path.display(), - "Preserving ownership for existing read_write path" - ); - Ok(false) - } else { +fn prepare_read_write_path(path: &Path, uid: Option, gid: Option) -> Result { + let created = crate::identity::prepare_read_write_path_owned( + path, + uid.map(Uid::as_raw), + gid.map(Gid::as_raw), + )?; + if created { debug!(path = %path.display(), "Creating read_write directory"); - std::fs::create_dir_all(path).into_diagnostic()?; - Ok(true) - } -} - -/// Update `/etc/passwd` and `/etc/group` so the "sandbox" user/group entries -/// match the driver-injected UID/GID from environment variables. -/// -/// When `OPENSHELL_SANDBOX_UID` is set, the image-baked "sandbox" entry may -/// have a different UID. Updating the files ensures `whoami`, `id`, `ls -l`, -/// SSH sessions, and `initgroups` resolve the sandbox identity correctly. -/// If no "sandbox" entry exists, one is appended. -#[cfg(unix)] -pub fn update_sandbox_passwd_entries() -> Result<()> { - let uid_str = match std::env::var(openshell_core::sandbox_env::SANDBOX_UID) { - Ok(v) if !v.is_empty() => v, - _ => return Ok(()), - }; - let gid_str = match std::env::var(openshell_core::sandbox_env::SANDBOX_GID) { - Ok(v) if !v.is_empty() => v, - _ => uid_str.clone(), - }; - - let _: u32 = uid_str - .parse() - .map_err(|e| miette::miette!("invalid OPENSHELL_SANDBOX_UID '{uid_str}': {e}"))?; - let _: u32 = gid_str - .parse() - .map_err(|e| miette::miette!("invalid OPENSHELL_SANDBOX_GID '{gid_str}': {e}"))?; - - update_passwd_file(&uid_str, &gid_str)?; - update_group_file(&gid_str)?; - - info!( - uid = %uid_str, - gid = %gid_str, - "Updated /etc/passwd and /etc/group for sandbox identity" - ); - Ok(()) -} - -/// Rewrite the `sandbox` line in `/etc/passwd` with the given UID/GID, -/// or append a new entry if none exists. -#[cfg(unix)] -fn update_passwd_file(uid: &str, gid: &str) -> Result<()> { - rewrite_passwd_at(Path::new("/etc/passwd"), uid, gid) -} - -/// Rewrite the `sandbox` line in `/etc/group` with the given GID, -/// or append a new entry if none exists. -#[cfg(unix)] -fn update_group_file(gid: &str) -> Result<()> { - rewrite_group_at(Path::new("/etc/group"), gid) -} - -#[cfg(unix)] -fn rewrite_passwd_at(path: &Path, uid: &str, gid: &str) -> Result<()> { - let content = std::fs::read_to_string(path).into_diagnostic()?; - - let mut found = false; - let mut lines: Vec = content - .lines() - .map(|line| { - if line.starts_with("sandbox:") { - found = true; - let fields: Vec<&str> = line.split(':').collect(); - if fields.len() >= 7 { - format!( - "{}:{}:{}:{}:{}:{}:{}", - fields[0], fields[1], uid, gid, fields[4], fields[5], fields[6] - ) - } else { - line.to_string() - } - } else { - line.to_string() - } - }) - .collect(); - - if !found { - lines.push(format!("sandbox:x:{uid}:{gid}::/sandbox:/bin/sh")); - } - - let mut output = lines.join("\n"); - if content.ends_with('\n') || !found { - output.push('\n'); - } - - std::fs::write(path, output).into_diagnostic()?; - Ok(()) -} - -#[cfg(unix)] -fn rewrite_group_at(path: &Path, gid: &str) -> Result<()> { - let content = std::fs::read_to_string(path).into_diagnostic()?; - - let mut found = false; - let mut lines: Vec = content - .lines() - .map(|line| { - if line.starts_with("sandbox:") { - found = true; - let fields: Vec<&str> = line.split(':').collect(); - if fields.len() >= 4 { - format!("{}:{}:{}:{}", fields[0], fields[1], gid, fields[3]) - } else { - line.to_string() - } - } else { - line.to_string() - } - }) - .collect(); - - if !found { - lines.push(format!("sandbox:x:{gid}:")); - } - - let mut output = lines.join("\n"); - if content.ends_with('\n') || !found { - output.push('\n'); - } - - std::fs::write(path, output).into_diagnostic()?; - Ok(()) -} - -/// Recursively chown a directory tree to the given UID/GID. -/// -/// Symlinks are skipped (not followed) to prevent privilege escalation via -/// malicious container images. The TOCTOU window is not exploitable because -/// no untrusted process is running yet. -/// -/// The root path is chowned unconditionally — EROFS there is a hard error -/// (a read-only `/sandbox` is a misconfiguration). For children, `EROFS` -/// causes the walker to skip that path and its entire subtree — descending -/// into a read-only mount we do not control would be a TOCTOU risk -/// (CWE-367/CWE-59). Siblings of the read-only path are still visited. -#[cfg(unix)] -fn chown_sandbox_home(root: &Path, uid: Option, gid: Option) -> Result<()> { - let meta = std::fs::symlink_metadata(root).into_diagnostic()?; - if meta.file_type().is_symlink() { - return Err(miette::miette!( - "path '{}' is a symlink — refusing to chown (potential privilege escalation)", - root.display() - )); - } - - nix::unistd::chown(root, uid, gid).into_diagnostic()?; - - if meta.is_dir() { - chown_children(root, uid, gid, &nix::unistd::chown)?; - } - - Ok(()) -} - -/// Walk directory children and chown each entry, skipping symlinks and -/// EROFS subtrees. Called after the parent has already been chowned. -#[cfg(unix)] -fn chown_children( - dir: &Path, - uid: Option, - gid: Option, - do_chown: &impl Fn(&Path, Option, Option) -> nix::Result<()>, -) -> Result<()> { - match std::fs::read_dir(dir) { - Ok(entries) => { - for entry in entries { - let entry = entry.into_diagnostic()?; - let child = entry.path(); - chown_recursive(&child, uid, gid, do_chown)?; - } - } - Err(e) => { - debug!(path = %dir.display(), error = %e, "Cannot list directory during sandbox home chown"); - } + } else { + debug!(path = %path.display(), "Preserving ownership for existing read_write path"); } - Ok(()) + Ok(created) } +/// Chown only the workspace root, never image content or nested mounts. #[cfg(unix)] -fn chown_recursive( - path: &Path, - uid: Option, - gid: Option, - do_chown: &impl Fn(&Path, Option, Option) -> nix::Result<()>, -) -> Result<()> { - let meta = std::fs::symlink_metadata(path).into_diagnostic()?; - - if meta.file_type().is_symlink() { - debug!(path = %path.display(), "Skipping symlink during sandbox home chown"); - return Ok(()); - } - - if let Err(e) = do_chown(path, uid, gid) { - if e == nix::errno::Errno::EROFS { - debug!(path = %path.display(), "Skipping read-only path during sandbox home chown"); - return Ok(()); - } - return Err(e).into_diagnostic(); - } - - if meta.is_dir() { - chown_children(path, uid, gid, do_chown)?; - } - - Ok(()) +fn chown_sandbox_home(root: &Path, uid: Option, gid: Option) -> Result { + crate::identity::chown_directory_no_follow(root, uid.map(Uid::as_raw), gid.map(Gid::as_raw)) } /// Prepare filesystem for the sandboxed process. @@ -1256,7 +1077,6 @@ fn chown_recursive( /// UIDs/GIDs (passed directly to `chown` without a passwd lookup). #[cfg(unix)] pub fn prepare_filesystem(policy: &SandboxPolicy) -> Result<()> { - use nix::unistd::chown; use nix::unistd::{Gid, Uid}; let user_name = match policy.process.run_as_user.as_deref() { @@ -1291,29 +1111,36 @@ pub fn prepare_filesystem(policy: &SandboxPolicy) -> Result<()> { _ => None, }; + #[cfg(target_os = "linux")] + crate::identity::validate_user_namespace_mappings( + uid.unwrap_or_else(nix::unistd::geteuid).as_raw(), + gid.unwrap_or_else(nix::unistd::getegid).as_raw(), + )?; + // Create missing read_write paths and only chown the ones we created. for path in &policy.filesystem.read_write { - if prepare_read_write_path(path)? { + if prepare_read_write_path(path, uid, gid)? { debug!( path = %path.display(), ?uid, ?gid, "Setting ownership on newly created read_write path" ); - chown(path, uid, gid).into_diagnostic()?; } } - // When a driver injects a custom UID/GID via environment variables, the - // /sandbox home directory may already exist with image-default ownership - // (e.g. UID 1000) that differs from the driver-assigned identity. - // Recursively chown /sandbox so the sandbox process can use its home - // directory. - if std::env::var(openshell_core::sandbox_env::SANDBOX_UID).is_ok() { + // Driver-selected identities own the workspace mount point itself. Never + // traverse descendants because they may contain image data or nested mounts. + if std::env::var(openshell_core::sandbox_env::SANDBOX_UID).is_ok() + || std::env::var(openshell_core::sandbox_env::IDENTITY_SOURCE).is_ok() + { let sandbox_home = Path::new("/sandbox"); - if sandbox_home.exists() { - info!(?uid, ?gid, "Chowning /sandbox for driver-injected UID/GID"); - chown_sandbox_home(sandbox_home, uid, gid)?; + if chown_sandbox_home(sandbox_home, uid, gid)? { + info!( + ?uid, + ?gid, + "Chowning /sandbox root for driver-selected identity" + ); } } @@ -1389,10 +1216,14 @@ pub fn drop_privileges(policy: &SandboxPolicy) -> Result<()> { }, }; + #[cfg(target_os = "linux")] + crate::identity::validate_user_namespace_mappings(target_uid.as_raw(), target_gid.as_raw())?; + // Resolve the user record for initgroups only when identity is name-based. // Numeric UIDs may not have a /etc/passwd entry; skip the lookup rather than // failing with a spurious "user record not found" error. let user_name_is_numeric = user_name.is_some_and(|n| n.parse::().is_ok()); + let group_name_is_numeric = group_name.is_some_and(|n| n.parse::().is_ok()); let user = if user_name.is_some() && !user_name_is_numeric { Some( User::from_uid(target_uid) @@ -1405,9 +1236,16 @@ pub fn drop_privileges(policy: &SandboxPolicy) -> Result<()> { None }; - // Set supplementary groups only when we have a name-based identity. - // Numeric UIDs may not have a passwd entry, so initgroups would fail. - if let Some(ref user) = user + // Resolved image/fixed identities are normalized to numeric UID/GID and + // must never inherit supervisor supplementary groups. Legacy named policy + // identities retain their existing initgroups behavior. + #[cfg(target_os = "linux")] + if user_name_is_numeric || group_name_is_numeric { + clear_supplementary_groups()?; + } + if !user_name_is_numeric + && !group_name_is_numeric + && let Some(ref user) = user && target_uid != nix::unistd::geteuid() { let user_cstr = @@ -1481,6 +1319,39 @@ pub fn drop_privileges(policy: &SandboxPolicy) -> Result<()> { Ok(()) } +#[cfg(target_os = "linux")] +fn clear_supplementary_groups() -> Result<()> { + let before = rustix::process::getgroups().into_diagnostic()?; + let clear_result = + rustix::thread::set_thread_groups(&[]).map_err(rustix::io::Errno::raw_os_error); + validate_supplementary_group_clear(clear_result, before.is_empty(), true)?; + let remaining = rustix::process::getgroups().into_diagnostic()?; + if !remaining.is_empty() { + return Err(miette::miette!( + "supplementary group clear verification failed: groups remain: {remaining:?}" + )); + } + Ok(()) +} + +#[cfg(any(test, target_os = "linux"))] +fn validate_supplementary_group_clear( + clear_result: std::result::Result<(), i32>, + groups_were_empty: bool, + allow_denied_when_empty: bool, +) -> Result<()> { + match clear_result { + Ok(()) => Ok(()), + Err(error) if error == libc::EPERM && groups_were_empty && allow_denied_when_empty => { + debug!("setgroups is denied, but the supplementary group list is already empty"); + Ok(()) + } + Err(error) => Err(miette::miette!( + "failed to clear supplementary groups before privilege drop: {error}" + )), + } +} + /// Process exit status. #[derive(Debug, Clone, Copy)] pub struct ProcessStatus { @@ -1997,24 +1868,43 @@ mod tests { } } + #[cfg(unix)] + fn prepare_read_write_path_as_current(path: &Path) -> Result { + prepare_read_write_path( + path, + Some(nix::unistd::geteuid()), + Some(nix::unistd::getegid()), + ) + } + #[cfg(unix)] #[test] fn prepare_read_write_path_creates_missing_directory() { + use std::os::unix::fs::MetadataExt; + let dir = tempfile::tempdir().unwrap(); - let missing = dir.path().join("missing").join("nested"); + let missing = dir + .path() + .canonicalize() + .unwrap() + .join("missing") + .join("nested"); - assert!(prepare_read_write_path(&missing).unwrap()); + assert!(prepare_read_write_path_as_current(&missing).unwrap()); assert!(missing.is_dir()); + let metadata = std::fs::metadata(&missing).unwrap(); + assert_eq!(metadata.uid(), nix::unistd::geteuid().as_raw()); + assert_eq!(metadata.gid(), nix::unistd::getegid().as_raw()); } #[cfg(unix)] #[test] fn prepare_read_write_path_preserves_existing_directory() { let dir = tempfile::tempdir().unwrap(); - let existing = dir.path().join("existing"); + let existing = dir.path().canonicalize().unwrap().join("existing"); std::fs::create_dir(&existing).unwrap(); - assert!(!prepare_read_write_path(&existing).unwrap()); + assert!(!prepare_read_write_path_as_current(&existing).unwrap()); assert!(existing.is_dir()); } @@ -2024,18 +1914,39 @@ mod tests { use std::os::unix::fs::symlink; let dir = tempfile::tempdir().unwrap(); - let target = dir.path().join("target"); - let link = dir.path().join("link"); + let root = dir.path().canonicalize().unwrap(); + let target = root.join("target"); + let link = root.join("link"); std::fs::create_dir(&target).unwrap(); symlink(&target, &link).unwrap(); - let error = prepare_read_write_path(&link).unwrap_err(); + let error = prepare_read_write_path_as_current(&link).unwrap_err(); + assert!( + error.to_string().contains("refusing unsafe directory path"), + "unexpected error: {error}" + ); + } + + #[cfg(unix)] + #[test] + fn prepare_read_write_path_rejects_symlink_ancestor() { + use std::os::unix::fs::symlink; + + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().canonicalize().unwrap(); + let real = root.join("real"); + let linked = root.join("linked"); + std::fs::create_dir(&real).unwrap(); + symlink(&real, &linked).unwrap(); + let requested = linked.join("nested/path"); + + let error = prepare_read_write_path_as_current(&requested).unwrap_err(); assert!( error .to_string() - .contains("is a symlink — refusing to chown"), - "unexpected error: {error}" + .contains("symlink or non-directory ancestor") ); + assert!(!real.join("nested/path").exists()); } #[cfg(unix)] @@ -2059,7 +1970,7 @@ mod tests { } let dir = tempfile::tempdir().unwrap(); - let existing = dir.path().join("existing"); + let existing = dir.path().canonicalize().unwrap().join("existing"); std::fs::create_dir(&existing).unwrap(); let before = std::fs::metadata(&existing).unwrap(); @@ -2076,54 +1987,15 @@ mod tests { assert_eq!(after.gid(), before.gid()); } - #[cfg(unix)] - #[test] - #[allow(clippy::similar_names)] - fn chown_sandbox_home_changes_ownership_recursively() { - use std::os::unix::fs::MetadataExt; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path().join("sandbox"); - std::fs::create_dir(&root).unwrap(); - std::fs::write(root.join("file.txt"), "hello").unwrap(); - std::fs::create_dir(root.join("subdir")).unwrap(); - std::fs::write(root.join("subdir").join("nested.txt"), "world").unwrap(); - - let expected_uid = nix::unistd::geteuid(); - let expected_gid = nix::unistd::getegid(); - - chown_sandbox_home(&root, Some(expected_uid), Some(expected_gid)).unwrap(); - - for path in &[ - root.clone(), - root.join("file.txt"), - root.join("subdir"), - root.join("subdir").join("nested.txt"), - ] { - let meta = std::fs::metadata(path).unwrap(); - assert_eq!( - meta.uid(), - expected_uid.as_raw(), - "uid mismatch for {}", - path.display() - ); - assert_eq!( - meta.gid(), - expected_gid.as_raw(), - "gid mismatch for {}", - path.display() - ); - } - } - #[cfg(unix)] #[test] fn chown_sandbox_home_rejects_symlink_root() { use std::os::unix::fs::symlink; let dir = tempfile::tempdir().unwrap(); - let target = dir.path().join("real"); - let link = dir.path().join("link"); + let temp_root = dir.path().canonicalize().unwrap(); + let target = temp_root.join("real"); + let link = temp_root.join("link"); std::fs::create_dir(&target).unwrap(); symlink(&target, &link).unwrap(); @@ -2141,185 +2013,17 @@ mod tests { #[cfg(unix)] #[test] - fn chown_sandbox_home_skips_symlink_children() { - use std::os::unix::fs::symlink; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path().join("sandbox"); - std::fs::create_dir(&root).unwrap(); - let target = dir.path().join("outside"); - std::fs::write(&target, "secret").unwrap(); - symlink(&target, root.join("link")).unwrap(); - - chown_sandbox_home( - &root, - Some(nix::unistd::geteuid()), - Some(nix::unistd::getegid()), - ) - .expect("should skip symlink children without error"); - } - - #[cfg(unix)] - #[test] - fn chown_recursive_skips_erofs_subtree_but_continues_siblings() { - use std::sync::{Arc, Mutex}; - - let dir = tempfile::tempdir().unwrap(); - let root = dir.path().join("sandbox"); - std::fs::create_dir(&root).unwrap(); - - let readonly_dir = root.join("ro-mount"); - std::fs::create_dir(&readonly_dir).unwrap(); - std::fs::write(readonly_dir.join("child-under-ro.txt"), "data").unwrap(); - - std::fs::write(root.join("writable-sibling.txt"), "data").unwrap(); - - let uid = Some(nix::unistd::geteuid()); - let gid = Some(nix::unistd::getegid()); - - let chowned: Arc>> = Arc::new(Mutex::new(Vec::new())); - let chowned_ref = Arc::clone(&chowned); - - let readonly_dir_clone = readonly_dir.clone(); - let fake_chown = - move |path: &Path, _uid: Option, _gid: Option| -> nix::Result<()> { - if path == readonly_dir_clone { - return Err(nix::errno::Errno::EROFS); - } - chowned_ref.lock().unwrap().push(path.to_path_buf()); - Ok(()) - }; - - chown_children(&root, uid, gid, &fake_chown).expect("EROFS should be handled gracefully"); - - let chowned = chowned.lock().unwrap(); + fn supplementary_group_clear_accepts_success() { assert!( - !chowned.contains(&readonly_dir.join("child-under-ro.txt")), - "children under EROFS directory must NOT be descended into" + validate_supplementary_group_clear(Ok(()), true, cfg!(target_os = "linux")).is_ok() ); - assert!( - chowned.contains(&root.join("writable-sibling.txt")), - "writable sibling should still be chowned" - ); - } - - #[cfg(unix)] - #[test] - fn chown_recursive_propagates_non_erofs_errors() { - let dir = tempfile::tempdir().unwrap(); - let root = dir.path().join("sandbox"); - std::fs::create_dir(&root).unwrap(); - - let uid = Some(nix::unistd::geteuid()); - let gid = Some(nix::unistd::getegid()); - - let fake_chown = |_path: &Path, _uid: Option, _gid: Option| -> nix::Result<()> { - Err(nix::errno::Errno::EPERM) - }; - - let result = chown_recursive(&root, uid, gid, &fake_chown); - assert!(result.is_err(), "non-EROFS errors should propagate"); - } - - #[cfg(unix)] - #[test] - fn chown_children_skips_all_erofs_children_gracefully() { - let dir = tempfile::tempdir().unwrap(); - let root = dir.path().join("sandbox"); - std::fs::create_dir(&root).unwrap(); - std::fs::create_dir(root.join("a")).unwrap(); - std::fs::write(root.join("b.txt"), "data").unwrap(); - - let uid = Some(nix::unistd::geteuid()); - let gid = Some(nix::unistd::getegid()); - - let always_erofs = |_path: &Path, - _uid: Option, - _gid: Option| - -> nix::Result<()> { Err(nix::errno::Errno::EROFS) }; - - chown_children(&root, uid, gid, &always_erofs) - .expect("EROFS on all children should be skipped gracefully"); - } - - #[cfg(unix)] - #[test] - fn rewrite_passwd_modifies_existing_sandbox_entry() { - let dir = tempfile::tempdir().unwrap(); - let passwd = dir.path().join("passwd"); - std::fs::write( - &passwd, - "root:x:0:0:root:/root:/bin/bash\nsandbox:x:1000:1000::/sandbox:/bin/bash\n", - ) - .unwrap(); - - rewrite_passwd_at(&passwd, "5000", "6000").unwrap(); - - let content = std::fs::read_to_string(&passwd).unwrap(); - assert!(content.contains("sandbox:x:5000:6000::/sandbox:/bin/bash")); - assert!(content.contains("root:x:0:0:root:/root:/bin/bash")); - } - - #[cfg(unix)] - #[test] - fn rewrite_passwd_appends_when_no_sandbox_entry() { - let dir = tempfile::tempdir().unwrap(); - let passwd = dir.path().join("passwd"); - std::fs::write(&passwd, "root:x:0:0:root:/root:/bin/bash\n").unwrap(); - - rewrite_passwd_at(&passwd, "5000", "6000").unwrap(); - - let content = std::fs::read_to_string(&passwd).unwrap(); - assert!(content.contains("root:x:0:0:root:/root:/bin/bash")); - assert!(content.contains("sandbox:x:5000:6000::/sandbox:/bin/sh")); - } - - #[cfg(unix)] - #[test] - fn rewrite_group_modifies_existing_sandbox_entry() { - let dir = tempfile::tempdir().unwrap(); - let group = dir.path().join("group"); - std::fs::write(&group, "root:x:0:\nsandbox:x:1000:\n").unwrap(); - - rewrite_group_at(&group, "6000").unwrap(); - - let content = std::fs::read_to_string(&group).unwrap(); - assert!(content.contains("sandbox:x:6000:")); - assert!(content.contains("root:x:0:")); - } - - #[cfg(unix)] - #[test] - fn rewrite_group_appends_when_no_sandbox_entry() { - let dir = tempfile::tempdir().unwrap(); - let group = dir.path().join("group"); - std::fs::write(&group, "root:x:0:\n").unwrap(); - - rewrite_group_at(&group, "6000").unwrap(); - - let content = std::fs::read_to_string(&group).unwrap(); - assert!(content.contains("root:x:0:")); - assert!(content.contains("sandbox:x:6000:")); } - #[cfg(unix)] #[test] - fn rewrite_passwd_preserves_other_entries() { - let dir = tempfile::tempdir().unwrap(); - let passwd = dir.path().join("passwd"); - std::fs::write( - &passwd, - "root:x:0:0:root:/root:/bin/bash\nnobody:x:65534:65534:nobody:/:/usr/sbin/nologin\nsandbox:x:1000:1000::/sandbox:/bin/bash\n", - ) - .unwrap(); - - rewrite_passwd_at(&passwd, "1234567", "1234567").unwrap(); - - let content = std::fs::read_to_string(&passwd).unwrap(); - assert!(content.contains("root:x:0:0:root:/root:/bin/bash")); - assert!(content.contains("nobody:x:65534:65534:nobody:/:/usr/sbin/nologin")); - assert!(content.contains("sandbox:x:1234567:1234567::/sandbox:/bin/bash")); - assert_eq!(content.lines().count(), 3); + fn supplementary_group_clear_accepts_linux_eperm_only_when_already_empty() { + assert!(validate_supplementary_group_clear(Err(libc::EPERM), true, true).is_ok()); + assert!(validate_supplementary_group_clear(Err(libc::EPERM), false, true).is_err()); + assert!(validate_supplementary_group_clear(Err(libc::EPERM), true, false).is_err()); } #[tokio::test] diff --git a/crates/openshell-supervisor-process/src/run.rs b/crates/openshell-supervisor-process/src/run.rs index 3cb29d1e6b..aeb1677abd 100644 --- a/crates/openshell-supervisor-process/src/run.rs +++ b/crates/openshell-supervisor-process/src/run.rs @@ -36,6 +36,8 @@ use openshell_core::denial::DenialEvent; use crate::managed_children; use crate::process::{ProcessEnforcementMode, ProcessHandle}; +const MANAGED_IDENTITY_SESSION_TIMEOUT: Duration = Duration::from_secs(60); + fn ocsf_ctx() -> &'static openshell_ocsf::SandboxContext { openshell_ocsf::ctx::ctx() } @@ -66,20 +68,14 @@ pub async fn run_process( provider_env: std::collections::HashMap, ca_file_paths: Option<(std::path::PathBuf, std::path::PathBuf)>, agent_proposals: AgentProposals, + managed_identity_required: bool, + resolved_identity: Option, #[cfg(target_os = "linux")] netns: Option<&NetworkNamespace>, #[cfg(target_os = "linux")] bypass_denial_tx: Option< tokio::sync::mpsc::UnboundedSender, >, #[cfg(target_os = "linux")] bypass_activity_tx: Option, ) -> Result { - // When a driver injects a custom UID/GID, update /etc/passwd and - // /etc/group so the "sandbox" entry matches. Must run before - // validate_sandbox_user so passwd lookups see the correct identity. - #[cfg(unix)] - if enforcement_mode.uses_privileged_process_setup() { - crate::process::update_sandbox_passwd_entries()?; - } - // Validate that the sandbox user exists in the image. All sandbox images // must include a "sandbox" user for privilege dropping; failing fast here // beats silently running children as root. @@ -222,6 +218,34 @@ pub async fn run_process( let ssh_proxy_url = ssh_proxy_url_for_policy(policy, None); let ssh_socket_path: Option = ssh_socket_path.map(std::path::PathBuf::from); + + // A marked sandbox must persist its exact identity at the gateway before + // exposing either the SSH child-launch path or the direct entrypoint path. + if managed_identity_required { + let identity = resolved_identity.clone().ok_or_else(|| { + miette::miette!( + "gateway requires managed agent identity, but no resolved identity is available" + ) + })?; + let (Some(endpoint), Some(id), Some(socket)) = + (openshell_endpoint, sandbox_id, ssh_socket_path.as_ref()) + else { + return Err(miette::miette!( + "gateway requires managed agent identity, but supervisor session metadata is incomplete (endpoint, sandbox ID, and SSH socket are required)" + )); + }; + let (_session, ready) = crate::supervisor_session::spawn( + endpoint.to_string(), + id.to_string(), + socket.clone(), + ssh_netns_fd, + None, + Some(identity), + ); + info!("supervisor session task spawned"); + await_managed_identity_acceptance(ready).await?; + } + if let Some(listen_path) = ssh_socket_path.clone() { let policy_clone = policy.clone(); let workdir_clone = workdir.map(str::to_string); @@ -294,18 +318,19 @@ pub async fn run_process( } } - // Spawn the persistent supervisor session if we have a gateway endpoint - // and sandbox identity. The session provides relay channels for SSH - // connect and ExecSandbox through the gateway. - if let (Some(endpoint), Some(id), Some(socket)) = - (openshell_endpoint, sandbox_id, ssh_socket_path.as_ref()) + // Legacy sessions preserve the prior ordering: bind SSH first, then start + // the gateway relay session without blocking entrypoint launch. + if !managed_identity_required + && let (Some(endpoint), Some(id), Some(socket)) = + (openshell_endpoint, sandbox_id, ssh_socket_path.as_ref()) { - crate::supervisor_session::spawn( + let _ = crate::supervisor_session::spawn( endpoint.to_string(), id.to_string(), socket.clone(), ssh_netns_fd, None, + None, ); info!("supervisor session task spawned"); } @@ -392,6 +417,24 @@ pub async fn run_process( Ok(status.code()) } +async fn await_managed_identity_acceptance( + ready: tokio::sync::oneshot::Receiver<()>, +) -> Result<()> { + match timeout(MANAGED_IDENTITY_SESSION_TIMEOUT, ready).await { + Ok(Ok(())) => { + info!("managed agent identity accepted by gateway"); + Ok(()) + } + Ok(Err(_)) => Err(miette::miette!( + "supervisor session ended before managed agent identity was accepted" + )), + Err(_) => Err(miette::miette!( + "gateway did not accept managed agent identity within {} seconds", + MANAGED_IDENTITY_SESSION_TIMEOUT.as_secs() + )), + } +} + fn ssh_proxy_url_for_policy( policy: &SandboxPolicy, netns_proxy_host: Option, @@ -509,4 +552,45 @@ mod tests { assert_eq!(ssh_proxy_url_for_policy(&policy, None), None); } + + #[tokio::test] + async fn managed_identity_gate_blocks_ssh_and_direct_setup_until_acceptance() { + let (accept_tx, accept_rx) = tokio::sync::oneshot::channel(); + let (waiting_tx, waiting_rx) = tokio::sync::oneshot::channel(); + let ssh_setup = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let direct_setup = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let ssh_setup_task = Arc::clone(&ssh_setup); + let direct_setup_task = Arc::clone(&direct_setup); + + let gated_setup = tokio::spawn(async move { + waiting_tx.send(()).unwrap(); + await_managed_identity_acceptance(accept_rx).await.unwrap(); + ssh_setup_task.store(true, Ordering::Release); + direct_setup_task.store(true, Ordering::Release); + }); + + waiting_rx.await.unwrap(); + tokio::task::yield_now().await; + assert!(!ssh_setup.load(Ordering::Acquire)); + assert!(!direct_setup.load(Ordering::Acquire)); + + accept_tx.send(()).unwrap(); + gated_setup.await.unwrap(); + assert!(ssh_setup.load(Ordering::Acquire)); + assert!(direct_setup.load(Ordering::Acquire)); + } + + #[tokio::test] + async fn managed_identity_gate_fails_when_session_ends_before_acceptance() { + let (tx, rx) = tokio::sync::oneshot::channel::<()>(); + drop(tx); + let error = await_managed_identity_acceptance(rx) + .await + .expect_err("closed session must keep launch gate closed"); + assert!( + error + .to_string() + .contains("before managed agent identity was accepted") + ); + } } diff --git a/crates/openshell-supervisor-process/src/ssh.rs b/crates/openshell-supervisor-process/src/ssh.rs index f5a3ee0793..8d67f90215 100644 --- a/crates/openshell-supervisor-process/src/ssh.rs +++ b/crates/openshell-supervisor-process/src/ssh.rs @@ -681,33 +681,10 @@ impl Default for PtyRequest { } } -/// Derive the session USER and HOME from the policy's `run_as_user`. -/// -/// For name-based identities, looks up the home directory via `/etc/passwd` -/// (or defaults to `/home/{user}`). -/// -/// For numeric UIDs, there is no passwd entry — falls back to -/// `("{uid}", "/sandbox")` so the agent session still has a meaningful -/// USER identifier. +/// Derive session presentation from the resolved identity with legacy fallback. fn session_user_and_home(policy: &SandboxPolicy) -> (String, String) { - match policy.process.run_as_user.as_deref() { - Some(user) if !user.is_empty() => { - // Numeric UID — no passwd entry expected; use default HOME. - if user.parse::().is_ok() { - return (user.to_string(), "/sandbox".to_string()); - } - // Name-based identity — look up home from /etc/passwd. - let home = nix::unistd::User::from_name(user) - .ok() - .flatten() - .map_or_else( - || format!("/home/{user}"), - |u| u.dir.to_string_lossy().into_owned(), - ); - (user.to_string(), home) - } - _ => ("sandbox".to_string(), "/sandbox".to_string()), - } + let identity = child_env::identity_env_vars(policy); + (identity[1].1.clone(), identity[0].1.clone()) } #[allow(clippy::too_many_arguments)] @@ -725,8 +702,6 @@ fn apply_child_env( cmd.env_clear() .env(openshell_core::sandbox_env::SANDBOX, "1") - .env("HOME", session_home) - .env("USER", session_user) .env("SHELL", "/bin/bash") .env("PATH", &path) .env("TERM", term); @@ -755,6 +730,11 @@ fn apply_child_env( } cmd.env(key, value); } + + // Identity environment is authoritative over user/provider input. + cmd.env("HOME", session_home) + .env("USER", session_user) + .env("LOGNAME", session_user); } #[allow(clippy::too_many_arguments)] @@ -1675,7 +1655,7 @@ mod tests { } #[test] - fn session_user_and_home_returns_name_from_passwd() { + fn session_user_and_home_preserves_legacy_policy_name() { use openshell_core::policy::{ FilesystemPolicy, LandlockPolicy, NetworkPolicy, ProcessPolicy, }; @@ -1691,8 +1671,7 @@ mod tests { }; let (user, home) = session_user_and_home(&policy); assert_eq!(user, "sandbox"); - // Name-based — should resolve via passwd (or /home/{user}). - assert!(!home.is_empty()); + assert_eq!(home, "/sandbox"); } #[test] diff --git a/crates/openshell-supervisor-process/src/supervisor_session.rs b/crates/openshell-supervisor-process/src/supervisor_session.rs index 594d865599..10f7cc415e 100644 --- a/crates/openshell-supervisor-process/src/supervisor_session.rs +++ b/crates/openshell-supervisor-process/src/supervisor_session.rs @@ -21,12 +21,13 @@ use openshell_core::proto::{ SupervisorHello, SupervisorMessage, TcpRelayTarget, gateway_message, relay_open, supervisor_message, }; +use openshell_core::sandbox_env::ResolvedAgentIdentity as RuntimeResolvedAgentIdentity; use openshell_ocsf::{ ActivityId, ConnectionInfo, Endpoint, NetworkActivityBuilder, OcsfEvent, SandboxContext, SeverityId, StatusId, ocsf_emit, }; use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; -use tokio::sync::mpsc; +use tokio::sync::{mpsc, oneshot}; use tokio_stream::StreamExt; use tracing::{debug, warn}; @@ -236,14 +237,19 @@ pub fn spawn( ssh_socket_path: std::path::PathBuf, netns_fd: Option, expected_ssh_peer_pid: Option, -) -> tokio::task::JoinHandle<()> { - tokio::spawn(run_session_loop( + resolved_identity: Option, +) -> (tokio::task::JoinHandle<()>, oneshot::Receiver<()>) { + let (ready_tx, ready_rx) = oneshot::channel(); + let handle = tokio::spawn(run_session_loop( endpoint, sandbox_id, ssh_socket_path, netns_fd, expected_ssh_peer_pid, - )) + resolved_identity, + Some(ready_tx), + )); + (handle, ready_rx) } async fn run_session_loop( @@ -252,6 +258,8 @@ async fn run_session_loop( ssh_socket_path: std::path::PathBuf, netns_fd: Option, expected_ssh_peer_pid: Option, + resolved_identity: Option, + mut ready_tx: Option>, ) { let mut backoff = INITIAL_BACKOFF; let mut attempt: u64 = 0; @@ -265,6 +273,8 @@ async fn run_session_loop( &ssh_socket_path, netns_fd, expected_ssh_peer_pid, + resolved_identity.as_ref(), + &mut ready_tx, ) .await { @@ -295,6 +305,8 @@ async fn run_single_session( ssh_socket_path: &std::path::Path, netns_fd: Option, expected_ssh_peer_pid: Option, + resolved_identity: Option<&RuntimeResolvedAgentIdentity>, + ready_tx: &mut Option>, ) -> Result<(), Box> { // Connect to the gateway. The same `Channel` is used for both the // long-lived control stream and all data-plane `RelayStream` calls, so @@ -312,10 +324,11 @@ async fn run_single_session( // Send hello as the first message. let instance_id = uuid::Uuid::new_v4().to_string(); tx.send(SupervisorMessage { - payload: Some(supervisor_message::Payload::Hello(SupervisorHello { - sandbox_id: sandbox_id.to_string(), - instance_id: instance_id.clone(), - })), + payload: Some(supervisor_message::Payload::Hello(supervisor_hello( + sandbox_id, + &instance_id, + resolved_identity, + ))), }) .await .map_err(|_| "failed to queue hello")?; @@ -342,6 +355,9 @@ async fn run_single_session( }; let heartbeat_secs = accepted.heartbeat_interval_secs.max(5); + if let Some(ready_tx) = ready_tx.take() { + let _ = ready_tx.send(()); + } let event = session_established_event( openshell_ocsf::ctx::ctx(), endpoint, @@ -383,6 +399,19 @@ async fn run_single_session( } } +fn supervisor_hello( + sandbox_id: &str, + instance_id: &str, + resolved_identity: Option<&RuntimeResolvedAgentIdentity>, +) -> SupervisorHello { + SupervisorHello { + sandbox_id: sandbox_id.to_string(), + instance_id: instance_id.to_string(), + resolved_identity: resolved_identity + .map(openshell_core::proto::ResolvedAgentIdentity::from), + } +} + fn handle_gateway_message( msg: &GatewayMessage, sandbox_id: &str, @@ -646,7 +675,7 @@ async fn connect_tcp_target( netns_fd: Option, ) -> Result> { if let Some(fd) = netns_fd { - let (tx, rx) = tokio::sync::oneshot::channel(); + let (tx, rx) = oneshot::channel(); std::thread::spawn(move || { let result = (|| -> std::io::Result { #[allow(unsafe_code)] @@ -753,6 +782,54 @@ mod target_tests { let err = validate_tcp_target(&tcp("127.0.0.1", 70000)).expect_err("too large rejected"); assert_eq!(err, "tcp target port must be between 1 and 65535"); } + + #[test] + fn managed_supervisor_hello_reports_resolved_identity() { + let runtime_identity = RuntimeResolvedAgentIdentity { + uid: 1234, + gid: 2345, + presentation_user: "agent".to_string(), + source: openshell_core::sandbox_env::IdentitySource::Image, + image_id: Some("sha256:image".to_string()), + }; + + let hello = supervisor_hello("sbx", "instance", Some(&runtime_identity)); + let identity = hello + .resolved_identity + .expect("managed hello must carry identity"); + assert_eq!( + identity.version, + openshell_core::sandbox_env::RESOLVED_AGENT_IDENTITY_VERSION + ); + assert_eq!(identity.source, "image"); + assert_eq!(identity.image_id, "sha256:image"); + assert_eq!(identity.uid, 1234); + assert_eq!(identity.gid, 2345); + assert!(identity.supplementary_gids.is_empty()); + } + + #[test] + fn legacy_supervisor_hello_omits_resolved_identity() { + let hello = supervisor_hello("sbx", "instance", None); + assert!(hello.resolved_identity.is_none()); + } + + #[test] + fn fixed_supervisor_hello_preserves_immutable_image_id() { + let runtime_identity = RuntimeResolvedAgentIdentity { + uid: 1234, + gid: 2345, + presentation_user: "1234".to_string(), + source: openshell_core::sandbox_env::IdentitySource::Fixed, + image_id: Some("sha256:fixed-image".to_string()), + }; + + let identity = supervisor_hello("sbx", "instance", Some(&runtime_identity)) + .resolved_identity + .unwrap(); + assert_eq!(identity.source, "fixed"); + assert_eq!(identity.image_id, "sha256:fixed-image"); + } } #[cfg(test)] diff --git a/deploy/docker/gateway.toml b/deploy/docker/gateway.toml index 4fe84d633a..e590ef3b5b 100644 --- a/deploy/docker/gateway.toml +++ b/deploy/docker/gateway.toml @@ -34,13 +34,22 @@ compute_drivers = ["docker"] disable_tls = true [openshell.drivers.docker] -# Default image pulled for `openshell sandbox create` without --from. +# Default image pulled for `openshell sandbox create` without --from. In the +# default image identity mode, this image must declare a non-root OCI USER. default_image = "ghcr.io/nvidia/openshell-community/sandboxes/base:latest" # Supervisor image from which the openshell-sandbox binary is extracted on # first start. The binary is cached to XDG_DATA_HOME and reused on restart. supervisor_image = "ghcr.io/nvidia/openshell/supervisor:latest" # Only pull images that are not already cached locally. image_pull_policy = "IfNotPresent" +# Run agent children as the image's non-root OCI USER. The supervisor still +# runs as root for sandbox setup. This is the default when omitted. +identity_source = "image" +# For operator-controlled external or shared storage, use fixed mode and set +# both IDs. Docker has no implicit 10001:10001 identity. +# identity_source = "fixed" +# fixed_uid = 1000 +# fixed_gid = 1000 # Prefix applied to sandbox container names. sandbox_namespace = "openshell" # Address sandbox containers use to call back to the gateway. diff --git a/docs/reference/gateway-config.mdx b/docs/reference/gateway-config.mdx index 75ac6f7382..14fff8d50e 100644 --- a/docs/reference/gateway-config.mdx +++ b/docs/reference/gateway-config.mdx @@ -325,6 +325,13 @@ socket_path = "/var/run/docker.sock" default_image = "ghcr.io/nvidia/openshell/sandbox:latest" # Docker vocabulary: Always | IfNotPresent | Never. Empty behaves like IfNotPresent. image_pull_policy = "IfNotPresent" +# Use the image's non-root OCI USER for agent children. This is the default. +identity_source = "image" +# For operator-controlled external or shared storage, select "fixed" and set +# both IDs. There is no implicit 10001:10001 local-driver identity. +# identity_source = "fixed" +# fixed_uid = 1000 +# fixed_gid = 1000 sandbox_namespace = "docker-dev" # Empty auto-detects https://host.openshell.internal: when guest TLS is set. grpc_endpoint = "https://host.openshell.internal:17670" @@ -367,6 +374,13 @@ compute_drivers = ["podman"] socket_path = "/run/user/1000/podman/podman.sock" default_image = "ghcr.io/nvidia/openshell/sandbox:latest" image_pull_policy = "missing" # always | missing | never | newer +# Use the image's non-root OCI USER for agent children. This is the default. +identity_source = "image" +# For operator-controlled external or shared storage, select "fixed" and set +# both IDs. There is no implicit 10001:10001 local-driver identity. +# identity_source = "fixed" +# fixed_uid = 1000 +# fixed_gid = 1000 grpc_endpoint = "https://host.containers.internal:17670" # The gateway overwrites gateway_port from bind_address at runtime. gateway_port = 17670 @@ -453,6 +467,10 @@ health_check_interval_secs = 10 # proxy_connect_by_hostname = true ``` +For Docker and Podman, `identity_source` accepts `image` or `fixed` and defaults to `image`. In image mode, omit `fixed_uid` and `fixed_gid`. In fixed mode, set both fields to values from `1000` through `2000000000`, inclusive. OpenShell rejects partial fixed identity configuration and does not supply an implicit `10001:10001` identity. + +Image mode reads the non-root OCI `Config.User` from the final inspected image. It rejects images with no `USER`, identities that resolve to UID or GID 0, and creator-selected bind or named-volume mounts. Use fixed mode when an operator intentionally attaches external or shared storage under a known numeric principal. See [Sandbox Compute Drivers](./sandbox-compute-drivers#sandbox-user-identity) for accepted OCI `USER` forms, mount restrictions, and runtime behavior. + ### MicroVM Each sandbox runs inside its own libkrun microVM managed by the standalone `openshell-driver-vm` subprocess. Use this driver when you want stronger isolation than container namespaces alone. diff --git a/docs/reference/sandbox-compute-drivers.mdx b/docs/reference/sandbox-compute-drivers.mdx index e0eec81285..7470ffb479 100644 --- a/docs/reference/sandbox-compute-drivers.mdx +++ b/docs/reference/sandbox-compute-drivers.mdx @@ -115,7 +115,7 @@ The gateway talks to the Docker daemon to create sandbox containers. Docker is a For maintainer-level implementation details, refer to the [Docker driver README](https://github.com/NVIDIA/OpenShell/blob/main/crates/openshell-driver-docker/README.md). -Select Docker with `compute_drivers = ["docker"]` in `[openshell.gateway]`. Configure Docker driver values such as `socket_path`, `grpc_endpoint`, `network_name`, `supervisor_bin`, `supervisor_image`, `image_pull_policy`, `ssh_socket_path`, `sandbox_pids_limit`, and `guest_tls_*` in `[openshell.drivers.docker]`. When `socket_path` is unset, the driver uses the same responsive local socket selected by auto-detection. An explicitly selected Docker driver falls back to `/var/run/docker.sock` when no candidate responds. +Select Docker with `compute_drivers = ["docker"]` in `[openshell.gateway]`. Configure Docker driver values such as `socket_path`, `grpc_endpoint`, `network_name`, `supervisor_bin`, `supervisor_image`, `image_pull_policy`, `identity_source`, `fixed_uid`, `fixed_gid`, `ssh_socket_path`, `sandbox_pids_limit`, and `guest_tls_*` in `[openshell.drivers.docker]`. When `socket_path` is unset, the driver uses the same responsive local socket selected by auto-detection. An explicitly selected Docker driver falls back to `/var/run/docker.sock` when no candidate responds. For GPU-backed Docker sandboxes, configure Docker CDI before starting the gateway so OpenShell can detect the daemon capability. @@ -123,12 +123,21 @@ For GPU-backed Docker sandboxes, configure Docker CDI before starting the gatewa Docker driver config accepts user-supplied `volume` and `tmpfs` mounts. It also accepts `bind` mounts when `[openshell.drivers.docker]` sets -`enable_bind_mounts = true` in `gateway.toml`. See Docker's [storage documentation](https://docs.docker.com/engine/storage/) for more information. +`enable_bind_mounts = true` in `gateway.toml`. Creator-selected `bind` and +`volume` mounts require `identity_source = "fixed"`; image mode rejects them +even when bind mounts are globally enabled. See Docker's [storage documentation](https://docs.docker.com/engine/storage/) for more information. Docker local-driver named volumes created with bind options also expose gateway-host paths, so OpenShell treats them like bind mounts and requires `enable_bind_mounts = true`. -Use a `volume` mount for existing Docker named volumes: +Use fixed identity mode for an existing Docker named volume: + +```toml +[openshell.drivers.docker] +identity_source = "fixed" +fixed_uid = 1000 +fixed_gid = 1000 +``` ```shell docker volume create openshell-work @@ -180,7 +189,7 @@ The gateway talks to the Podman API socket. The Podman driver requires Podman 5. For maintainer-level implementation details, refer to the [Podman driver README](https://github.com/NVIDIA/OpenShell/blob/main/crates/openshell-driver-podman/README.md) and [Podman networking notes](https://github.com/NVIDIA/OpenShell/blob/main/crates/openshell-driver-podman/NETWORKING.md). -Select Podman with `compute_drivers = ["podman"]` in `[openshell.gateway]`. Configure Podman driver values such as `socket_path`, `network_name`, `supervisor_image`, `stop_timeout_secs`, `image_pull_policy`, `grpc_endpoint`, `host_gateway_ip`, `sandbox_ssh_socket_path`, `sandbox_pids_limit`, and `guest_tls_*` in `[openshell.drivers.podman]`. +Select Podman with `compute_drivers = ["podman"]` in `[openshell.gateway]`. Configure Podman driver values such as `socket_path`, `network_name`, `supervisor_image`, `stop_timeout_secs`, `image_pull_policy`, `identity_source`, `fixed_uid`, `fixed_gid`, `grpc_endpoint`, `host_gateway_ip`, `sandbox_ssh_socket_path`, `sandbox_pids_limit`, and `guest_tls_*` in `[openshell.drivers.podman]`. For proxy-required networks, the Podman driver also accepts the corporate egress proxy keys `https_proxy`, `no_proxy`, `proxy_auth_file`, `proxy_auth_allow_insecure`, and `proxy_connect_by_hostname`. The supervisor chains policy-approved TLS tunnels through the proxy with HTTP CONNECT instead of dialing destinations directly. See the [Gateway Configuration File](./gateway-config) reference for the full contract, including the cleartext-credential acknowledgement and the validated-IP CONNECT behavior. @@ -190,13 +199,24 @@ On macOS with `podman machine`, the driver uses gvproxy's host-loopback IP, `192 Podman driver config accepts user-supplied `volume`, `tmpfs`, and `image` mounts. It also accepts `bind` mounts when `[openshell.drivers.podman]` sets -`enable_bind_mounts = true` in `gateway.toml`. Podman local-driver named -volumes created with bind options also expose gateway-host paths, so -OpenShell treats them like bind mounts and requires `enable_bind_mounts = true`. +`enable_bind_mounts = true` in `gateway.toml`. Creator-selected `bind` and +`volume` mounts require `identity_source = "fixed"`; image mode rejects them +even when bind mounts are globally enabled. Image mode permits Podman `image` +mounts only when they are read-only. The driver resolves those image sources +to immutable image IDs before container creation. Podman local-driver named +volumes created with bind options also expose gateway-host paths, so OpenShell +treats them like bind mounts and requires `enable_bind_mounts = true`. Host bind mounts expose gateway host paths to sandbox requests, so they are disabled by default. -Use a `volume` mount for existing Podman named volumes: +Use fixed identity mode for an existing Podman named volume: + +```toml +[openshell.drivers.podman] +identity_source = "fixed" +fixed_uid = 1000 +fixed_gid = 1000 +``` ```shell podman volume create openshell-work @@ -413,11 +433,69 @@ image. ## Sandbox User Identity -OpenShell accepts both the hardcoded username `"sandbox"` and numeric UIDs in `[1000, 2_000_000_000]` for the supervisor's process identity (the policy's `run_as_user` field). The driver resolves the UID at sandbox creation time and passes it to the supervisor via environment variables. +Compute drivers select the identity for agent child processes. Docker and Podman preserve the image identity by default. Kubernetes, OpenShift, and VM keep their existing platform-specific behavior. + +### Docker and Podman + +Docker and Podman use these driver fields: + +| Field | Accepted values | Default | Description | +|---|---|---|---| +| `identity_source` | `image`, `fixed` | `image` | Select the image-declared identity or an operator-controlled numeric identity. | +| `fixed_uid` | `1000` through `2000000000` | None | Set the agent UID in fixed mode. | +| `fixed_gid` | `1000` through `2000000000` | None | Set the agent primary GID in fixed mode. | + +Set both `fixed_uid` and `fixed_gid` when `identity_source = "fixed"`. Omit both fields in image mode. OpenShell rejects a partial pair, fixed fields in image mode, and values outside the accepted range. Docker and Podman do not use an implicit `10001:10001` identity. + +Use image mode for isolated local sandboxes. Use fixed mode when the operator attaches external or shared storage that expects a known UID and GID: + +```toml +[openshell.drivers.docker] +identity_source = "fixed" +fixed_uid = 2000 +fixed_gid = 2000 +``` + +The standalone Podman driver exposes the same settings as `--identity-source` with `OPENSHELL_PODMAN_IDENTITY_SOURCE`, `--fixed-uid` with `OPENSHELL_PODMAN_FIXED_UID`, and `--fixed-gid` with `OPENSHELL_PODMAN_FIXED_GID`. + +For newly created managed Docker and Podman sandboxes, the driver identity is authoritative. OpenShell rejects explicitly supplied process policy `run_as_user` or `run_as_group` values instead of claiming to apply an identity that the managed runtime ignores. + +#### Image User Forms + +In image mode, OpenShell accepts these OCI `USER` forms: + +| OCI `USER` | Resolution | +|---|---| +| `USER app` | Resolve `app` in the image's `/etc/passwd` and use its UID and primary GID. | +| `USER app:staff` | Resolve `app` in `/etc/passwd` and `staff` in `/etc/group`. | +| `USER app:1235` | Resolve the user name and use numeric GID `1235`. | +| `USER 1234:staff` | Use numeric UID `1234` and resolve the group name. | +| `USER 1234` | Use UID `1234` and the primary GID from the matching `/etc/passwd` entry. | +| `USER 1234:1235` | Use the numeric pair without requiring passwd or group entries. | + +A passwd-less image can therefore declare `USER 1234:1235` without running `useradd` or `groupadd`. A numeric UID without an explicit group still requires a unique matching passwd entry so OpenShell can determine its primary GID. + +OpenShell rejects a missing or empty `USER`, malformed declarations, unknown names, ambiguous or malformed matching account records, and any declaration that resolves to UID or GID 0. This includes `USER root`, `USER 0`, root group aliases, and non-root names mapped to UID or GID 0. The supervisor reads regular `/etc/passwd` and `/etc/group` files without following symlinks and does not use image-controlled NSS modules. + +#### Runtime and Storage Boundaries + +The container runtime starts the OpenShell supervisor as root, with Docker `user = "0"` or Podman `user = "0:0"`. That identity belongs only to the supervisor so it can prepare namespaces, mounts, filesystem policy, and process restrictions. Direct and SSH-launched agent children run with the resolved non-root UID and GID and no supplementary groups. + +Agent children always receive `HOME=/sandbox`. For a named OCI user, `USER` and `LOGNAME` contain the declared user name. Numeric and fixed identities use the numeric UID for those variables. OpenShell does not add, rewrite, or otherwise mutate `/etc/passwd` or `/etc/group`. + +Before changing ownership or dropping privileges, the supervisor verifies that the resolved UID and GID appear in its Linux user namespace maps. An unmapped ID fails rather than falling back to another identity. + +Image-selected identities can access the image rootfs and the driver-owned per-sandbox `/sandbox` workspace. Image mode rejects creator-selected bind and named-volume mounts because an untrusted image must not choose the principal used for unrelated storage. It still permits `tmpfs`, the driver-owned workspace and internal mounts, and read-only immutable image mounts where the driver supports them. Podman supports read-only `image` mounts; Docker does not expose image mounts through driver config. Select fixed mode for creator-selected external or shared storage. Bind mounts continue to require the driver's `enable_bind_mounts = true` operator setting. + +#### Image Pinning and Observability + +Docker and Podman apply the configured pull policy, inspect the final sandbox image, and capture its OCI `Config.User` and immutable image ID. They create the container from that immutable ID rather than the mutable tag that was inspected. Podman also pins the supervisor image and driver-config image mounts and disables another pull decision during container creation. + +The supervisor resolves the identity once, persists a versioned record, and reuses it on restart instead of re-resolving a mutable tag or changed account file. The gateway stores the resolved record in sandbox status and exposes `version`, `source`, `image_id`, `uid`, `gid`, and the empty `supplementary_gids` list through the API. The supervisor emits the same fields in an OCSF configuration-state event. Existing sandboxes without managed identity metadata retain their creation-time legacy behavior. Changes to identity mode or fixed IDs apply only to newly created sandboxes; recreate existing sandboxes or migrate their persisted workspaces explicitly. ### Kubernetes / OpenShift -The Kubernetes driver auto-detects the sandbox UID from OpenShift SCC namespace annotations: +Kubernetes and OpenShift identity selection is unchanged. The Kubernetes driver auto-detects the sandbox UID from OpenShift SCC namespace annotations: - `openshift.io/sa.scc.uid-range` (format: `/`, e.g. `1000000000/10000`) provides the UID. - `openshift.io/sa.scc.supplemental-groups` provides the GID when present; otherwise the resolved UID is used as the GID. @@ -432,8 +510,8 @@ The resolved UID/GID appear in: ### VM Driver -The VM driver injects the sandbox UID into the rootfs guest's `/etc/passwd`, `/etc/group`, and `/etc/gshadow` during rootfs preparation. Default UID is `10001`; configure `sandbox_uid` in `[openshell.drivers.vm]` to use a different value. +VM identity selection is unchanged. The VM driver injects the sandbox UID into the rootfs guest's `/etc/passwd`, `/etc/group`, and `/etc/gshadow` during rootfs preparation. Default UID is `10001`; configure `sandbox_uid` in `[openshell.drivers.vm]` to use a different value. ### Custom Images -Custom sandbox images no longer need a baked-in `"sandbox"` user. If your image requires a passwd entry for tools like `sudo` or `ssh`, add one manually (e.g. `RUN useradd -m -u 1500 deploy`). The supervisor resolves the numeric UID directly via `setuid()` without needing `/etc/passwd`. +Custom Docker and Podman sandbox images must declare a non-root OCI `USER` when the driver uses its default image identity mode. They do not need a user named `sandbox`. Use a numeric pair such as `USER 1234:1235` for an image without account database entries, or declare a named user when tools require a passwd entry. diff --git a/e2e/rust/Cargo.toml b/e2e/rust/Cargo.toml index b36a32203f..4d44ac55d1 100644 --- a/e2e/rust/Cargo.toml +++ b/e2e/rust/Cargo.toml @@ -35,7 +35,7 @@ e2e-vm = ["e2e", "e2e-host-gateway"] [[test]] name = "custom_image" path = "tests/custom_image.rs" -required-features = ["e2e-docker"] +required-features = ["e2e-local-container-driver"] [[test]] name = "docker_preflight" diff --git a/e2e/rust/tests/custom_image.rs b/e2e/rust/tests/custom_image.rs index fa905bbf19..8b3ec21161 100644 --- a/e2e/rust/tests/custom_image.rs +++ b/e2e/rust/tests/custom_image.rs @@ -1,69 +1,416 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -#![cfg(feature = "e2e")] +#![cfg(feature = "e2e-local-container-driver")] -//! E2E test: build a custom container image and run a sandbox with it. +//! OCI image identity E2E coverage for the Docker and Podman local drivers. //! -//! Prerequisites: -//! - A running Docker-backed openshell gateway (`mise run gateway:docker`) -//! - Docker daemon running (for image build) -//! - The `openshell` binary (built automatically from the workspace) +//! Docker fixtures intentionally go through `openshell sandbox create --from +//! Dockerfile`. Podman fixtures are built into the local Podman store first and +//! passed to `--from` by tag because the CLI's Dockerfile builder targets the +//! Docker daemon. -use std::io::Write; +use std::path::Path; +use std::process::Stdio; +use std::time::{SystemTime, UNIX_EPOCH}; -use openshell_e2e::harness::output::strip_ansi; +use openshell_e2e::harness::binary::openshell_cmd; +use openshell_e2e::harness::container::{ContainerEngine, e2e_driver}; +use openshell_e2e::harness::output::{extract_field, strip_ansi}; use openshell_e2e::harness::sandbox::SandboxGuard; +use serde_json::{Map, Value}; -const DOCKERFILE_CONTENT: &str = r#"FROM public.ecr.aws/docker/library/python:3.13-slim +const NUMERIC_DOCKERFILE: &str = r"FROM public.ecr.aws/docker/library/python:3.13-slim + +RUN apt-get update && apt-get install -y --no-install-recommends iproute2 \ + && rm -rf /var/lib/apt/lists/* \ + && ! getent passwd 1234 \ + && ! getent group 1235 +RUN printf numeric-image-marker > /etc/openshell-image-marker +USER 1234:1235 +"; + +const NAMED_DOCKERFILE: &str = r"FROM public.ecr.aws/docker/library/python:3.13-slim + +RUN apt-get update && apt-get install -y --no-install-recommends iproute2 \ + && rm -rf /var/lib/apt/lists/* \ + && groupadd --gid 2346 appgroup \ + && useradd --uid 2345 --gid 2346 --no-create-home --home-dir /sandbox app +RUN printf named-image-marker > /etc/openshell-image-marker +USER app +"; + +const MISSING_USER_DOCKERFILE: &str = r"FROM public.ecr.aws/docker/library/python:3.13-slim -# iproute2 is required for sandbox network namespace isolation. RUN apt-get update && apt-get install -y --no-install-recommends iproute2 \ && rm -rf /var/lib/apt/lists/* +"; -# Create the sandbox user/group so the supervisor can switch to it. -# Use a high UID range to avoid conflicts with host users when running without -# user namespace remapping (UID in container = UID on host). -RUN groupadd -g 1000660000 sandbox && \ - useradd -m -u 1000660000 -g sandbox sandbox +const ROOT_USER_DOCKERFILE: &str = r"FROM public.ecr.aws/docker/library/python:3.13-slim -# Write a marker file so we can verify this is our custom image. -# Place under /etc (Landlock baseline read-only path) so the sandbox -# can read it when filesystem restrictions are properly enforced. -RUN echo "custom-image-e2e-marker" > /etc/marker.txt +RUN apt-get update && apt-get install -y --no-install-recommends iproute2 \ + && rm -rf /var/lib/apt/lists/* +USER root +"; -CMD ["sleep", "infinity"] +const PROBE_LABEL: &str = "OPENSHELL_IDENTITY_PROBE"; +const PROBE_SCRIPT: &str = r#"set -eu +uid=$(id -u) +gid=$(id -g) +groups=$(id -G) +if grep -Eq "^[^:]*:[^:]*:${uid}:" /etc/passwd; then passwd=present; else passwd=absent; fi +if grep -Eq "^[^:]*:[^:]*:${gid}:" /etc/group; then group=present; else group=absent; fi +marker=$(cat /etc/openshell-image-marker) +printf sandbox-write-ok > /sandbox/identity-probe +write=$(cat /sandbox/identity-probe) +printf 'OPENSHELL_IDENTITY_PROBE uid=%s gid=%s groups=%s home=%s user=%s logname=%s passwd=%s group=%s marker=%s write=%s\n' \ + "$uid" "$gid" "$groups" "${HOME-}" "${USER-}" "${LOGNAME-}" \ + "$passwd" "$group" "$marker" "$write" "#; -const MARKER: &str = "custom-image-e2e-marker"; +struct Fixture { + _directory: tempfile::TempDir, + engine: ContainerEngine, + source: String, + image_tag: Option, + cli_build: bool, +} -/// Build a custom Docker image from a Dockerfile and verify that a sandbox -/// created from it contains the expected marker file. -#[tokio::test] -async fn sandbox_from_custom_dockerfile() { - // Step 1: Write a temporary Dockerfile. - let tmpdir = tempfile::tempdir().expect("create tmpdir"); - let dockerfile_path = tmpdir.path().join("Dockerfile"); - { - let mut f = std::fs::File::create(&dockerfile_path).expect("create Dockerfile"); - f.write_all(DOCKERFILE_CONTENT.as_bytes()) - .expect("write Dockerfile"); +impl Fixture { + fn create(name: &str, dockerfile: &str) -> Result { + let driver = local_driver(); + let engine = ContainerEngine::from_env()?; + let directory = tempfile::tempdir().map_err(|err| format!("create fixture dir: {err}"))?; + let dockerfile_path = directory.path().join("Dockerfile"); + std::fs::write(&dockerfile_path, dockerfile) + .map_err(|err| format!("write {}: {err}", dockerfile_path.display()))?; + + if driver == "docker" { + return Ok(Self { + _directory: directory, + engine, + source: path_string(&dockerfile_path)?, + image_tag: None, + cli_build: true, + }); + } + + let image_tag = unique_image_tag(name); + let output = engine + .command() + .arg("build") + .arg("--tag") + .arg(&image_tag) + .arg("--file") + .arg(&dockerfile_path) + .arg(directory.path()) + .output() + .map_err(|err| format!("run podman build for {name}: {err}"))?; + if !output.status.success() { + return Err(format!( + "podman build for {name} failed (exit {:?}):\n{}{}", + output.status.code(), + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + )); + } + + Ok(Self { + _directory: directory, + engine, + source: image_tag.clone(), + image_tag: Some(image_tag), + cli_build: false, + }) + } + + fn capture_cli_built_image(&mut self, output: &str) -> Result<(), String> { + if !self.cli_build || self.image_tag.is_some() { + return Ok(()); + } + self.image_tag = cli_built_image(output); + self.image_tag.as_ref().map_or_else( + || Err("could not identify CLI-built Docker fixture image for cleanup".to_string()), + |_| Ok(()), + ) } - // Step 2: Create a sandbox from the Dockerfile. - let dockerfile_str = dockerfile_path.to_str().expect("Dockerfile path is UTF-8"); - let mut guard = - SandboxGuard::create(&["--from", dockerfile_str, "--", "cat", "/etc/marker.txt"]) + fn cleanup(&mut self) -> Result<(), String> { + let Some(image_tag) = self.image_tag.take() else { + return Ok(()); + }; + let output = self + .engine + .command() + .args(["image", "rm", "--force", &image_tag]) + .output() + .map_err(|err| format!("remove fixture image {image_tag}: {err}"))?; + if output.status.success() { + return Ok(()); + } + Err(format!( + "remove fixture image {image_tag} failed (exit {:?}):\n{}{}", + output.status.code(), + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + )) + } +} + +impl Drop for Fixture { + fn drop(&mut self) { + let _ = self.cleanup(); + } +} + +#[tokio::test] +#[serial_test::serial] +async fn passwdless_numeric_user_preserves_declared_identity() { + require_image_identity_mode(); + let output = run_fixture( + "numeric", + NUMERIC_DOCKERFILE, + &[], + &["sh", "-lc", PROBE_SCRIPT], + ) + .await + .expect("run passwd-less numeric image"); + + assert_probe( + &output, + "OPENSHELL_IDENTITY_PROBE uid=1234 gid=1235 groups=1235 home=/sandbox user=1234 logname=1234 passwd=absent group=absent marker=numeric-image-marker write=sandbox-write-ok", + ); +} + +#[tokio::test] +#[serial_test::serial] +async fn named_user_uses_passwd_primary_group() { + require_image_identity_mode(); + let output = run_fixture("named", NAMED_DOCKERFILE, &[], &["sh", "-lc", PROBE_SCRIPT]) + .await + .expect("run named-user image"); + + assert_probe( + &output, + "OPENSHELL_IDENTITY_PROBE uid=2345 gid=2346 groups=2346 home=/sandbox user=app logname=app passwd=present group=present marker=named-image-marker write=sandbox-write-ok", + ); +} + +#[tokio::test] +#[serial_test::serial] +async fn missing_and_root_user_declarations_fail_provisioning() { + require_image_identity_mode(); + for (name, dockerfile, expected) in [ + ( + "missing-user", + MISSING_USER_DOCKERFILE, + "must declare a non-empty OCI Config.User", + ), + ( + "root-user", + ROOT_USER_DOCKERFILE, + // Local-driver watcher conditions intentionally do not include + // supervisor stderr; the stable E2E contract is terminal failure. + "sandbox entered error phase while provisioning", + ), + ] { + let error = run_fixture(name, dockerfile, &[], &["true"]) .await - .expect("sandbox create from Dockerfile"); + .expect_err("invalid image identity should fail sandbox provisioning"); + let normalized_error = normalize_whitespace(&strip_ansi(&error)); + assert!( + normalized_error.contains(expected), + "expected {name} failure to contain '{expected}':\n{error}" + ); + } +} + +#[tokio::test] +#[serial_test::serial] +async fn image_identity_rejects_external_bind_and_named_volume_mounts() { + require_image_identity_mode(); + let bind_directory = tempfile::tempdir().expect("create rejected bind source"); + let bind_source = path_string(bind_directory.path()).expect("bind source path is UTF-8"); + let driver = local_driver(); + let mounts = [ + ( + "bind", + serde_json::json!({ + "type": "bind", + "source": bind_source, + "target": "/sandbox/external-bind", + "read_only": false + }), + "bind mounts are not allowed when identity_source = 'image'", + ), + ( + "volume", + serde_json::json!({ + "type": "volume", + "source": "openshell-e2e-rejected-volume", + "target": "/sandbox/external-volume", + "read_only": false + }), + "named volume mounts are not allowed when identity_source = 'image'", + ), + ]; + + for (name, mount, expected) in mounts { + let driver_config = driver_config_mount_json(&driver, &mount); + let extra_args = ["--driver-config-json".to_string(), driver_config]; + let error = run_fixture( + &format!("rejected-{name}"), + NUMERIC_DOCKERFILE, + &extra_args, + &["true"], + ) + .await + .expect_err("image identity must reject creator-selected external storage"); + let normalized_error = normalize_whitespace(&strip_ansi(&error)); + assert!( + normalized_error.contains(expected), + "expected {name} rejection to contain '{expected}':\n{error}" + ); + } +} - // Step 3: Verify the marker file content appears in the output. - let clean_output = strip_ansi(&guard.create_output); +fn normalize_whitespace(value: &str) -> String { + value + .split_whitespace() + .filter(|token| *token != "│") + .collect::>() + .join(" ") +} + +async fn run_fixture( + name: &str, + dockerfile: &str, + extra_args: &[String], + command: &[&str], +) -> Result { + let mut fixture = Fixture::create(name, dockerfile)?; + let mut args = vec![ + "--no-keep".to_string(), + "--from".to_string(), + fixture.source.clone(), + ]; + args.extend_from_slice(extra_args); + args.push("--".to_string()); + args.extend(command.iter().map(|arg| (*arg).to_string())); + let arg_refs = args.iter().map(String::as_str).collect::>(); + + let result = SandboxGuard::create(&arg_refs).await; + match result { + Ok(mut sandbox) => { + let captured_image = fixture.capture_cli_built_image(&sandbox.create_output); + let output = sandbox.create_output.clone(); + sandbox.cleanup().await; + captured_image?; + fixture.cleanup()?; + Ok(output) + } + Err(error) => { + let captured_image = fixture.capture_cli_built_image(&error); + cleanup_failed_sandbox(&error).await?; + captured_image?; + fixture.cleanup()?; + Err(error) + } + } +} + +async fn cleanup_failed_sandbox(output: &str) -> Result<(), String> { + let name = extract_field(output, "Created sandbox").or_else(|| extract_field(output, "Name")); + let Some(name) = name else { + return Ok(()); + }; + + let delete = openshell_cmd() + .arg("sandbox") + .arg("delete") + .arg(&name) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .output() + .await + .map_err(|err| format!("delete failed fixture sandbox {name}: {err}"))?; + if delete.status.success() { + return Ok(()); + } + Err(format!( + "delete failed fixture sandbox {name} (exit {:?}):\n{}{}", + delete.status.code(), + String::from_utf8_lossy(&delete.stdout), + String::from_utf8_lossy(&delete.stderr) + )) +} + +fn assert_probe(output: &str, expected: &str) { + let clean = strip_ansi(output); + let record = clean + .lines() + .map(str::trim) + .find(|line| line.starts_with(PROBE_LABEL)) + .unwrap_or_else(|| panic!("identity probe record missing from output:\n{clean}")); + assert_eq!(record, expected, "unexpected identity probe record"); +} + +fn require_image_identity_mode() { + assert_eq!( + std::env::var("OPENSHELL_E2E_IDENTITY_SOURCE").as_deref(), + Ok("image"), + "custom image identity tests require an explicitly image-mode gateway" + ); +} + +fn local_driver() -> String { + let driver = e2e_driver().expect("OPENSHELL_E2E_DRIVER must be set by the e2e wrapper"); assert!( - clean_output.contains(MARKER), - "expected marker '{MARKER}' in sandbox output:\n{clean_output}" + matches!(driver.as_str(), "docker" | "podman"), + "custom image e2e requires docker or podman, got {driver}" ); + driver +} + +fn driver_config_mount_json(driver: &str, mount: &Value) -> String { + let mut root = Map::new(); + root.insert( + driver.to_string(), + serde_json::json!({ + "mounts": [mount] + }), + ); + Value::Object(root).to_string() +} + +fn unique_image_tag(name: &str) -> String { + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("system clock should be after Unix epoch") + .as_nanos(); + format!( + "openshell/e2e-image-identity-{name}:{}-{nanos}", + std::process::id() + ) +} + +fn cli_built_image(output: &str) -> Option { + const PREFIX: &str = "openshell/sandbox-from:"; + strip_ansi(output).split_whitespace().find_map(|token| { + let start = token.find(PREFIX)?; + let candidate = &token[start..]; + let end = candidate + .find(|character: char| { + !character.is_ascii_alphanumeric() + && !matches!(character, '/' | ':' | '.' | '_' | '-') + }) + .unwrap_or(candidate.len()); + Some(candidate[..end].to_string()) + }) +} - // Explicit cleanup (also happens in Drop, but explicit is clearer in tests). - guard.cleanup().await; +fn path_string(path: &Path) -> Result { + path.to_str() + .map(ToString::to_string) + .ok_or_else(|| format!("path is not UTF-8: {}", path.display())) } diff --git a/e2e/rust/tests/driver_config_volume.rs b/e2e/rust/tests/driver_config_volume.rs index ad8cffc2f9..47d55743a0 100644 --- a/e2e/rust/tests/driver_config_volume.rs +++ b/e2e/rust/tests/driver_config_volume.rs @@ -61,6 +61,10 @@ impl Drop for VolumeGuard { #[tokio::test] async fn sandbox_mounts_existing_driver_config_volume() { + if !fixed_identity_mode_enabled() { + return; + } + let driver = e2e_driver().expect("OPENSHELL_E2E_DRIVER must be set by the e2e wrapper"); assert!( matches!(driver.as_str(), "docker" | "podman"), @@ -103,6 +107,10 @@ async fn sandbox_mounts_existing_driver_config_volume() { #[tokio::test] async fn sandbox_mounts_enabled_driver_config_bind() { + if !fixed_identity_mode_enabled() { + return; + } + let driver = e2e_driver().expect("OPENSHELL_E2E_DRIVER must be set by the e2e wrapper"); assert!( matches!(driver.as_str(), "docker" | "podman"), @@ -171,16 +179,37 @@ filesystem_policy: landlock: compatibility: best_effort - -process: - run_as_user: sandbox - run_as_group: sandbox ", ) .map_err(|err| format!("write bind policy: {err}"))?; Ok(file) } +fn fixed_identity_mode_enabled() -> bool { + match std::env::var("OPENSHELL_E2E_IDENTITY_SOURCE").as_deref() { + Ok("fixed") => { + assert!( + std::env::var_os("OPENSHELL_E2E_FIXED_UID").is_some() + && std::env::var_os("OPENSHELL_E2E_FIXED_GID").is_some(), + "fixed-mode gateway tests require OPENSHELL_E2E_FIXED_UID and OPENSHELL_E2E_FIXED_GID" + ); + true + } + Ok("image") => { + eprintln!( + "skipping positive external-mount test: run against an explicitly fixed-mode gateway with OPENSHELL_E2E_IDENTITY_SOURCE=fixed and OPENSHELL_E2E_FIXED_UID/GID" + ); + false + } + Ok(other) => panic!( + "OPENSHELL_E2E_IDENTITY_SOURCE must describe the gateway as image or fixed, got {other}" + ), + Err(error) => panic!( + "gateway wrapper must export OPENSHELL_E2E_IDENTITY_SOURCE so external mounts cannot run against an unknown identity mode: {error}" + ), + } +} + async fn seed_volume(volume: &VolumeGuard) -> Result<(), String> { run_volume_container( volume, diff --git a/e2e/support/gateway-common.sh b/e2e/support/gateway-common.sh index 6b9e6a0956..f803d4a6a2 100644 --- a/e2e/support/gateway-common.sh +++ b/e2e/support/gateway-common.sh @@ -34,6 +34,45 @@ e2e_pick_port() { python3 -c 'import socket; s=socket.socket(); s.bind(("",0)); print(s.getsockname()[1]); s.close()' } +e2e_resolve_local_identity_config() { + local source + source="$(printf '%s' "${OPENSHELL_E2E_IDENTITY_SOURCE:-image}" | tr '[:upper:]' '[:lower:]')" + + case "${source}" in + image) + if [ -n "${OPENSHELL_E2E_FIXED_UID:-}" ] || [ -n "${OPENSHELL_E2E_FIXED_GID:-}" ]; then + echo "ERROR: OPENSHELL_E2E_FIXED_UID/GID must be unset in image identity mode." >&2 + exit 2 + fi + ;; + fixed) + for field in OPENSHELL_E2E_FIXED_UID OPENSHELL_E2E_FIXED_GID; do + local value="${!field:-}" + if ! [[ "${value}" =~ ^[0-9]+$ ]] \ + || [ "${value}" -lt 1000 ] \ + || [ "${value}" -gt 2000000000 ]; then + echo "ERROR: ${field} must be an integer in [1000, 2000000000] for fixed identity mode." >&2 + exit 2 + fi + done + ;; + *) + echo "ERROR: OPENSHELL_E2E_IDENTITY_SOURCE must be image or fixed, got '${source}'." >&2 + exit 2 + ;; + esac + + export OPENSHELL_E2E_IDENTITY_SOURCE="${source}" +} + +e2e_write_local_identity_config() { + printf 'identity_source = %s\n' "$(e2e_toml_string "${OPENSHELL_E2E_IDENTITY_SOURCE}")" + if [ "${OPENSHELL_E2E_IDENTITY_SOURCE}" = "fixed" ]; then + printf 'fixed_uid = %s\n' "${OPENSHELL_E2E_FIXED_UID}" + printf 'fixed_gid = %s\n' "${OPENSHELL_E2E_FIXED_GID}" + fi +} + e2e_generate_pki() { local gateway_bin=$1 local pki_dir=$2 diff --git a/e2e/with-docker-gateway.sh b/e2e/with-docker-gateway.sh index 64062b74d6..c860eaf5e6 100755 --- a/e2e/with-docker-gateway.sh +++ b/e2e/with-docker-gateway.sh @@ -17,6 +17,8 @@ # Sandbox image overrides: # OPENSHELL_E2E_DOCKER_SANDBOX_IMAGE=... # OPENSHELL_E2E_DOCKER_SANDBOX_IMAGE_PULL_POLICY=Always|IfNotPresent|Never +# OPENSHELL_E2E_IDENTITY_SOURCE=image|fixed (defaults to image) +# OPENSHELL_E2E_FIXED_UID/GID (required only for fixed identity mode) # # The default community sandbox image uses :latest. This wrapper refreshes it # before starting the gateway, while the Docker driver defaults to IfNotPresent @@ -34,6 +36,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" source "${ROOT}/e2e/support/gateway-common.sh" e2e_preserve_mise_dirs +e2e_resolve_local_identity_config require_container_engine_lane() { local lane=$1 @@ -490,6 +493,7 @@ GATEWAY_CONFIG="${STATE_DIR}/gateway.toml" printf 'guest_tls_cert = %s\n' "$(toml_string "${PKI_DIR}/client/tls.crt")" printf 'guest_tls_key = %s\n' "$(toml_string "${PKI_DIR}/client/tls.key")" printf 'enable_bind_mounts = true\n' + e2e_write_local_identity_config printf 'supervisor_image = %s\n' "$(toml_string "${SUPERVISOR_IMAGE}")" if [ -n "${GATEWAY_HOST_ALIAS_IP}" ]; then printf 'host_gateway_ip = %s\n' "$(toml_string "${GATEWAY_HOST_ALIAS_IP}")" diff --git a/e2e/with-podman-gateway.sh b/e2e/with-podman-gateway.sh index ba9179a841..bdd3ff8e5e 100755 --- a/e2e/with-podman-gateway.sh +++ b/e2e/with-podman-gateway.sh @@ -13,6 +13,8 @@ # # HTTPS endpoint-only mode is intentionally unsupported here. Use a named # gateway config when mTLS materials are needed. +# Set OPENSHELL_E2E_IDENTITY_SOURCE=fixed and OPENSHELL_E2E_FIXED_UID/GID to +# exercise operator-controlled external storage; image mode is the default. set -euo pipefail @@ -24,6 +26,7 @@ fi ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # shellcheck source=e2e/support/gateway-common.sh source "${ROOT}/e2e/support/gateway-common.sh" +e2e_resolve_local_identity_config require_container_engine_lane() { local lane=$1 @@ -421,6 +424,7 @@ cp "${ROOT}/deploy/rpm/gateway.toml.default" "${GATEWAY_CONFIG}" printf 'guest_tls_cert = %s\n' "$(toml_string "${PKI_DIR}/client/tls.crt")" printf 'guest_tls_key = %s\n' "$(toml_string "${PKI_DIR}/client/tls.key")" printf 'enable_bind_mounts = true\n' + e2e_write_local_identity_config # The in-process Podman driver reads `socket_path` from TOML only — the # OPENSHELL_PODMAN_SOCKET env var is honoured by the standalone driver # binary, not the in-process driver used here. Pin the socket to the one diff --git a/examples/bring-your-own-container/Dockerfile b/examples/bring-your-own-container/Dockerfile index fc65bd6956..132e80a1f2 100644 --- a/examples/bring-your-own-container/Dockerfile +++ b/examples/bring-your-own-container/Dockerfile @@ -14,16 +14,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ curl iproute2 nftables \ && rm -rf /var/lib/apt/lists/* -# The sandbox user is injected at runtime by the compute driver. -# Kubernetes: resolved from OpenShift SCC namespace annotations or explicit -# sandbox_uid config. VM: resolves to 10001 by default, configurable in -# gateway TOML. -# -# Images no longer need a baked-in "sandbox" user — numeric UIDs are accepted -# and the driver passes them directly to setuid()/chown() at sandbox start. -# If your image requires a passwd entry for tools like ssh or sudo, add one -# manually (e.g. RUN useradd -m -u 1500 deploy). - RUN install -d /sandbox WORKDIR /sandbox COPY app.py . @@ -33,3 +23,7 @@ EXPOSE 8080 # NOTE: The sandbox supervisor replaces CMD at runtime. Pass the start # command explicitly: openshell sandbox create ... -- python /sandbox/app.py CMD ["python", "app.py"] + +# Local Docker/Podman sandboxes preserve the image's non-root OCI USER. This +# numeric pair intentionally has no matching passwd or group entries. +USER 1234:1235 diff --git a/examples/bring-your-own-container/README.md b/examples/bring-your-own-container/README.md index ea4f1cb9e6..6ec51c1085 100644 --- a/examples/bring-your-own-container/README.md +++ b/examples/bring-your-own-container/README.md @@ -13,7 +13,7 @@ your local machine through port forwarding. | File | Description | | ------------ | ------------------------------------------------------- | -| `Dockerfile` | Builds a Python 3.12 image that starts a REST API | +| `Dockerfile` | Builds a Python 3.13 image that starts a REST API | | `app.py` | Minimal HTTP server with `/hello` and `/health` routes | ## Quick start @@ -59,17 +59,15 @@ key requirements are: - **Pass your start command explicitly** — use `-- ` on the CLI. The image's `CMD` / `ENTRYPOINT` is replaced by the sandbox supervisor at runtime. -- **Create a `sandbox` user** (uid/gid 1000660000) for non-root execution. - Use a high UID (1000000000+) to avoid conflicts with host users when running - without user namespace remapping. -- **Make your application workdir writable by `sandbox`**. This example creates - `/sandbox` with `sandbox:sandbox` ownership before copying `app.py`. +- **Declare a non-root OCI `USER`**. Local Docker and Podman gateways preserve + that identity for agent processes. This example uses the passwd-less numeric + pair `USER 1234:1235`; numeric pairs do not require account database entries. +- **Use `/sandbox` as the application workdir**. OpenShell owns the isolated + workspace root for the declared image identity when the sandbox starts. - **Install `iproute2`** for full network namespace isolation. - **Use a standard Linux base image** — distroless and `FROM scratch` images are not supported. -TODO(#70): Remove the sandbox user note once custom images are secure by default without requiring manual setup. - ## How it works OpenShell handles all the wiring automatically. You build a standard diff --git a/examples/private-ip-routing/Dockerfile b/examples/private-ip-routing/Dockerfile index 614b3b4738..d6e2a50b18 100644 --- a/examples/private-ip-routing/Dockerfile +++ b/examples/private-ip-routing/Dockerfile @@ -3,5 +3,7 @@ FROM python:3.13-slim COPY server.py /app/server.py +RUN chmod 0555 /app && chmod 0444 /app/server.py EXPOSE 8080 CMD ["python", "/app/server.py"] +USER 1234:1235 diff --git a/proto/openshell.proto b/proto/openshell.proto index 1b447a17e3..c0e7db9a91 100644 --- a/proto/openshell.proto +++ b/proto/openshell.proto @@ -458,6 +458,10 @@ message SandboxStatus { SandboxPhase phase = 6; // Currently active policy version (updated when sandbox reports loaded). uint32 current_policy_version = 7; + // Persisted creation-time identity selected for agent children, when managed. + openshell.sandbox.v1.ResolvedAgentIdentity resolved_identity = 8; + // Whether this sandbox requires gateway-authorized managed runtime identity. + bool managed_identity_required = 9; } // User-facing sandbox condition derived from driver-native conditions. @@ -1603,6 +1607,8 @@ message SupervisorHello { string sandbox_id = 1; // Supervisor instance ID (e.g. boot id or process epoch). string instance_id = 2; + // Creation-time identity resolved by a managed sandbox runtime. + openshell.sandbox.v1.ResolvedAgentIdentity resolved_identity = 3; } // Gateway accepts the supervisor session. diff --git a/proto/sandbox.proto b/proto/sandbox.proto index 16b3ca998d..c0f51bd2ca 100644 --- a/proto/sandbox.proto +++ b/proto/sandbox.proto @@ -57,6 +57,22 @@ message ProcessPolicy { string run_as_group = 2; } +// Creation-time numeric identity selected by the managed sandbox runtime. +message ResolvedAgentIdentity { + // Identity record schema version. + uint32 version = 1; + // Selection source: "image" or "fixed". + string source = 2; + // Immutable image identifier used to bind image and fixed identities to the rootfs. + string image_id = 3; + // Effective user ID for agent children. + uint32 uid = 4; + // Effective primary group ID for agent children. + uint32 gid = 5; + // Effective supplementary groups. Managed local identities require this to be empty. + repeated uint32 supplementary_gids = 6; +} + // A named network access policy rule. message NetworkPolicyRule { // Human-readable name for this policy rule. @@ -367,6 +383,10 @@ message GetSandboxConfigResponse { // Workspace the sandbox belongs to. Allows the supervisor to learn its // workspace context for subsequent workspace-scoped RPCs. string workspace = 10; + // Persisted creation-time identity selected for agent children, when managed. + ResolvedAgentIdentity resolved_identity = 11; + // Whether this sandbox requires gateway-authorized managed runtime identity. + bool managed_identity_required = 12; } // Connection details for one operator-registered supervisor middleware service.