feat: provider-aware body sanitization for inference proxy

[zanetworker]

Problem Statement

The inference proxy forwards Anthropic Messages API request bodies to non-Anthropic backends (Vertex AI rawPredict, AWS Bedrock) without systematically sanitizing provider-incompatible fields. This has caused production breakage: Claude Code sends context_management (an Anthropic beta parameter) on every request, and Vertex AI rejects it with HTTP 400 "Extra inputs are not permitted" (see #2444).

The current fix (#2444) adds context_management to a static denylist. This works but is reactive — every new Anthropic beta field is a potential silent breakage until someone discovers it and patches the list. The same class of problem applies to Bedrock and any future backend that uses strict input validation.

Proposed Design

Replace the per-field denylist with a provider-aware body sanitization layer in prepare_backend_request():

Option A — Allowlist per backend contract. Define the set of fields each backend accepts. For Vertex rawPredict, this is the Anthropic Messages API minus model (in URL path) minus beta fields. Strip everything not on the list. Downside: breaks when Anthropic adds a new standard field until the allowlist is updated.

Option B — Parse anthropic-beta header to derive strip list. The client already sends an anthropic-beta header declaring which beta features are active (e.g., context-management-2025-06-27). Maintain a mapping from beta feature name to its body field(s). For Vertex/Bedrock routes, strip body fields whose corresponding beta is declared. No beta header = only strip model. Upside: self-describing, the client tells you what's non-standard. Downside: mapping is still manual, and some beta fields may lack a header entry.

Option C — Hybrid. Use the beta-header approach (Option B) as the primary mechanism, with a small static denylist as a safety net for beta fields that don't advertise themselves via the header.

The design should also consider:

• Bedrock parity: Bedrock currently skips body mutation entirely (intentional for bridge-fronted setups), but direct Bedrock InvokeModel has the same strict validation issue.
• Logging: When a field is stripped, emit a debug-level log so operators can diagnose unexpected behavior.
• Testing: Each backend contract should have a test that sends a request with known beta fields and asserts they're stripped (Vertex/Bedrock) or preserved (direct Anthropic).

Alternatives Considered

• Keep the static denylist (current state after Inference proxy: support or strip unsupported beta parameters when forwarding to Vertex rawPredict #2444 fix): Simplest, but reactive. Every new Anthropic beta field is a potential production incident.
• Proxy the request through the Anthropic SDK's Vertex transport: Would get field stripping for free, but adds a Python dependency to a Rust proxy and defeats the purpose of the lightweight proxy.
• Reject unknown fields at the proxy layer with a clear error: Better DX than a cryptic Vertex 400, but still breaks the session.

Agent Investigation

The body transformation logic lives in crates/openshell-router/src/backend.rs, prepare_backend_request() (around line 274). Key observations:

• Header sanitization already exists: sanitize_request_headers() strips anthropic-beta for Vertex routes (lines 129-149). The body side lacks equivalent treatment.
• Route detection is structural: is_vertex_anthropic_rawpredict_route() checks model_in_path + anthropic_messages protocol + :rawPredict suffix. Any future provider with the same shape inherits transforms automatically.
• Bedrock has a separate code path: route_is_bedrock() deliberately skips body mutation, but this may need revisiting for direct Bedrock (non-bridge) support.
• Test coverage is strong: 66 unit tests + 17 integration tests cover the router. Adding a new sanitization layer has good test infrastructure to build on.

[lbelyaev]

Having run into this exact break through inference.local, one note on the options from the enforcement side.

The design question underneath the three options is which way each one fails when it meets a field it doesn't recognize. The current denylist and Option B both fail open — an unrecognized beta field passes through to the backend and you get the Vertex 400 you're trying to prevent. Option A (allowlist) fails closed — an unknown field is stripped, so a new standard Anthropic field breaks until the allowlist catches up, but you never leak an unexpected field upstream. For a proxy whose job is to keep inference calls well-formed against a strict backend, fail-closed is usually the safer default even though it's the higher-maintenance one — the failure is a stripped field you can see in a debug log, not a 400 mid-session. The hybrid (C) is the pragmatic middle, but it's worth naming in the design which direction the safety net leans, because "beta-header primary + static denylist backstop" still fails open on anything neither path knows about.