Inference proxy: support or strip unsupported beta parameters when forwarding to Vertex rawPredict

[zanetworker]

Summary

The OpenShell inference proxy (inference.local) forwards the full Anthropic Messages API request body to Vertex AI's rawPredict endpoint without stripping beta parameters that Vertex doesn't support. This causes all Claude Code sessions through inference.local to fail with a 400 error.

Reproduction

Environment: OpenShell v0.0.86, local podman gateway with a Vertex provider.

Steps:

1. Configure inference with a Vertex provider:

   openshell inference set --provider vertex --model claude-sonnet-4-6
   

2. Create a sandbox with the Vertex provider and run Claude Code:

   openshell sandbox create --provider vertex \
     --env ANTHROPIC_API_KEY=placeholder \
     --env ANTHROPIC_BASE_URL=https://inference.local
   

3. Inside the sandbox, run:

   echo "say pong" | claude -p
   

Result: API Error: 400 context_management: Extra inputs are not permitted

Expected: Claude Code responds normally (the proxy strips unsupported fields before forwarding to Vertex).

Root cause

Claude Code (v2.1.156+) sends context_management on every request. This is a beta parameter for the Anthropic Messages API (context-management-2025-06-27 beta) used for context editing (clearing stale tool results and thinking blocks).

The direct Anthropic API accepts it. Vertex rawPredict does not, and rejects it with a pydantic validation error. The Anthropic SDKs' Vertex transport (AnthropicVertex) strips unsupported parameters automatically before sending to Vertex. The OpenShell inference proxy does not.

Verified with manual requests from inside a sandbox

Request body	Result
{"model":"claude-sonnet-4-6","max_tokens":50,"messages":[...]}	200 OK, "Pong"
Same + "context_management":{"enabled":true}	400 "context_management: Extra inputs are not permitted"

Impact

No Claude Code session works through inference.local with a Vertex provider. Both Claude Code v2.1.156 and v2.1.185 are affected (all current versions send context_management).

Suggested fix

When the inference proxy forwards a request to a Vertex rawPredict endpoint, it should either support Anthropic API beta parameters like context_management (by processing them before forwarding) or strip them from the request body. At minimum: handle context_management. Ideally, maintain an allowlist of fields Vertex rawPredict accepts so that new Anthropic beta parameters don't break the proxy silently.

[matthewgrossman]

📋 triage-agent

Triage Assessment

Classification: bug-confirmed

Summary

Vertex rawPredict forwarding still preserves unsupported beta fields such as context_management on current main and v0.0.90.

Investigation

The rawPredict rewrite removes model and injects anthropic_version, but it does not remove context_management in crates/openshell-router/src/backend.rs. The reported failure mode matches the active code path.

Recommendation

Strip only explicitly known Vertex-incompatible fields and add focused forwarding tests. Avoid a broad allowlist that could silently erase future request semantics.