feat(proxy): expand model catalog, per-model reasoning effort, opencode variants + production hardening#2
Merged
Merged
Conversation
…on hardening
Found + verified against the live CC /alpha/generate stream (captured real
events + probed the validation schema). The biggest issue — tool_choice — was
invisible from code review alone.
Functional fixes
- tool_choice: was sent as a bare string, which CC rejects with 400
("expected object, received string at params.tool_choice"). Map both OpenAI
and Anthropic tool_choice to CC's object union {type:"auto"|"any"|"tool",
name?}. CC has no "required" (use "any") and no "none" (omit). Verified all
three shapes return 200 against live CC.
- streaming Anthropic input_tokens: CC's `start` event carries no usage, so
message_start reported input_tokens=0. Emit input_tokens in message_delta
from finish.totalUsage (the Anthropic SDK merges it over message_start).
Robustness / ops
- Retry retryable upstream failures (5xx/429/timeout/network) up to 2x with
backoff, before the stream starts only; client aborts are never retried.
(isRetryable was previously computed but unused.)
- Request body size limit (10 MiB → 413) to prevent OOM.
- Guard all response writes against destroyed/ended sockets (client disconnect
mid-request no longer throws).
- Top-level unhandledRejection/uncaughtException handlers + bounded graceful
shutdown (force-exit after 10s).
Config / packaging
- Docker: mount ~/.config/commandcode-api-proxy (was ~/.commandcode, so
auth.json never loaded); bind host port to 127.0.0.1 only.
- CORS_ORIGIN env (default *; empty disables); document the localhost-only
security posture.
- Drop key-prefix logging (length only).
- Remove dead NDJSONParser class (runtime parses via parseCCLine).
Tests: 127 pass; added tool_choice (object form) + streaming input_tokens
coverage.
Co-Authored-By: Claude <noreply@anthropic.com>
Researched the model catalog in the command-code CLI (the authoritative
source) and the /alpha/generate plan gating by probing each candidate live.
Catalog (src/models.json)
- Add 12 models reachable on the standard plan (verified 200, not 403):
GLM-5.2, GLM-5.2-Fast, MiniMax-M3, Kimi-K2.7-Code (+Highspeed), Kimi-K3,
mimo-v2.5-pro, Step-3.7-Flash, grok-4.5, nemotron-3-ultra, inkling, Hy3.
- Exclude plan-gated models (gemini-3.5-flash, gemini-3.1-flash-lite,
fugu-ultra, muse-spark) — they 403 MODEL_NOT_IN_PLAN on standard.
- Add reasoningEfforts map; fix stale contextWindows (Step-3.5-Flash 1M,
Kimi K2.5/K2.6 256k); add aliases for all new models.
Reasoning effort (the capability gap)
- Each CC model accepts a different effort subset (e.g. deepseek-v4-pro is
{high,max}; grok-4.5 is {low,medium,high}). CC silently coerces unsupported
values, so the proxy previously could send (or drop) the wrong level and
could never reach "max"/"xhigh".
- models.ts: resolveEffortForModel() clips a request to the nearest valid
level for the model; uncatalogued models pass through unchanged.
- OpenAI: widen reasoning_effort to low/medium/high/xhigh/max.
- Anthropic: extend thinking.budget_tokens mapping through xhigh/max, then
clip per model. Verified live: deepseek-v4-pro+low→high, +max→max;
grok-4.5+xhigh→high.
Tests: 130 pass; added effort-clipping coverage (per-model clip, xhigh/max,
grok cap) for both protocols. README aliases + reasoning-effort notes updated.
Co-Authored-By: Claude <noreply@anthropic.com>
OpenCode's provider config (written by --setup-opencode) keys each model by the bare last path segment with original casing (e.g. "GLM-5.2", "MiniMax-M3", "Kimi-K3") and sends that key as the model id. The proxy's resolveModel only matched lowercase aliases or org-prefixed ids, so mixed-case bare names passed through unchanged and CC rejected them (403 — needs org/model). Only deepseek/grok worked, by lowercase-alias coincidence. resolveModel now: (a) matches aliases case-insensitively, and (b) falls back to the builtin catalog's last path segment. Verified live: GLM-5.2, MiniMax-M3, Kimi-K3, Kimi-K2.7-Code-Highspeed, nemotron-3-ultra-550b-a55b all resolve to canonical ids and return 200. Tests: 131 pass; added bare-name resolution coverage. Co-Authored-By: Claude <noreply@anthropic.com>
OpenCode's `/variants` picker lists a model's `variants` map. OpenCode only
auto-generates reasoning variants from models.dev metadata, which does not
cover our custom commandcode model IDs — so the picker was empty.
The opencode setup now declares `variants` explicitly for effort-capable
models, driven by models.json's reasoningEfforts map:
deepseek-v4-pro/flash, GLM-5.2 → {high, max}
grok-4.5 → {low, medium, high}
Each variant maps to `{ reasoningEffort: <level> }`, which OpenCode forwards
to the proxy (and the proxy clips per model — see resolveEffortForModel).
Source-of-truth for the mechanism: sst/opencode —
provider/provider.ts (config variants are mergeDeep'd in) and
provider/transform.ts ({reasoningEffort} variant shape for openai-compatible).
Tests: 133 pass; added setup-opencode coverage (variants emitted for effort
models, omitted for others).
Co-Authored-By: Claude <noreply@anthropic.com>
…the model
OpenCode auto-adds WIDELY_SUPPORTED_EFFORTS (low/medium/high) to every
reasoning-capable openai-compatible model and mergeDeep's them with our config,
so deepseek-v4-pro (which CC only supports at {high, max}) showed 4 variants
including bogus low/medium (which the proxy would silently clip to high).
Declare the full effort universe per model and mark the unsupported tiers
`disabled: true`. OpenCode filters disabled variants out of the picker
(pickBy !disabled), so each model shows exactly what it supports:
deepseek-v4-pro/flash, GLM-5.2 → high, max
grok-4.5 → low, medium, high
Tests updated to assert the disabled tiers. 133 pass.
Co-Authored-By: Claude <noreply@anthropic.com>
An empty reasoningEfforts array ([] is truthy) bypassed the early return and reached reduce() on an empty array, which throws. No current model triggers it, but adding `reasoningEfforts: []` to models.json would have crashed the request. Added a length guard. Co-Authored-By: Claude <noreply@anthropic.com>
thaolaptrinh
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Verified end-to-end against the live CC
/alpha/generatestream (captured events, probed the validation schema) and thecommand-codeCLI bundle (authoritative model catalog). The biggest issue —tool_choicerejected by CC — was invisible from code review alone.Functional fixes (bugs found via live capture)
tool_choiceobject form — the proxy sent a bare string, which CC rejects with400 "expected object, received string at params.tool_choice". Any request withtool_choice(other than OpenAI"auto") broke. Now maps both OpenAI and Anthropic to CC's union{type:"auto"|"any"|"tool", name?}. Verified live (all shapes → 200). CC has no"required"(use"any") and no"none".input_tokens— CC'sstartevent carries no usage, somessage_startreportedinput_tokens: 0. Now emitsinput_tokens(andcache_read_input_tokens) inmessage_delta; the Anthropic SDK merges it overmessage_start. Verified against@anthropic-ai/sdksource (MessageStream).Production hardening
isRetryableis now actually used.)unhandledRejection/uncaughtExceptionbackstop + bounded graceful shutdown (10s force-exit).~/.config/commandcode-api-proxypath (was~/.commandcode, soauth.jsonnever loaded); bind host port to127.0.0.1only.CORS_ORIGINenv (default*, empty disables). Dropped key-prefix logging.Model catalog + per-model reasoning effort
reasoningEffortsmap (4 models, verified against the CC CLI catalog): deepseek-v4-pro/flash & GLM-5.2 →{high,max}; grok-4.5 →{low,medium,high}.resolveEffortForModel()clips an unsupported effort to the nearest valid level per model, andreasoning_effortnow acceptsxhigh/max(so deepseek'smaxis reachable). Verified live.GLM-5.2,Kimi-K3) — as OpenCode sends them — now resolve to canonical IDs (case-insensitive alias + last-segment fallback).OpenCode integration (
/variants)OpenCode's
/variantspicker lists a model'svariants, auto-generated from models.dev (which doesn't cover our custom IDs). The setup now declaresvariantsexplicitly for effort-capable models anddisabled:truethe tiers each model doesn't support (OpenCode filters them out), so/variantsshows exactly what each model supports.Verification
133 tests pass;
tsc/lint/check/buildgreen. Every functional change cross-checked three ways: read code → CC CLI bundle → live probe.Known minor (non-blocking)
maxOutputTokensinmodels.jsononly covers the original 14 models; new ones fall back to 128000. The CC catalog exposes no max-output field, so this is cosmetic (OpenCode's output-limit meter only) — left for the maintainer to fill when data is available.Suggested merge: squash
Several commits are fixups to earlier ones in this PR; squashing yields one clean commit on
main(full breakdown preserved above).🤖 Generated with Claude Code