From 6da3377a5b63062a4318b728d8421f379271d0bd Mon Sep 17 00:00:00 2001 From: Jeff Larson Date: Mon, 27 Jul 2026 22:29:56 -0700 Subject: [PATCH 1/2] =?UTF-8?q?docs(adr):=20ADR-0034=20cut-choice=20contra?= =?UTF-8?q?ct=20=E2=80=94=20target-choice,=20not=20a=20mechanism-menu?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Settle the ADR-0032 minimality fork: the model names the compromised on-path nodes ({assessment, reason, contain:[node-key]}); determinism resolves each to its narrowest legal cut. Per-target minimality is monotone/deterministic (no judgment to delegate), and a node-key is what a 1.7B judge can reliably copy from attended evidence — so this is the 1.7b-viable contract (JEF-568 re-scope). Collapses the 4-value verdict to a 3-value assessment. Menu is advisory input; guards downgrade to Uncertain never Refuted; skeptic default inert both ways; journal v2 stores resolved cut_signature+fingerprint so replay can't repoint. Refines ADR-0032 §3; + the idea brief. No code yet. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP --- docs/adr/0032-model-is-incident-responder.md | 5 + docs/adr/0034-cut-choice-contract.md | 141 +++++++++++++++++++ docs/adr/README.md | 1 + docs/ideas/cut-choice-contract.md | 103 ++++++++++++++ 4 files changed, 250 insertions(+) create mode 100644 docs/adr/0034-cut-choice-contract.md create mode 100644 docs/ideas/cut-choice-contract.md diff --git a/docs/adr/0032-model-is-incident-responder.md b/docs/adr/0032-model-is-incident-responder.md index 4730ed09..2b24eea8 100644 --- a/docs/adr/0032-model-is-incident-responder.md +++ b/docs/adr/0032-model-is-incident-responder.md @@ -2,6 +2,11 @@ - Status: Proposed - Date: 2026-07-27 +- Refined by [ADR-0034](0034-cut-choice-contract.md): §3's *mechanism-menu* (`cuts:[menu-id]`) + becomes **target-choice** (`contain:[node-key]` — the model names the compromised nodes, + determinism resolves each to its narrowest legal cut), and the 4-value verdict collapses to + a 3-value assessment. This ADR's vocabulary/shapes/ladder/entry-exclusion all survive as the + resolver + proposal fallback. ## Context diff --git a/docs/adr/0034-cut-choice-contract.md b/docs/adr/0034-cut-choice-contract.md new file mode 100644 index 00000000..03f9c89e --- /dev/null +++ b/docs/adr/0034-cut-choice-contract.md @@ -0,0 +1,141 @@ +# 0034. The cut-choice contract: the model names the compromised nodes; determinism resolves the narrowest cut + +- Status: Proposed +- Date: 2026-07-28 + +## Context + +[ADR-0032](0032-model-is-incident-responder.md) makes the model the incident responder but +left its **decision output** sketched as a *menu of mechanisms* (§3: the model emits +`cuts:[menu-id…]` selecting `QuarantineEntry` / `QuarantineWorkload` / `DenyNetworkPath` +edge-cut lines). Two things force that open question closed and, on examination, redirect it: + +1. **The judge is a 1.7B CPU model, first (JEF-568 re-scope).** The parent plan assumed a 4B + judge ("do not assume 1.7b"). Re-scoped: qwen3:1.7b is the deployed judge and passes the + current 4-value verdict (14/15 this session, the miss a mislabeled fixture; ADR-0026 12/12). + The contract must be one **1.7b can emit reliably** — strict JSON, correct ids, correct + minimality, no over-cut at temp>0 — or the refactor lands unarmed. Escalate to 4B/8B only + on measured failure. +2. **Mechanism choice is not a judgment (current vocabulary).** For any one target, minimality + is monotone and deterministically computable — entry: surgical `DenyNetworkPath` ⊂ + `QuarantineEntry` (the `containment_for` ladder); downstream compromised workload: + `QuarantineWorkload` is the *only* additive-live lever. No two incomparable mechanisms for + one target ⇒ nothing to delegate. ADR-0032's own rail says the model chooses **what**, the + rails bound **how** — mechanism is *how*. + +Option A (model picks mechanism ids) therefore front-loads the hardest cognitive task (pick +from 2^N opaque-id subsets, comparing mechanisms) onto the weakest link, **for zero authority +gained** — and its membership guard has no teeth against the worst small-model failure: +every menu entry is legal by construction, so "select the whole list" passes every check. + +## Decision + +**The model names the compromised nodes; determinism resolves each to its narrowest legal +cut.** (Option B. Supersedes ADR-0032 §3's mechanism-menu; ADR-0032's vocabulary, shapes, +ladder, and entry-exclusion all survive as the resolver + fallback.) + +1. **Output schema (the contract):** + ```json + {"assessment": "attack" | "no_attack" | "uncertain", + "reason": "", + "contain": ["", ...]} + ``` + `contain` elements are **workload node keys copied verbatim** from the containment-options + section (entry and/or downstream workloads). Empty array = leave everything running. + `attack` + empty `contain` is **valid** ("attack, but no cut warranted") and routes to the + human-proposal fallback. Internal type: `IncidentDecision { assessment: Assessment, reason: + String, cuts: Vec }`, `ChosenCut { node: NodeKey, action: ProposedAction, + cut_signature: String }` — cuts are resolved by the engine from `contain`, never carried as + model text. + +2. **The 4-value `Verdict` collapses to a 3-value `Assessment`** (`attack` / `no_attack` / + `uncertain`). `Confirmed` vs `Exploitable` encoded a *deterministic* fact (is there a live + signal) into the model's vocabulary; that fact lives in `ProvenChain::corroborated` and + never needed the model to restate it. Fewer output values = fewer temp>0 boundary flips. + +3. **Tolerant parser, skeptic default.** Extract first `{`…last `}`; any JSON failure → + `(uncertain, no cuts)`. Then: `assessment` out of range → uncertain/no-cuts; `contain` + absent → `[]`; non-array or non-string element → uncertain/no-cuts; normalize each element + (trim, strip echoed `<<< >>>` fencing) and **exact-match the selectable menu set** — any + non-member degrades the **whole** decision to uncertain/no-cuts (a partially hallucinated + list is ungrounded reasoning); dedup+sort; `assessment ∈ {no_attack, uncertain}` with + non-empty `contain` → uncertain/no-cuts + re-judge. **Every degradation is `Uncertain` — + never `Refuted`, never a hidden line of evidence (ADR-0029).** + +4. **The menu render (advisory input, pure reuse).** Per entry-incident, one selectable line + per containable on-path workload: the **entry line** (mechanism = `containment_for`'s + ladder result — surgical edge-cut if an additive-reversible one exists, else + `QuarantineEntry`), and one **downstream line** per evidence-bearing workload + `quarantine_targets_on_path` marks (mechanism = `QuarantineWorkload`). Entry-exclusion from + the workload-quarantine set preserved (ADR-0022). Each line: fenced node key, fixed-string + mechanism (`ProposedAction::describe` — no untrusted text in action words), and a + `predict_blast_radius` note (advisory; the actuator's blast gate still runs + post-decision). Only additive-live + reversible + labeled targets are **selectable**; + evidence-bearing-but-uncontainable nodes get one aggregate **non-selectable** line so the + model isn't baited into naming them. Deterministic render (sorted, deduped, same snapshot) + ⇒ the menu is part of the full-state prompt ⇒ `prompt_cache_key` covers it: a mapping + change is a prompt change is a re-judge (ADR-0023 unchanged). + +5. **Grounding guards (all ADR-0029-admissible; all → Uncertain + re-judge, never Refuted):** + menu-membership (structural, §3); **per-node containment grounding** — a contained + *downstream* node whose own block is "no evidence observed" downgrades (the "never contain + a merely-reached node" rule enforced as citation-grounding; the *entry* is exempt — any + evidence on its path grounds containing the front door, per ADR-0022); CVE/tag grounding + (`guard_fabricated_cve` / `guard_fabricated_reachability_tag` over the entry+downstream + union, unchanged); assessment↔cuts consistency (§3). `guard_unsupported_exploitable` + (zero-anchor → Refuted) is grandfathered by ADR-0029's scope note. + +6. **Ledger consumption (strengthened Q5).** `MitigationLedger::reconcile` takes per-entry + decisions as input. Desired set = model-chosen cuts whose entry still has a proven + justifying chain (they clear the JEF-566 auto-action gate), **plus** `containment_for` + fallback proposals for every breach-relevant entry with *no current decisive decision* + (model unavailable / uncertain / parse-degraded), stamped `adjudicated=false` so they can + never auto-apply. The deterministic `quarantine_targets` desired-set insertion in + `reconcile` is **deleted** (completing the ADR-0032 auto-fire removal). `containment_for` + is thereby demoted to exactly the human-proposal fallback. + +7. **Retirement asymmetry (safety-critical).** A cut self-retires when (a) no proven chain + justifies it, or (b) a **fresh decisive decision** for its entry omits it. A fresh + `Uncertain` retires **nothing** and cuts nothing — the skeptic default is inert in both + directions, so a transient model outage can neither open a live attack path nor sever one. + +8. **Journal schema v2 (replay can't repoint a cut).** New tagged variant `IncidentDecision { + entry, objectives, assessment, reason, cuts:[{node, action, cut_signature}], menu_hash, + fingerprint }` — cuts resolved *at decision time*. Two locks: re-seeding the cache requires + the current full-prompt `fingerprint` to match (fingerprint ⊇ menu, so a shifted mapping + cold-re-judges); re-arming a cut requires the *recomputed* node→action resolution to yield + the stored `cut_signature` byte-identically (a label/ladder drift drops the cut and + cold-re-judges). Old `Breach` lines replay display-only; entries cold-re-judge for cuts + (accepted ~20-min startup cost). + +9. **Prompt shape.** Holistic single document, **no few-shot, no numbered procedure** (JEF-134). + The containment-options section goes **last, immediately before the output instruction** + (recency maximizes copy fidelity). The word "quarantine" appears only inside fixed mechanism + strings, never in the instructions (JEF-451 — don't make the cut words the most-primed + n-grams). `incident/` module dir keeps every file < 1000 lines. + +10. **Transport unchanged; constrained decoding is escalation step 1, not a dependency.** Keep + the current call + tolerant parser. If T2b's failing bar is *JSON validity* (not content), + the first escalation is Ollama grammar-constrained structured output (native `format` + schema), A/B'd like any prompt change. Only if *content* fails does the model tier escalate + (4B → 8B), per JEF-568 → recorded in ADR-0033. + +## Consequences + +- **T3 (JEF-570)** builds against a fixed target (D1–D9); its Option-A description is + superseded. **T2b (JEF-568)** extends the bakeoff to score assessment (ground truth remapped + 4→3), cut-set (exact-set primary), the refute traps (incl. downstream-CVE-only must not + appear in `contain`), minimality, and **temp-0.8 over-cut mass** (the one metric guards + can't backstop), and gates the judge on the deployed 1.7B before wiring. +- **The residual risk is grounded over-cut** — a reversible, blast-gated, shadow-default + proposal in the worst case. The rails (shadow-default, blast/alive-collateral gate, + reversible/additive + self-revert, enforceScope) hold; the bench is what tells us whether + 1.7B's over-cut mass is acceptable. +- **Model unavailable** ⇒ fallback proposals only; standing cuts persist until chain-clear + (§7's deliberate asymmetry). +- Refines [ADR-0032](0032-model-is-incident-responder.md) §3 (mechanism-menu → target-choice) + and its 4-value output (→ 3-value). Judge tier remains [ADR-0033] (pending T2b). +- Deferred: edge-granular downstream cuts (no actuator lever — the schema versions forward via + an optional qualifier, so B forecloses nothing); model-chosen mechanism (revisit only if the + vocabulary ever holds two incomparable mechanisms for one target); internal-only incidents + stay propose-only (ADR-0032). diff --git a/docs/adr/README.md b/docs/adr/README.md index f24a7c8a..08c49edb 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -40,5 +40,6 @@ Copy [`0000-template.md`](0000-template.md) to start one. | [0030](0030-app-level-oidc-verification-supersedes-edge-trust.md) | App-level, provider-agnostic OIDC verification supersedes edge-only trust: the dashboard + `/api/*.json` today carry only CSP (auth trusted entirely at the Cloudflare Access edge), so one `kubectl port-forward` reads the whole graph. Protector becomes an OAuth RESOURCE SERVER (verifies sig/iss/aud/exp/nbf, alg pinned to the issuer's asymmetric family — never from the token's own `alg`; extracts subject + a configurable tier) against a CONFIGURABLE issuer; it ships NO IdP (Dex/Keycloak out of scope; supersedes the in-cluster-Dex idea) and cannot actuate. ID-JAG is just a JWT the same verifier accepts. The JWKS/discovery GET is the ADR-0015 same-request-for-everyone lane (public keys in, no cluster datum out), not an egress breach. FAIL-CLOSED when configured (every error → 401/403/503, JWKS-unreachable is 503 not bypass); UNCONFIGURED behaves as today but logs loudly — the only bypass, and it announces itself. CF Access already issues a verifiable JWT, so pointing the verifier at the CF issuer closes the hole with no new interactive flow. Reaffirms 0016 (auth gates WHO MAY VIEW, not the verdict path) | Proposed | | [0031](0031-read-only-mcp-server-tiered-redaction.md) | A read-only, tiered-redaction MCP server — the second sanctioned egress carve-out (pull-side sibling of 0018's push notifier). Four read-only tools (`list_findings` / `explain_verdict` / `get_coverage` / `signing_inventory`); NO actuation tool exists by construction (view-never-a-gate / shadow-first, 0016). Three tiers `redacted`/`forensic`/`raw` with the tier a server-enforced CEILING from a verified ADR-0030 claim (arg may only narrow) — `redacted` safe-by-construction, `forensic`/`raw` off-by-default, opt-in, journaled operator-owned egress; secret VALUES have no unlock tier. Redaction is server-side/in-cluster BEFORE egress, so protector IS the remote HTTP MCP server (reusing 0018's lifted scrubbers). Transport: RMCP behind our OIDC verifier (single-pathed, in-tree trust decisions; hand-rolled JSON-RPC fallback if it won't compose), ID-JAG protected-resource discovery for zero-touch enterprise auth. Depends on 0030 | Proposed | | [0032](0032-model-is-incident-responder.md) | The model is the **incident responder**: over the whole internet-facing path it decides what is an attack and which cut to apply — choosing from a menu that determinism ENUMERATES — at minimum scope. Determinism proves/enriches/feeds/bounds; it does NOT decide the cut. The model now SEES per-node downstream evidence (was entry-only); the `is_live_corroborated` unconditional `QuarantineWorkload` auto-fire is deleted; internal-only actively-exploited pods become propose-only. Supersedes 0022's JEF-284 decision procedure (its vocabulary/shapes/ladder survive as the menu + proposal fallback); evolves 0009 (adjudicator veto → cut selector); resolves JEF-322/547. Judge tier deferred to 0033 (do NOT assume qwen3:1.7b; expected qwen3:4b). Realizes the VISION north star | Proposed | +| [0034](0034-cut-choice-contract.md) | The **cut-choice contract**: refines 0032 §3 from a *mechanism-menu* to **target-choice** — the model outputs `{assessment: attack/no_attack/uncertain, reason, contain:[node-key…]}`, naming the compromised on-path nodes; determinism resolves each to its narrowest legal cut (the `containment_for` ladder / `QuarantineWorkload`), because per-target minimality is monotone/deterministic (no judgment to delegate) and a node-key is what a 1.7B can reliably copy from attended evidence. Collapses the 4-value verdict → 3-value assessment. Menu rendered as advisory input (mechanism + blast-radius); guards (menu-membership + per-node containment-grounding) downgrade to Uncertain, never Refuted; skeptic default is inert both ways (a model outage neither cuts nor lifts a standing cut); journal v2 stores the resolved `cut_signature`+fingerprint so replay can't repoint. Chosen for 1.7B-viability (JEF-568 re-scope: 1.7b-first); residual risk is grounded over-cut, measured by the T2b temp-0.8 bench. Supersedes 0032 §3 | Proposed | See also [`../VISION.md`](../VISION.md) for the longer-form narrative this ADR realizes. diff --git a/docs/ideas/cut-choice-contract.md b/docs/ideas/cut-choice-contract.md new file mode 100644 index 00000000..ba044ca6 --- /dev/null +++ b/docs/ideas/cut-choice-contract.md @@ -0,0 +1,103 @@ +# Idea — the incident-responder cut-choice contract + +**Status:** decided (2026-07-28). Realized by [ADR-0034](../adr/0034-cut-choice-contract.md); +refines [ADR-0032](../adr/0032-model-is-incident-responder.md) §3. Sprint tickets: JEF-568 +(bench), JEF-570 (build), JEF-569 (shadow+arm), plus the new `incident/` module chunk. + +## Idea + +Fix the output shape + machinery by which the model decides **what to cut** along an +internet-facing attack path — the [ADR-0032](../adr/0032-model-is-incident-responder.md) +keystone left open. Settle the "minimality fork" so T3 can build it and T2b can score it. + +## Problem & context + +ADR-0032 commits protector to "the model is the incident responder" but left the decision +contract's exact shape open, sketched as a *menu of mechanisms* (Option A). The re-scoped +constraint (JEF-568: **qwen3:1.7b-first**, escalate only on measured failure) invalidates the +parent idea's "plan of record: 4B." The contract must be one a **1.7B CPU judge can emit +reliably**, or the refactor lands unarmed. Everything around it is settled: per-node +downstream evidence in the prompt (JEF-565, live), the uniform auto-action gate (JEF-566, +live in `respond/mod.rs::is_live_corroborated`), the delta/cache gate (ADR-0023), +grounding-guard doctrine (ADR-0029). Only the decision output and its consumers remain. + +## Assumptions challenged + +1. **"A menu-of-mechanisms is what ADR-0032's authority requires" — overclaimed.** ADR-0032's + own rail: the model chooses *what*; the rails bound *how*. Mechanism (edge-cut vs + quarantine) is *how*. The model's authority is over *what is an attack / what to cut / + what to leave* — **targets**, not which NetworkPolicy shape severs them. +2. **"Mechanism choice involves judgment" — false for the current actuator vocabulary.** For + any one target, minimality is **monotone and deterministically computable**: for the + entry, surgical edge-cut ⊂ entry-quarantine (the existing `containment_for` ladder); for a + downstream compromised workload, `QuarantineWorkload` is the *only* additive-live lever. + No two incomparable mechanisms exist for one target → no judgment left to delegate. Asking + a 1.7B to pick the mechanism is asking it to re-derive a theorem, at tail-flip risk, for + zero authority gained. +3. **"1.7b can produce the cut contract" — shaky but shapeable.** It provably does the 4-value + verdict (14/15 this session; ADR-0026 12/12). Its documented failures are n-gram parroting + of primed instruction phrases (JEF-134, JEF-451) and size-correlated tail-flips at temp>0 + (ADR-0029). Both hit an *opaque-id, mechanism-comparing* output (A) far harder than a + *copy-the-node-key-you-just-analyzed* output (B). Choose the contract that sits inside what + it provably does; T2b decides whether it holds. + +## Approaches considered + +- **A — model chooses cuts from a mechanism menu** (`cuts:[menu-id…]`). Rejected: output space + explodes from 4 verdict values to 2^N cut subsets (every temp>0 flip has N places to land); + "fewest/narrowest" is exactly the comparative instruction small models drop; and crucially + **the menu-membership guard has no teeth against over-cut** — every menu entry is legal by + construction, so "select the whole list" (the worst parrot failure) passes every check. +- **B — model names the compromised nodes; determinism maps node → narrowest legal action** + (`contain:[node-key…]`). The selector is a node key the model just reasoned over in that + node's own evidence block (copy-from-attended-context — the one structured act small + transformers do reliably). Output space is the on-path node set (~2–6), not mechanism×node. + The grounding guard gains real teeth: *a contained downstream node must carry evidence in + its own block* (closed membership, ADR-0029-admissible) — catches most over-cut + mechanically. **Chosen.** +- **C — two calls (assess, then cut).** Rejected: re-carries full evidence (same context + cost), doubles CPU latency per positive, splits the cache/delta machinery, and breaks the + single-document holism the parent idea already chose. + +## Decision — B, with the menu rendered into the prompt as *advisory* context + +The model **sees** a deterministic per-node containment menu (resolved mechanism + +blast-radius note — a responder should weigh collateral), but its **output names targets, not +mechanisms**. Not a hedge — it's B with honest inputs. It wins on all three axes in priority +order: (1) smallest delta from the proven 4-value contract → 1.7B-viable; (2) genuine +north-star authority (the model decides what's an attack, which nodes are in it, what to +leave, and that empty-set is legal — determinism only renders each chosen target into its +narrowest actuator-legal object, the *how* ADR-0032 already assigns to the rails); (3) +node keys are the most content-derived ids possible → replay-safe. Closed vocabulary +throughout; no free-form action ever. "Edge-cut the entry, don't quarantine it" is preserved: +`contain:[entry]` resolves through the ladder to the surgical edge-cut when one exists. + +See [ADR-0034](../adr/0034-cut-choice-contract.md) for the full contract (schema, parser, +menu render, guards, ledger, journal v2, prompt shape, file plan) — reproduced there as the +authoritative record. + +## The one genuinely open risk + +**Grounded over-cut** — the model contains a node that *is* evidenced but should have been +left running. No guard can catch it (it passes every grounding check); only the **temp-0.8 +over-cut-mass** metric sees it, and the worst case is a reversible, blast-gated, +shadow-default *proposal* — the rails hold. This is exactly what T2b's bench measures on the +deployed 1.7B; it is the honest gate on "does 1.7B hold or do we escalate." + +## Sequence + +1. `incident/` module (types / menu resolver / parser / guards) — **pure, exhaustively + unit-tested, no engine wiring; unblocked NOW** (depends on the contract, not the judge + tier). +2. Prompt splice + bakeoff sync + **T2b bench on the deployed 1.7B → ADR-0033** (the gate, + *before* wiring). +3. Engine wiring: `adj_pass` folds `IncidentDecision`; `reconcile` consumes decisions + + `containment_for` demotion + delete the deterministic quarantine desired-path. +4. Journal v2 + replay locks. +5. Shadow bake with a model-chosen-vs-`containment_for` comparator; arm per class (ADR-0021). + +## Handoff + +Tickets already exist (JEF-568/570/569) — this brief **reconciles** them to B rather than +creating new ones; the one addition is the pure `incident/` module as the unblocked first +chunk. JEF-570's Option-A description is superseded by ADR-0034. From d975b05f4d461fa16dcf1fb417a791976d23e368 Mon Sep 17 00:00:00 2001 From: Jeff Larson Date: Tue, 28 Jul 2026 20:11:02 -0700 Subject: [PATCH 2/2] =?UTF-8?q?docs(adr):=20ADR-0033=20cut-choice=20judge?= =?UTF-8?q?=20tier=20=E2=80=94=20qwen3:1.7b=20holds=20with=20an=20exact-se?= =?UTF-8?q?t=20prompt=20(JEF-568)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the T2b measurement: the deployed qwen3:1.7b judge lands the ADR-0034 cut-choice contract (assessment 8/8, all refute/cut traps, and the cut-set with a tightened output instruction that pins contain to exactly the evidence-bearing set). No escalation to 4B. Includes the methodology finding — the cut-choice bench must run on the deployed pod (local arm64 backend diverges from deployed amd64 on the cut-set). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01VtjoJttCvBY4dzCoE4f9vP --- docs/adr/0033-cut-choice-judge-tier.md | 62 ++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 docs/adr/0033-cut-choice-judge-tier.md diff --git a/docs/adr/0033-cut-choice-judge-tier.md b/docs/adr/0033-cut-choice-judge-tier.md new file mode 100644 index 00000000..1ce29f1d --- /dev/null +++ b/docs/adr/0033-cut-choice-judge-tier.md @@ -0,0 +1,62 @@ +# 0033. Cut-choice judge: qwen3:1.7b holds the contract with an exact-set containment prompt + +- Status: Accepted +- Date: 2026-07-28 + +## Context + +[ADR-0034](0034-cut-choice-contract.md) fixes the cut-choice contract — the model emits +`{assessment, reason, contain:[node-key…]}`, determinism resolves each named node to its +narrowest reversible cut — but left one question to be **measured, not assumed** (JEF-568 / +T2b): can the deployed judge (qwen3:1.7b, a 1.7B CPU model) emit it reliably — correct 3-value +assessment, exact minimal cut-set, no over-cut — or must the judge escalate to a 4B model? +ADR-0034's own premise is that the contract "must be one **1.7b can emit reliably** … escalate +to 4B/8B only on measured failure." This ADR records that measurement and the resulting tier + +prompt. + +## Decision + +**The judge stays qwen3:1.7b.** A cut-choice bench (`scripts/judge_bakeoff_cutchoice.py`, +JEF-568) scores the ADR-0034 schema on the deployed judge across both evidence directions +(entry loaded-CVE; downstream behavioral / exposed-secret), the minimality centerpiece (clean +entry + live-compromised downstream → contain the downstream *only*), the JEF-588 +downstream-CVE cut trap, and the JEF-402 / broad-RBAC refute traps. On the **deployed pod** +(temp-0, the greedy prod path): + +- **Assessment: 8/8**, and **every refute/cut trap passes** — 1.7b never over-cuts a clean + workload, and correctly returns `no_attack` / `[]` on a downstream loaded-CVE behind a clean + edge (JEF-588) and on broad RBAC / reachable-secret-no-evidence (JEF-402). +- **The cut-set lands with a tightened output instruction.** The first deployed run exposed an + *under-cut*: 1.7b recognized the attack but returned `contain=[]` on 3 of 4 real attacks — a + recognized breach with no proposed cut. Pinning `contain` to **exactly the evidence-bearing + set** — "name every compromised workload and no others; an `attack` with an empty `contain` + is contradictory" — fixed all three on the deployed pod *without* inducing over-cut: + entry-loaded-CVE → `{entry}`; clean-entry + live-downstream → `{downstream}` only (the + minimality centerpiece, no entry over-cut); both-evidenced → `{entry, downstream}`. + +**That tuned prompt is the one JEF-570 wires into `build_judgment_prompt`.** It is validated on +the deployed judge, not guessed. Escalation to 4B is deferred — unnecessary on this evidence, +and not authoritatively comparable without first adding 4B to a cluster ollama pod (see +Methodology). + +### Methodology: the cut-choice bench must run on deployed hardware + +A local (Mac arm64/Metal) run of the *same* model tag, prompt, and `num_ctx=16384` **diverges +from the deployed pod (amd64/CPU) on the cut-set**: local *over*-cut the clean entry; deployed +*under*-cut to `[]`. Assessment and the traps agreed across backends; only the cut-set — +decided at greedy tie-breaks a different llama.cpp backend resolves differently — diverged. The +judgement of a cut-choice change is therefore trustworthy only on the deployed judge; a local +proxy is unfaithful for the cut-set. The bench points at a port-forwarded pod via `OLLAMA_URL` +for exactly this reason. + +## Consequences + +- The cut-choice contract is armed on the deployed judge — no model swap, and no arming a 1.7b + that could not emit it. +- The residual risk ADR-0034 named (grounded over-cut) is bounded by the exact-set prompt + the + ADR-0034 guards, and is now a measurable, regression-tracked quantity — the bench's cut-set + score and `--flip` over-cut mass, **run on the deployed pod**. +- A future escalation to 4B (e.g. if the downstream/pivot lane stresses 1.7b) requires an + on-cluster 4B bench first; local numbers do not transfer. +- [JEF-570] wires the `incident/` module (ADR-0034, merged in #296) and this validated prompt + into `adj_pass` / `reconcile` / the journal.