Live verification scoring + standalone score command

[mkultraWasHere]

Standalone dreadgoad score command that tests agent-reported credentials against the running GOAD lab via nxc/secretsdump on the Kali attack box, replacing the old inference engine. Also adds a score reset subcommand for cleaning up between agent runs.

Added

• dreadgoad score command — scores an agent JSONL report against the answer key with optional --live-verify for authoritative results
• dreadgoad score generate-key — moved from scoreboard generate-key (hidden alias kept)
• dreadgoad score reset — cleans Kali attack box (nxc databases, tickets, NTDS dumps, Responder logs, Dreadnode sessions) and Windows hosts (webshells, share drops, temp scripts, registry dumps) between agent runs. Supports dry-run mode, rogue AD computer account purge, group membership diff, and attack tool exe blacklist
• Live credential verification via nxc smb (auth check [+], admin check (Pwn3d!)) and secretsdump.py -just-dc-user (DCSync proof)
• ShellRunner interface with AWS SSM and Azure Bastion SSH implementations
• Azure auto-discovery of Bastion host, Kali VM (Role=AttackBox tag), and SSH key from environment tags — zero flags needed with -p azure
• Human-readable summary printed before JSON output
• docs/scoring.md user-facing documentation and docs/design-live-verification.md design doc
• netbios_name field in answer key domain objectives for correct secretsdump -just-dc-user targeting
• Second agent prompt variant (scoreboard/agent_prompt-2.md) with informal/defensive framing

Changed

• Answer key classifies credentials as live_auth (ACL abuse targets whose passwords may change) vs password_match (static comparison) based on config ACL edges
• Scoreboard TUI shows a warning banner directing operators to dreadgoad score --live-verify for verified results
• kaliKeyPath() moved from cli/cmd/bastion.go to exported azure.KaliKeyPath() in cli/internal/azure/kali.go
• Agent prompt (scoreboard/agent_prompt.md) restructured: generic examples, hostname required for host findings, report path changed from /tmp/report.jsonl to ./report.jsonl
• Variant generator randomizes share names to prevent GOAD fingerprinting

Breaking

• Technique scoring removed entirely — answer keys no longer contain technique objectives; generate-key output drops from ~69 to ~38 objectives. Techniques were redundant with credential/host/domain proof and suffered from ID alignment issues between agent and answer key.
• Inference engine removed — inferHosts(), inferDomains(), inferTechniques() and ~170 lines of credential-graph inference deleted from verify.go. The scoreboard TUI now shows 0 for hosts/domains (static-only); use dreadgoad score --live-verify for those.

Fixed

• secretsdump -just-dc-user no longer fails with ERROR_DS_NAME_ERROR_NOT_FOUND — was passing the FQDN (hq.deltasystems.local/krbtgt) instead of the NetBIOS name (HQ/krbtgt)
• nxc [+] false positives from informational output lines and (Guest) fallback auth are now filtered
• Passwords with @ or : no longer break impacket's user:password@host parsing — always uses -hashes with computed NT hash
• Azure Bastion SSH no longer hangs waiting for passphrase input (cmd.Stdin = nil)
• Variant generator now re-encrypts PowerShell SecureString blobs with mapped passwords — previously leaked original GOAD passwords (e.g. powerkingftw135) into variant deployments via secret.ps1 in SYSVOL
• Host admin checks now fall back to --local-auth when domain auth fails — fixes scoring of local Administrator SAM hashes reported with a domain FQDN target (e.g. Administrator@vortexindustries.local from a SAM dump)
• Host scoring falls back to admin_users list when findings lack a hostname tag — DA creds from DCSync can now verify host access without explicit hostname tagging
• buildNXCCommand no longer passes both -d and --local-auth which nxc rejects as mutually exclusive
• Azure Bastion SSH command passing fixed — was splitting the command into separate args causing bash -c to only execute the first word
• Validation checks now distinguish WinRM transport errors from empty results — 20 runPS() callers migrated to runPSErr() so SOCKS/WinRM failures emit WARN instead of false FAIL (or false PASS for firewall). Fixes intermittent false negatives on Azure where SOCKS proxy flakiness caused checks like CertEnroll, SMB shares, domain trusts, and GPOs to report "not found" when the resource was present.
• IIS upload missing-directory severity promoted from INFO to FAIL — a missing upload dir on a host configured for IIS permissions is a real vulnerability gap
• Empty username guard and credential deduplication in scoring pipeline
• SSM poll deadline aligned with command timeout to prevent premature abandonment
• Improved error message when Azure resource ID used without -p azure

Notes

• Host and domain IPs are not auto-populated by generate-key — must be patched into the answer key after deployment (documented in scoring.md)
• Bastion SSH has ~15s overhead per command vs SSM ~2-5s; a persistent tunnel would improve Azure performance

[Copilot]

Pull request overview

This PR introduces a standalone dreadgoad score command that can optionally live-verify agent-reported credentials against a running GOAD lab (via nxc smb + secretsdump.py executed on the Kali attack box). It also removes the prior inference/technique-scoring approach, keeping the scoreboard TUI fast by remaining static-only.

Changes:

• Adds dreadgoad score + score generate-key, with shared scoring logic (ScoreReport) and optional live verification (LiveVerifier).
• Removes inference-driven host/domain/technique credit; static mode now only scores credentials, while live mode verifies hosts/domains directly.
• Adds AWS SSM + Azure Bastion SSH command runners to execute verification tooling on the Kali attack box; updates docs and agent reporting prompt accordingly.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
scoreboard/agent_prompt.md	Updates agent reporting spec (header + ./report.jsonl, host findings require hostname, scoring guidance).
docs/scoring.md	New standalone scoring documentation including live verification workflow and JSON output format.
docs/scoreboard.md	Updates scoreboard docs to point to score generate-key and clarifies static-only scoreboard behavior.
docs/design-live-verification.md	Adds design doc for live verification architecture and scoring model.
cli/internal/scoreboard/verify.go	Strips inference/technique logic; keeps static credential verification for the TUI.
cli/internal/scoreboard/verify_test.go	Updates tests to reflect static-only verification (credentials only) + new verify types.
cli/internal/scoreboard/types.go	Extends objective/output types (host_ip/dc_ip/netbios, JSON tags, ScoreResult, FailedCheck).
cli/internal/scoreboard/tui.go	Removes technique group and adds a static-only warning banner; adjusts layout sizing.
cli/internal/scoreboard/transport_ares.go	Simplifies Ares report synthesis (drops exploited-technique emission; keeps domain signals).
cli/internal/scoreboard/shell_ssm.go	Adds an SSM-based shell runner for executing commands on Kali (AWS-RunShellScript).
cli/internal/scoreboard/shell_bastion.go	Adds an Azure Bastion SSH-based shell runner for executing commands on Kali via az network bastion ssh.
cli/internal/scoreboard/score.go	Adds shared scoring routine (ScoreReport) including host/domain live verification phases.
cli/internal/scoreboard/live.go	Implements ShellRunner + LiveVerifier with nxc/secretsdump command building, parsing, and caching.
cli/internal/scoreboard/generate.go	Removes technique objective generation; adds ACL-based live_auth classification and new verify types/NetBIOS.
cli/internal/azure/kali.go	Exports azure.KaliKeyPath() and adds Azure discovery of the Kali VM by tag.
cli/cmd/scoreboard.go	Removes generate-key command, keeping a hidden alias pointing to score generate-key.
cli/cmd/score.go	Adds dreadgoad score command (summary + JSON output) and wiring for AWS/Azure live verification.
cli/cmd/bastion.go	Switches to the exported azure.KaliKeyPath() helper.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 3 comments.

Files not reviewed (1)

• modules/terraform-azure-kali/.terraform.lock.hcl: Generated file

Comments suppressed due to low confidence (1)

scoreboard/agent_prompt-2.md:59

• This section says findings are scored on "passwords obtained", but the format and examples allow evidence to be an NTLM hash as well. Updating the wording to "credentials obtained" keeps the scoring description consistent with the required fields and examples.

[Copilot]

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 3 comments.

Files not reviewed (1)

• modules/terraform-azure-kali/.terraform.lock.hcl: Generated file

Comments suppressed due to low confidence (1)

cli/internal/variant/securestring.go:113

• decryptSecureString() splits the inner payload into 3 fields but doesn’t validate the SecureString version field. If the blob isn’t version 2, the function may produce a misleading downstream error (or decode garbage). It’s better to explicitly require parts[0] == "2" and return a clear error when it doesn’t match.

	// Format: "2|iv_base64|ct_hex" (version 2 with AES key).
	parts := strings.SplitN(inner, "|", 3)
	if len(parts) != 3 {
		return "", fmt.Errorf("expected version|iv|ct format, got %d parts", len(parts))
	}