Skip to content

feat(groom): builder authors the PR body per the comfy-pr convention (BE-4346)#67

Open
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-4346-groom-builder-pr-body
Open

feat(groom): builder authors the PR body per the comfy-pr convention (BE-4346)#67
mattmillerai wants to merge 2 commits into
mainfrom
matt/be-4346-groom-builder-pr-body

Conversation

@mattmillerai

Copy link
Copy Markdown
Contributor

ELI-5

When the groom sweep auto-builds a fix, it opens a pull request. Until now the PR description was machine-assembled — a loud banner followed by the reviewer-bot's raw "here's the problem" paragraph, with the text hard-wrapped so GitHub rendered it as a wall of mid-sentence line breaks. It didn't follow the team's PR convention (a plain-language ## ELI-5 first, then a structured what/why, one line per paragraph). This change hands the PR-writing job to the agent that actually made the change — it now writes a proper ELI-5-first, structured description — while the workflow keeps prepending the "review-only, never auto-merged" banner and appending the ledger marker.

What changed

  • .github/groom/builder.md — the builder brief now instructs the agent to write the human-facing PR body to a new {{PR_BODY_OUT}} file (when it patches): ## ELI-5 first, then ## What changed / ## Why, with the "never hard-wrap — one line per paragraph, let GitHub soft-wrap" rule embedded directly (the builder runs claude -p --bare --setting-sources "" and cannot invoke the comfy-pr skill, so the convention is spelled out in the brief). On a bail it writes no body.
  • .github/groom/ledger.py — new builder_pr_body(banner, eli5_body, verifier_rationale, signature) helper assembles the final body: banner first, the builder's ELI-5 body, the verifier rationale demoted to a secondary <details> section, and the signature_marker LAST. Falls back to the original banner + ## Verifier rationale + marker template when the body is empty or doesn't lead with an ELI-5 heading.
  • .github/workflows/groom.yml — adds the PR_BODY_OUT env var and the builder's Edit grant + prompt substitution for it (mirroring the proven BUILDER_OUT write path); the pre-publish secret scan now covers the PR body alongside the patch and summary (fail-closed bail if the key appears); the capture step hands a size-capped, scanned body to build_pr via the build artifact; build_pr calls builder_pr_body instead of assembling the string inline.
  • .github/groom/tests/test_ledger.pyBuilderPrBodyTest covers the happy path (banner first / marker last / rationale in <details>), the empty-body and non-ELI-5 fallbacks, ELI-5 heading variants, a whitespace-only body, and the spoofed-marker injection guard.

Why

Groom builder PRs are read by humans, so they should follow the same PR convention as hand-written ones (evidence: comfy-cloud-mcp-server #837 opened with the raw rationale and no ELI-5). The builder agent knows exactly what it changed, so it's the right author. The banner and ledger marker stay load-bearing (the banner sets review expectations; the ledger keys dedup on the marker), so they remain workflow-controlled — the marker is appended LAST so an embedded/spoofed marker in the model body can't shadow it (extract_signature reads the last marker). Behavior is preserved on every non-happy path: the fallback template is byte-identical to today's output.

Acceptance criteria

  • PR opens with ## ELI-5 first, structured what/why, verifier rationale as a secondary <details> section (enforced: build_pr falls back to the plain template unless the body leads with an ELI-5 heading).
  • Prose not hard-wrapped — one line per paragraph/bullet (embedded rule in builder.md).
  • Banner first, signature_marker last (both in builder_pr_body; unit-tested).
  • Builder bail / empty / oversized body → fallback to the existing banner + rationale template; never an empty-body PR.
  • Pre-publish secret scan covers the builder-authored body before gh pr create (extended the capture-step scan; build_pr holds no model key, so scanning must and does happen in the credential-free build job before the artifact is uploaded).
  • .github/groom/tests covers the new body path incl. the fallback (54 tests pass).

Judgment calls

  • Separate PR_BODY_OUT file rather than a pr_body JSON field. The ticket offered either; a markdown file mirrors the finder/verifier *_OUT pattern and avoids the model having to emit a large multiline markdown blob as an escaped JSON string. It uses the identical Edit(//<path>) grant that the shipped BUILDER_OUT write relies on.
  • The verifier rationale is assembled by build_pr (deterministic), not by the builder. build_pr already has the verifier text in scope; keeping it out of the model's hands means the rationale is never rewritten and its own upstream secret-scan still stands. The builder only authors the ELI-5 + what/why.
  • Body cap of 20000 bytes — well under GitHub's 65536-char PR-body limit, leaving headroom for the wrapped banner/rationale/marker; oversized bodies are dropped (→ fallback), never truncated.
  • Negative-claim falsification: N/A — this change adds a capability (builder-authored bodies) and its fallbacks produce a more complete PR body; it denies no user-facing capability, so there is no dead-end to falsify.

Out of scope (per ticket): the groom issue body, groom-security findings (filed as issues), and the finder/verifier briefs.

…(BE-4346)

The groom auto-builder (builder: true, BE-4003) assembled its PR body in the
workflow as banner + raw verifier rationale + marker, so every groom PR opened
with a dense rationale paragraph and no ELI-5, no structured what/why, and
hard-wrapped prose — the opposite of the comfy-pr convention.

The builder agent — which made the change — now authors the human-facing body
into $PR_BODY_OUT: an ## ELI-5 section first, then What changed / Why, one line
per paragraph (no hard-wrap). build_pr wraps it with the load-bearing banner
(first) and ledger signature_marker (last), retains the verifier rationale as a
secondary <details> section, and falls back to the original banner+rationale
template on a builder bail / empty / oversized / non-ELI-5 body — never an
empty-body PR. The pre-publish secret scan now covers the PR body alongside the
patch and summary before it is published.

Body assembly is factored into ledger.builder_pr_body() with unit tests
covering the happy path, the fallback, and the spoofed-marker injection guard.
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 57 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f999b7ba-fed3-4e3f-be59-67a68f6fd13b

📥 Commits

Reviewing files that changed from the base of the PR and between fd35bf2 and 6fa063b.

📒 Files selected for processing (4)
  • .github/groom/builder.md
  • .github/groom/ledger.py
  • .github/groom/tests/test_ledger.py
  • .github/workflows/groom.yml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch matt/be-4346-groom-builder-pr-body
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch matt/be-4346-groom-builder-pr-body

Comment @coderabbitai help to get the list of available commands.

@mattmillerai mattmillerai added cursor-review Multi-model cursor review agent-coded Authored by the agent-work loop labels Jul 24, 2026
@mattmillerai
mattmillerai marked this pull request as ready for review July 24, 2026 06:44

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 5 finding(s).

Severity Count
🟡 Medium 2
🟢 Low 3

Panel: 8/8 reviewers contributed findings.

Comment thread .github/workflows/groom.yml Outdated
Comment thread .github/workflows/groom.yml
Comment thread .github/groom/ledger.py Outdated
Comment thread .github/workflows/groom.yml
Comment thread .github/groom/ledger.py Outdated
… markup injection (BE-4346)

Address cursor-review panel findings on the builder-authored PR body:

- groom.yml: reading /tmp/build/pr_body.md caught only FileNotFoundError, so a
  non-UTF-8 body (its bytes pass the byte-wise secret scan + wc -c cap upstream)
  would raise UnicodeDecodeError and crash AFTER `git push` — orphaning the
  branch, opening no PR, and re-proposing the finding every run. Catch
  (OSError, UnicodeDecodeError) and fall back to the template.
- ledger.py builder_pr_body: cap the assembled body under GitHub's 65536-char
  PR-body limit by truncating the (previously unbounded) verifier rationale;
  the banner and trailing ledger marker are always preserved.
- ledger.py _leads_with_eli5: strip fenced code blocks before locating the first
  heading so a decoy `## ELI-5` fenced at the top is no longer a false accept.
- ledger.py: neutralize `<!--`/`-->`/`</details>` in the builder body and
  verifier rationale so a prompt-injected builder can't hide the rationale/marker
  behind an unclosed comment or close the wrapping <details> early.

Adds tests covering each case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-coded Authored by the agent-work loop cursor-review Multi-model cursor review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant