Skip to content

ci: split PR benchmarks by topic#2303

Merged
zeitlinger merged 1 commit into
mainfrom
ci/pr-benchmark-topic-split
Jul 20, 2026
Merged

ci: split PR benchmarks by topic#2303
zeitlinger merged 1 commit into
mainfrom
ci/pr-benchmark-topic-split

Conversation

@zeitlinger

Copy link
Copy Markdown
Member

Summary

Replace the parallel base/head PR benchmark comparison with topic-split benchmark jobs. Each topic compares base and head sequentially on the same runner, while topics still run in parallel for better wall-clock time.

Changes

  • Split PR benchmarks into topic jobs: counter, histogram, and exposition.
  • Run base and head sequentially inside each topic job so each comparison uses the same runner.
  • Merge per-topic JMH JSON results in the summary job before generating the PR comment.
  • Add a comparison note clarifying that base/head are compared on the same runner within each topic.
  • Add histogram benchmark variants for classic-only histograms:
    • shared contended 4-thread benchmark remains as prometheusClassic
    • single-thread benchmark: prometheusClassicSingleThread
    • per-thread 4-thread benchmark: prometheusClassicPerThread

Why

Running base and head in separate jobs was faster but not reliable enough because GitHub Actions can schedule them on different CPU models. Topic splitting keeps the reliable same-runner comparison within each topic while still allowing unrelated benchmark topics to run in parallel.

Validation

  • Ran mise run lint:fix.
  • Ran mise run build.
  • Ran python3 -m py_compile .mise/tasks/generate_benchmark_summary.py.
  • Ran git diff --check.
  • Verified the JMH include patterns select the intended benchmarks with java -jar benchmarks/target/benchmarks.jar -l ....
  • Smoke-tested the topic result merge and summary generation using the latest perf: optimize classic-only histogram observe path #2252 benchmark artifact.

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger
zeitlinger merged commit 0a91771 into main Jul 20, 2026
18 checks passed
@zeitlinger
zeitlinger deleted the ci/pr-benchmark-topic-split branch July 20, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants