Skip to content

refactor(unparser): centralize aggregate-scope rendering in the SQL unparser#23789

Merged
kosiew merged 2 commits into
apache:mainfrom
naman-modi:refactor/unparser-agg-scope
Jul 25, 2026
Merged

refactor(unparser): centralize aggregate-scope rendering in the SQL unparser#23789
kosiew merged 2 commits into
apache:mainfrom
naman-modi:refactor/unparser-agg-scope

Conversation

@naman-modi

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

  • When printing a GROUP BY query, the unparser sometimes wraps the aggregate's input in an inner subquery (... FROM (SELECT ...)).
  • Table aliases like cs only exist inside that subquery. Any clause outside it (SELECT, GROUP BY, HAVING, QUALIFY, ORDER BY) must use the subquery's output columns, not the aliases, so the unparser drops the alias.
  • Each clause does that dropping on its own, so it is easy to miss one.
  • ORDER BY on an aggregate that is not in the SELECT list was missed: it printed ORDER BY round(sum("cs"."total_revenue"), 2), while the SELECT list in the same query correctly printed sum("total_revenue").
  • DataFusion reads that SQL back fine, but stricter databases reject it because cs is out of scope there.

What changes are included in this PR?

I moved the rule into one helper, UnparserAggScope, that checks once per aggregate whether the input is an inner subquery and then prepares expressions for each clause. SELECT, GROUP BY, HAVING, QUALIFY, and both ORDER BY paths now go through it, which fixes the ORDER BY case. The window-over-aggregate path is left as-is with a comment: it is only reachable from hand-built plans, since a window in SQL always sits inside a SELECT that already drops the alias. Only ORDER BY output changes; every already-correct case stays the same.

Are these changes tested?

Yes. New tests in datafusion/core/tests/sql/unparser.rs cover the inner-subquery shape for a window sorting by an aggregate, ORDER BY on an unselected aggregate (the fixed case), and a top-level ORDER BY (the second sort path). Existing unparser tests and the TPC-H/Clickbench roundtrips still pass.

Are there any user-facing changes?

ORDER BY over an aggregate is now printed without an out-of-scope table qualifier, so the generated SQL is valid for stricter databases. No API changes.

@github-actions github-actions Bot added sql SQL Planner core Core DataFusion crate labels Jul 22, 2026
@naman-modi

Copy link
Copy Markdown
Contributor Author

Hey @kosiew, can you please review this PR, when you get a chance?

One small design note: I went with a small UnparserAggScope struct that holds the aggregate and the "is the input a derived subquery" flag, computed once and reused by every clause, rather than a bare helper fn taking the flag as a parameter.

IMO, it reads a bit cleaner at the call sites and keeps the flag from being recomputed per clause. If you'd rather keep it consistent with the existing impl Unparser helpers, I'm happy to drop the struct and move it to a plain associated fn instead.

TIA!

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naman-modi
Nice fix overall. I like the direction of consolidating the aggregate scope handling into a small helper. I just have a couple of minor suggestions that could make the abstraction a little cleaner and the test coverage a bit clearer.

Comment thread datafusion/sql/src/unparser/plan.rs
Comment thread datafusion/core/tests/sql/unparser.rs Outdated
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.10638% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.65%. Comparing base (4a40101) to head (38cf748).

Files with missing lines Patch % Lines
datafusion/sql/src/unparser/plan.rs 85.10% 0 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23789      +/-   ##
==========================================
- Coverage   80.65%   80.65%   -0.01%     
==========================================
  Files        1091     1091              
  Lines      370692   370702      +10     
  Branches   370692   370702      +10     
==========================================
+ Hits       298972   298974       +2     
+ Misses      53885    53884       -1     
- Partials    17835    17844       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@naman-modi

naman-modi commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing @kosiew! I've addressed both review comments. It would be great, if you could trigger the CI.

@naman-modi
naman-modi force-pushed the refactor/unparser-agg-scope branch 2 times, most recently from e4c275d to d9e58ed Compare July 24, 2026 20:22
@naman-modi
naman-modi force-pushed the refactor/unparser-agg-scope branch from d9e58ed to 38cf748 Compare July 25, 2026 04:09
@kosiew
kosiew added this pull request to the merge queue Jul 25, 2026
Merged via the queue into apache:main with commit abec311 Jul 25, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Centralize aggregate-scope expression rendering in the SQL unparser

3 participants