gh-152685: Apply PEP 479 to StopIteration thrown into a not-yet-started generator#154492
Open
Joel-Wwalker wants to merge 1 commit into
Open
gh-152685: Apply PEP 479 to StopIteration thrown into a not-yet-started generator#154492Joel-Wwalker wants to merge 1 commit into
Joel-Wwalker wants to merge 1 commit into
Conversation
…-started generator The PEP 479 bytecode handler's exception table covers the generator body but not the prologue. A generator created by RETURN_GENERATOR resumes at the following POP_TOP, so a throw into it faults at instr_ptr - 1, the RETURN_GENERATOR slot, before the handler's range: the StopIteration escaped unchanged (and tripped an assert in debug builds). Convert at the point the exception would otherwise escape: when a generator frame propagates a StopIteration (or StopAsyncIteration for an async generator), gen_send_ex2 raises RuntimeError with the original chained, matching the in-body handler. Only reachable via throw into a not-yet-started generator; a started generator's own handler converts first, and gen_close never enters an unstarted frame.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gen.throw(StopIteration)on a generator that hasn't started yet lets the StopIteration escape unchanged instead of converting it to RuntimeError. A debug build trips the!PyErr_ExceptionMatches(PyExc_StopIteration)assert ingen_send_ex2; a release build silently violates PEP 479 (the issue shows a transaction manager committing where it should roll back).Root cause: PEP 479 is implemented by a bytecode handler whose exception table covers the generator body but not the prologue. A generator created by
RETURN_GENERATORresumes at the followingPOP_TOP, so a throw into it faults atinstr_ptr - 1, theRETURN_GENERATORslot, before the handler's range. Extending the table doesn't work cleanly: the handler records one unwind depth, which differs between the prologue and the body (and between generator functions and generator expressions), and coveringRETURN_GENERATORwould also cover it during the call that creates the generator.Instead, convert at the point the exception would otherwise escape. When a generator frame propagates a StopIteration (or, for an async generator, a StopAsyncIteration),
gen_send_ex2raises RuntimeError with the original chained as cause and context, matching the in-body handler. This path is only reachable via the throw-into-unstarted case: a started generator's own handler converts first, andgen_closenever enters an unstarted frame. Tests cover generators, generator expressions, coroutines, and async generators, and check that a non-StopIteration throw still propagates unchanged.Fixes #152685