Make :stopdoc:/:startdoc:/:enddoc: lexically scoped in the Ruby parser

[tompng]

Part of #1591. This PR only changes the region directives (:stopdoc:/:startdoc:/:enddoc:). :nodoc: semantics are unchanged and will be addressed separately.

Problem

These directives are currently implemented as mutations of shared CodeObject state (document_self/document_children/done_documenting). Since classes are open and shared across files in the store, a :stopdoc: without a matching :startdoc: leaks into later reopenings of the class in the same file and into other files (#398). The actual behavior is a product of mutation order, and neither the documentation nor the implementation defines what these directives mean.

New semantics

The parser now tracks the document control state (:startdoc:/:stopdoc:/:enddoc:) as a lexical region:

• The state is inherited when entering a class/module scope and automatically restored at its end. A :stopdoc: can no longer leak out of the scope it was written in.
• :startdoc: inside a nested scope re-enables documentation for that scope only.
• :enddoc: disables documentation for the rest of the current scope and cannot be cancelled within it, even by :startdoc:. It no longer marks the container object as done_documenting globally.
• Classes/modules defined inside a suppressed region are still created as namespaces (marked as ignored) and their bodies are visited, so an inner :startdoc: works. They become documentable when they receive documentable contents outside the region, possibly from another file. Namespaces that never receive contents are removed from the store on Store#complete (Context#remove_from_documentation? now also covers ignored contexts, which is what CodeObject#ignore was documented to do in GH :stopdoc: directive produces Object reference #55).
• Document control scopes are the syntactic units that change the definition context (class/module bodies, class << self, and definition-creating blocks). if/while/begin do not create scopes, consistent with Ruby's own scoping rules.

Compatibility notes

• :nodoc: is untouched. However, :startdoc: additionally calls start_doc on the current container, because module Net #:nodoc: followed by :stopdoc:/:startdoc: regions is a common pattern that expects :startdoc: to make the container documentable again. This is a transitional measure until :nodoc: gets lexical semantics, at which point the pattern itself becomes unnecessary. Containers created inside a suppressed region are excluded, so a bare :startdoc: cannot revive an empty hidden class.
• A comment linked to a class << X line is now processed in the enclosing scope. Nothing consumes such a comment as documentation, and processing it inside the singleton scope would make a :startdoc: written there expire at the singleton's end (this pattern appears in net/http).
• Region directives in modifier position (e.g. class Foo # :stopdoc: at the end of a line) are no longer applied. There are no occurrences of this form in ruby/ruby, rails, or the bundled gems.

Validation

Compared generated HTML for ruby/ruby lib and six Rails frameworks between master and this branch. The sets of generated files are identical, and every remaining difference is the directives working as their authors intended:

• Module aliases written inside :stopdoc: regions (Ripper = Prism::Translation::Ripper in prism's shim.rb, HashWithIndifferentAccess = ... in activesupport) are no longer documented. This also removes a duplicated "Ripper" entry from the navigation and stops bare "Ripper" mentions (which refer to the stdlib Ripper) from being cross-referenced to Prism::Translation::Ripper. Note: the old accidental link on HashWithIndifferentAccess in Hash.html was arguably useful; writing the full name in the comment restores it.
• Net::HTTPResponse regains its include Net::HTTPHeader. The old implementation lost it because a :stopdoc: inside class << self leaked out of the singleton scope; net/http.rb contains a "this is to fix bug in RDoc" workaround comment for this exact class of leak.
• ActionDispatch::Journey (# :nodoc:) no longer appears in the navigation. Its page is still generated because it has documented children.
• Namespaces fully contained in suppressed regions (e.g. Net::HTTPBadResponse region in net/http.rb, Timeout::Request) no longer produce empty pages.

Follow-ups (out of scope here)

• Lexical/shallow semantics for :nodoc: (the hard part of Lexical scope document control directives #1591); the start_doc compatibility shim and the received_nodoc guard in promotion can be removed then.
• Warnings for degenerate usages (redundant :stopdoc:, directives after :enddoc:, standalone :nodoc: in the middle of a class body).
• A visibility trace output (file:line: stopdoc applied to X) as proposed in Lexical scope document control directives #1591 (comment); the parser now has the lexical information to make this cheap.
• The C parser still uses the old CodeObject-state mechanism and is intentionally untouched.

[Copilot]

Pull request overview

This PR updates the Prism-based Ruby parser to make region directives (:stopdoc:, :startdoc:, :enddoc:) lexically scoped to the current syntactic container, preventing directive state from leaking across class/module reopenings and across files. It also aligns store cleanup behavior with the documented meaning of CodeObject#ignore and adds targeted regression tests for the new semantics.

Changes:

• Track document-control state in the Ruby parser (@doc_state) and restore it on scope exit (with_container) so region directives are lexical rather than mutating shared CodeObject state.
• Ensure namespaces created in suppressed regions are created as ignored placeholders, and promote them to documentable when they later receive documentable contents outside the suppressed region.
• Add extensive parser tests covering leakage prevention, nested overrides, singleton-class behavior, and interactions with :nodoc:; adjust Darkfish generator expectations for ignored contexts being removed on Store#complete.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
lib/rdoc/parser/ruby.rb	Implements lexically-scoped startdoc/stopdoc/enddoc via parser state, suppression checks, and namespace promotion logic.
lib/rdoc/code_object/context.rb	Extends remove_from_documentation? to also prune ignored contexts that never gain documentable contents.
test/rdoc/parser/ruby_test.rb	Adds regression tests to validate lexical scoping and non-leaking behavior across scopes, singleton classes, and files.
test/rdoc/generator/darkfish_test.rb	Updates expectations to reflect ignored contexts being removed during store completion.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[matzbot]

🚀 Preview deployment available at: https://a3bd5bd4.rdoc-6cd.pages.dev (commit: 5aec0e4)

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (2)

lib/rdoc/parser/ruby.rb:875

• add_module_or_class currently bails out when @container.document_children is false. Containers created in a :stopdoc:/:enddoc: region are marked ignored via #ignore, which also sets document_children = false, so nested class/module declarations inside such an ignored container will never be created/visited (preventing an inner :startdoc: from working inside a namespace created in a suppressed region). Allow parsing nested namespaces when the parent is ignored, while still respecting :nodoc: all (which sets document_children = false without ignored?).

    return unless @container.document_children

lib/rdoc/parser/ruby.rb:1263

• require calls don’t consume a linked preceding comment via consecutive_comment, so a directive like # :stopdoc: placed immediately before require 'x' won’t be applied until the end-of-file comment sweep (too late), and the require will be recorded even though documentation should be suppressed. Process the linked comment/directives at the require line before checking document_suppressed?.

    def _visit_call_require(call_node)
      return if @scanner.document_suppressed?
      return unless call_node.arguments&.arguments&.size == 1