Skip to content

Read stubgen documentation once per run - #22379

Open
den-run-ai wants to merge 1 commit into
python:masterfrom
den-run-ai:stubgen-doc-dir-cache
Open

den-run-ai wants to merge 1 commit into
python:masterfrom
den-run-ai:stubgen-doc-dir-cache

Conversation

@den-run-ai

Copy link
Copy Markdown

Fixes #22370.

Reuse the external signature index across C modules within one stubgen invocation. Keep loading lazy and scoped to the invocation so later runs pick up documentation changes.

With four C modules and 319 documentation files, file reads fall from 1,276 to 319. Benchmark details are in the issue.

Add a regression test covering reuse across modules and documentation changes between invocations.

Validation: 379 stubgen tests passed (1 skipped, 2 xfailed); serial self-check (196 files); Black and Ruff.

@agammann agammann left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the loader’s lifetime and call paths. The cached loader is created inside generate_stubs, so C modules in one run share the parsed documentation index while a later invocation reloads changed docs. Direct callers of generate_stub_for_c_module still use its uncached default. The two-module/two-invocation test covers both reuse and invalidation. I did not run the full suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read --doc-dir once when stubgen processes multiple C modules

2 participants