Inline with-suppress return blocks and extend return duplication#7622
Inline with-suppress return blocks and extend return duplication#7622youknowone merged 4 commits intoRustPython:mainfrom
Conversation
- Add inline_with_suppress_return_blocks pass to inline return epilogues after with-suppress cleanup sequences - Extend duplicate_end_returns to handle conditional jumps to the final return block, not just unconditional ones - Process jump targets in reverse order to preserve indices - Add extra deoptimize_store_fast_store_fast pass after superinstructions - Add tests for listcomp cleanup tail and with-suppress tail
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughDeferred early collection-literal folding to later IR passes; rewrote t-string lowering to CPython ordering; tightened annotation and generic-class bookkeeping; added metadata-aware IR return duplication and new cleanup passes; refined jump/NOP/threading behavior; expanded constant folding; consolidated VM mapping lookup. Changes
Sequence Diagram(s)(omitted) Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
- Remove unnecessary NOP between FOR_ITER and unpack/store by compiling loop target directly on target range - Fix t-string compilation to match stack order: build strings tuple first, then evaluate interpolations - Split compile_tstring_into into collect_tstring_strings and compile_tstring_interpolations - Handle debug text literals and default repr conversion for debug specifier in t-strings - Always set bit 1 in BUILD_INTERPOLATION oparg encoding
553b04a to
e21cabc
Compare
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: (module 'compile test_peepholer test_tstring test_typing' not found) Legend:
|
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (3)
crates/codegen/src/ir.rs (3)
2397-2431:prev_real/next_realwalk includes NOPs.
find_map(|info| info.instr.real())returns the firstreal()hit, butInstruction::Nopis aRealvariant — so the "previous real instruction" can be a NOP and theForIterguard silently fails if any NOP remains betweenForIterand theStoreFast/LoadFastpair. This happens to be safe here becauseinsert_superinstructionsis always invoked afterremove_nops/remove_redundant_nops_and_jumps, but the current expression does not encode that invariant.Suggest filtering NOPs explicitly for clarity and to be robust to future pipeline reordering:
♻️ Suggested refactor
- let prev_real = block.instructions[..i] - .iter() - .rev() - .find_map(|info| info.instr.real()); - let next_real = block.instructions[(j + 1)..] - .iter() - .find_map(|info| info.instr.real()); + let prev_real = block.instructions[..i] + .iter() + .rev() + .find_map(|info| info.instr.real()) + .filter(|i| !matches!(i, Instruction::Nop)); + let next_real = block.instructions[(j + 1)..] + .iter() + .find_map(|info| info.instr.real()) + .filter(|i| !matches!(i, Instruction::Nop));🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/ir.rs` around lines 2397 - 2431, The current prev_real/next_real lookup uses find_map(|info| info.instr.real()) which treats Instruction::Nop as a "real" instruction and can make the ForIter guard misfire; modify the searches around block.instructions[..i].iter().rev() and block.instructions[(j + 1)..].iter() to explicitly skip Nop results (e.g. use find_map(|info| match info.instr.real() { Some(Instruction::Nop) => None, other => other })) so prev_real and next_real never return Nop and the Generator/ForIter checks behave correctly.
5554-5616:duplicate_named_except_cleanup_returns: inner-loop scan is O(n²).The inner
for block_idx in 0..blocks.len()runs for every named-except cleanuptarget, andfind_layout_predecessoritself is O(n). For codebases with manytry/except ... as name:patterns this can become a hot spot during compilation. Not blocking, but consider precomputing layout predecessors and a map of trailing-conditional-jump targets if compile-time regressions are observed.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/ir.rs` around lines 5554 - 5616, duplicate_named_except_cleanup_returns currently does an O(n²) scan by calling find_layout_predecessor and iterating all blocks per target; instead precompute data once: compute layout_predecessor for every block (cache results of find_layout_predecessor), and build a map from target BlockIdx to a Vec of (block_idx, instr_idx) by scanning each block once with trailing_conditional_jump_index and next_nonempty_block to record which blocks jump to which target; then iterate only the targets that are named-except-cleanup and use the precomputed predecessors, layout_predecessor and the jump-map to populate clones, avoiding repeated full scans of blocks. Ensure you reuse the existing predecessors vector and update references to find_layout_predecessor, trailing_conditional_jump_index, next_nonempty_block, clones, and blocks accordingly.
1201-1237: Constant folding misses(negative int) * str/bytes.
n.try_into().ok()?on a negativeBigIntfails and aborts folding, but Python defines-2 * "ab" == ""and-2 * b"ab" == b""(any non-positive multiplier yields an empty sequence). Same for the existingStr * Integerbranch at line 1189. Minor missed optimization, not a correctness issue.♻️ Suggested refactor
- (ConstantData::Integer { value: n }, ConstantData::Str { value: s }) - if matches!(op, BinOp::Multiply) => - { - let n: usize = n.try_into().ok()?; - if n > 4096 { - return None; - } - let result = s.to_string().repeat(n); - Some(ConstantData::Str { - value: result.into(), - }) - } + (ConstantData::Integer { value: n }, ConstantData::Str { value: s }) + if matches!(op, BinOp::Multiply) => + { + let n: usize = n.try_into().ok().unwrap_or(0); + if n > 4096 { + return None; + } + let result = s.to_string().repeat(n); + Some(ConstantData::Str { + value: result.into(), + }) + }(and similar for the Bytes branches)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/ir.rs` around lines 1201 - 1237, The integer multiplier branches for ConstantData::Str/Bytes currently call n.try_into().ok()? which aborts folding for negative BigInt multipliers; change each BinOp::Multiply branch handling (the patterns with ConstantData::Integer + ConstantData::Str and both Bytes+Integer / Integer+Bytes and the existing Str*Integer branch) to first check if the BigInt n is <= 0 and in that case return an empty ConstantData::Str or ConstantData::Bytes respectively, otherwise convert n to usize (try_into) and proceed with the existing 4096 cap and repeat logic; reference the ConstantData::Integer, ConstantData::Str, ConstantData::Bytes patterns and BinOp::Multiply in ir.rs to locate and update the branches.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/codegen/src/compile.rs`:
- Around line 2132-2140: The helper scope_needs_conditional_annotations_cell
currently returns false for CompilerScope::Class unless
has_conditional_annotations is true, but symboltable.rs registers the
"__conditional_annotations__" symbol for class scopes even when
has_conditional_annotations is false; update
scope_needs_conditional_annotations_cell (and the Class arm) to also return true
when the SymbolTable already contains the "__conditional_annotations__" symbol
(e.g., check whatever lookup/contains API exists on SymbolTable for
"__conditional_annotations__"), so class scopes that registered that symbol
still create the implicit cell and prologue.
- Around line 9928-9942: The current ConstantData::Str fold converts the Wtf8Buf
via to_string() which can replace surrogates with REPLACEMENT_CHARACTER; update
the match arm in compile.rs so you first call value.to_string(), check if it
contains the REPLACEMENT_CHARACTER constant (as done in compile_string_value /
compile_fstring_literal_value / compile_fstring_part_literal_value) and if so
return Ok(None) to skip the fold; only proceed to collect chars, compute idx,
and build the folded ConstantData::Str when no replacement characters are
present.
- Around line 10839-10843: The t-string literal loop in the tstring.elements
handling currently appends lit.value directly, which loses escaped surrogate
(WTF-8) data; change the handling to reparse the literal like the f-string path:
use the same approach as compile_fstring_literal_value(...) plus push_wtf8(...)
to append the reparsed bytes instead of push_str(&lit.value). Because
compile_fstring_literal_value takes ast::FStringFlags while tstring.flags is
ast::TStringFlags, either add a parallel helper (e.g.,
compile_tstring_literal_value or a helper that accepts TStringFlags) or provide
a safe conversion/adaptor from TStringFlags to FStringFlags/AnyStringFlags, then
call that helper inside the loop over tstring.elements and replace
current_string.push_str(&lit.value) with pushing the returned WTF-8 bytes via
push_wtf8 on the same buffer used for f-strings.
In `@crates/codegen/src/symboltable.rs`:
- Around line 1295-1312: The guard that skips registration when
tables.last().and_then(|table|
table.lookup("__conditional_annotations__")).is_none() prevents adding the
bookkeeping flags for existing symbols; instead always call register_name to
merge ASSIGNED/REFERENCED flags so future-annotation bookkeeping is preserved.
Update the block guarded by needs_module_future_annotation_bookkeeping to
unconditionally invoke self.register_name("__conditional_annotations__",
SymbolUsage::Assigned, annotation.range()) and
self.register_name("__conditional_annotations__", SymbolUsage::Used,
annotation.range()) (register_name is idempotent and merges flags), referencing
the needs_module_future_annotation_bookkeeping check,
tables.last().lookup("__conditional_annotations__"), and the register_name
method to locate the change.
---
Nitpick comments:
In `@crates/codegen/src/ir.rs`:
- Around line 2397-2431: The current prev_real/next_real lookup uses
find_map(|info| info.instr.real()) which treats Instruction::Nop as a "real"
instruction and can make the ForIter guard misfire; modify the searches around
block.instructions[..i].iter().rev() and block.instructions[(j + 1)..].iter() to
explicitly skip Nop results (e.g. use find_map(|info| match info.instr.real() {
Some(Instruction::Nop) => None, other => other })) so prev_real and next_real
never return Nop and the Generator/ForIter checks behave correctly.
- Around line 5554-5616: duplicate_named_except_cleanup_returns currently does
an O(n²) scan by calling find_layout_predecessor and iterating all blocks per
target; instead precompute data once: compute layout_predecessor for every block
(cache results of find_layout_predecessor), and build a map from target BlockIdx
to a Vec of (block_idx, instr_idx) by scanning each block once with
trailing_conditional_jump_index and next_nonempty_block to record which blocks
jump to which target; then iterate only the targets that are
named-except-cleanup and use the precomputed predecessors, layout_predecessor
and the jump-map to populate clones, avoiding repeated full scans of blocks.
Ensure you reuse the existing predecessors vector and update references to
find_layout_predecessor, trailing_conditional_jump_index, next_nonempty_block,
clones, and blocks accordingly.
- Around line 1201-1237: The integer multiplier branches for
ConstantData::Str/Bytes currently call n.try_into().ok()? which aborts folding
for negative BigInt multipliers; change each BinOp::Multiply branch handling
(the patterns with ConstantData::Integer + ConstantData::Str and both
Bytes+Integer / Integer+Bytes and the existing Str*Integer branch) to first
check if the BigInt n is <= 0 and in that case return an empty ConstantData::Str
or ConstantData::Bytes respectively, otherwise convert n to usize (try_into) and
proceed with the existing 4096 cap and repeat logic; reference the
ConstantData::Integer, ConstantData::Str, ConstantData::Bytes patterns and
BinOp::Multiply in ir.rs to locate and update the branches.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: e08cb85e-c3d4-431d-8ff7-8e9a28a71e14
⛔ Files ignored due to path filters (4)
Lib/test/test_compile.pyis excluded by!Lib/**Lib/test/test_peepholer.pyis excluded by!Lib/**Lib/test/test_tstring.pyis excluded by!Lib/**Lib/test/test_typing.pyis excluded by!Lib/**
📒 Files selected for processing (4)
crates/codegen/src/compile.rscrates/codegen/src/ir.rscrates/codegen/src/symboltable.rscrates/vm/src/frame.rs
Constant folding: - Add string/bytes multiply and bytes concat folding in IR - Add constant subscript folding (str, bytes, tuple indexing) - Delegate list/set constant folding to IR passes - Stream big non-const list/set via BUILD+LIST_APPEND Class/generic compilation: - Reorder class body prologue: __type_params__ before __classdict__ - Build class function before .generic_base in generic classes - Register .type_params/.generic_base symbols in proper scopes - Use load_name/store_name helpers for synthetic variables Return block handling: - Only duplicate return-None epilogues, not arbitrary returns - Add inline_pop_except_return_blocks pass - Add duplicate_named_except_cleanup_returns pass Other fixes: - Fix eliminate_dead_stores to only collapse adjacent duplicates - Skip STORE_FAST_LOAD_FAST superinstruction in generators after FOR_ITER - Thread jumps through NOP-only blocks - Transfer NOP line info to following unconditional jumps - Extract scope_needs_conditional_annotations_cell helper - Register __conditional_annotations__ for module future annotations
e21cabc to
caf8d55
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
crates/codegen/src/symboltable.rs (1)
1269-1309: Deduplicate the__conditional_annotations__registration.The future and non-future paths now share the same two
register_namecalls; compute the condition first and perform the common registration once.♻️ Proposed refactor
let current_scope = self.tables.last().map(|t| t.typ); + let is_annotation_bookkeeping_scope = matches!( + current_scope, + Some(CompilerScope::Module | CompilerScope::Class) + ); let needs_future_annotation_bookkeeping = is_ann_assign && self.future_annotations - && matches!( - current_scope, - Some(CompilerScope::Module | CompilerScope::Class) - ); + && is_annotation_bookkeeping_scope; + let mut needs_conditional_annotation_bookkeeping = + needs_future_annotation_bookkeeping; // PEP 649: Only AnnAssign annotations can be conditional. // Function parameter/return annotations are never conditional. if is_ann_assign && !self.future_annotations { let is_conditional = matches!(current_scope, Some(CompilerScope::Module)) || (matches!(current_scope, Some(CompilerScope::Class)) && self.in_conditional_block); if is_conditional && !self.tables.last().unwrap().has_conditional_annotations { self.tables.last_mut().unwrap().has_conditional_annotations = true; - self.register_name( - "__conditional_annotations__", - SymbolUsage::Assigned, - annotation.range(), - )?; - self.register_name( - "__conditional_annotations__", - SymbolUsage::Used, - annotation.range(), - )?; + needs_conditional_annotation_bookkeeping = true; } } - if needs_future_annotation_bookkeeping { + if needs_conditional_annotation_bookkeeping { self.register_name( "__conditional_annotations__", SymbolUsage::Assigned, annotation.range(),As per coding guidelines, “When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code.”
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/codegen/src/symboltable.rs` around lines 1269 - 1309, The two branches duplicate the same register_name calls for "__conditional_annotations__"; compute a single boolean (e.g., should_register_conditional_annotations) by combining the existing conditions (is_ann_assign && self.future_annotations && matches!(current_scope, Some(CompilerScope::Module|CompilerScope::Class)) OR the non-future path: is_ann_assign && !self.future_annotations && (matches!(current_scope, Some(CompilerScope::Module)) || (matches!(current_scope, Some(CompilerScope::Class)) && self.in_conditional_block)) and also ensure the table flag (self.tables.last().unwrap().has_conditional_annotations) is respected for the non-future path; then if should_register_conditional_annotations is true, set the table flag as needed and call the two register_name("__conditional_annotations__", SymbolUsage::Assigned, annotation.range()) and register_name("__conditional_annotations__", SymbolUsage::Used, annotation.range()) once.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@crates/codegen/src/symboltable.rs`:
- Around line 1269-1309: The two branches duplicate the same register_name calls
for "__conditional_annotations__"; compute a single boolean (e.g.,
should_register_conditional_annotations) by combining the existing conditions
(is_ann_assign && self.future_annotations && matches!(current_scope,
Some(CompilerScope::Module|CompilerScope::Class)) OR the non-future path:
is_ann_assign && !self.future_annotations && (matches!(current_scope,
Some(CompilerScope::Module)) || (matches!(current_scope,
Some(CompilerScope::Class)) && self.in_conditional_block)) and also ensure the
table flag (self.tables.last().unwrap().has_conditional_annotations) is
respected for the non-future path; then if
should_register_conditional_annotations is true, set the table flag as needed
and call the two register_name("__conditional_annotations__",
SymbolUsage::Assigned, annotation.range()) and
register_name("__conditional_annotations__", SymbolUsage::Used,
annotation.range()) once.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 77ed5473-a8a3-4c25-a56e-324539041d69
⛔ Files ignored due to path filters (4)
Lib/test/test_compile.pyis excluded by!Lib/**Lib/test/test_peepholer.pyis excluded by!Lib/**Lib/test/test_tstring.pyis excluded by!Lib/**Lib/test/test_typing.pyis excluded by!Lib/**
📒 Files selected for processing (5)
crates/codegen/src/compile.rscrates/codegen/src/ir.rscrates/codegen/src/symboltable.rscrates/vm/src/builtins/dict.rscrates/vm/src/frame.rs
✅ Files skipped from review due to trivial changes (2)
- crates/vm/src/builtins/dict.rs
- crates/codegen/src/ir.rs
🚧 Files skipped from review as they are similar to previous changes (1)
- crates/vm/src/frame.rs
dda6ff2 to
a5b9f0e
Compare
Summary by CodeRabbit
Bug Fixes
Performance
Features
Tests