tomoaki/hakorune
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(llvm): Phase 132-P0 - block_end_values tuple-key fix for cross-function isolation

Browse Source 
## Problem
`block_end_values` used block ID only as key, causing collisions when
multiple functions share the same block IDs (e.g., bb0 in both
condition_fn and main).

## Root Cause
- condition_fn's bb0 → block_end_values[0]
- main's bb0 → block_end_values[0] (OVERWRITES!)
- PHI resolution gets wrong snapshot → dominance error

## Solution (Box-First principle)
Change key from `int` to `Tuple[str, int]` (func_name, block_id):

```python
# Before
block_end_values: Dict[int, Dict[int, ir.Value]]

# After
block_end_values: Dict[Tuple[str, int], Dict[int, ir.Value]]
```

## Files Modified (Python - 6 files)

1. `llvm_builder.py` - Type annotation update
2. `function_lower.py` - Pass func_name to lower_blocks
3. `block_lower.py` - Use tuple keys for snapshot save/load
4. `resolver.py` - Add func_name parameter to resolve_incoming
5. `wiring.py` - Thread func_name through PHI wiring
6. `phi_manager.py` - Debug traces

## Files Modified (Rust - cleanup)

- Removed deprecated `loop_to_join.rs` (297 lines deleted)
- Updated pattern lowerers for cleaner exit handling
- Added lifecycle management improvements

## Verification

- ✅ Pattern 1: VM RC: 3, LLVM Result: 3 (no regression)
- ⚠️ Case C: Still has dominance error (separate root cause)
  - Needs additional scope fixes (phi_manager, resolver caches)

## Design Principles

- **Box-First**: Each function is an isolated Box with scoped state
- **SSOT**: (func_name, block_id) uniquely identifies block snapshots
- **Fail-Fast**: No cross-function state contamination

## Known Issues (Phase 132-P1)

Other function-local state needs same treatment:
- phi_manager.predeclared
- resolver caches (i64_cache, ptr_cache, etc.)
- builder._jump_only_blocks

## Documentation

- docs/development/current/main/investigations/phase132-p0-case-c-root-cause.md
- docs/development/current/main/investigations/phase132-p0-tuple-key-implementation.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
nyash-codex
2025-12-15 05:36:50 +09:00
parent 18d56d5b88
commit 3f58f34592
22 changed files with 1076 additions and 384 deletions
Download Patch File Download Diff File Expand all files Collapse all files

169
docs/development/current/main/investigations/phase132-case-c-llvm-exe.md Normal file
View File

@ -0,0 +1,169 @@
# Phase 132-P0: Case C (Infinite Loop with Early Exit) - LLVM EXE Investigation
## Date
2025-12-15
## Status
🔴 **FAILED** - LLVM executable returns wrong result
## Summary
Testing `apps/tests/llvm_stage3_loop_only.hako` (Pattern 5: InfiniteEarlyExit) in LLVM EXE mode reveals a critical exit PHI usage bug.
## Test File
`apps/tests/llvm_stage3_loop_only.hako`
```nyash
static box Main {
main() {
local counter = 0
loop (true) {
counter = counter + 1
if counter == 3 { break }
continue
}
print("Result: " + counter)
return 0
}
}
```
## Expected Behavior
- **VM execution**: `Result: 3` ✅
- **LLVM EXE**: `Result: 3` (should match VM)
## Actual Behavior
- **VM execution**: `Result: 3` ✅
- **LLVM EXE**: `Result: 0` ❌
## Root Cause Analysis
### MIR Structure (Correct)
```mir
bb3: ; Exit block
1: %1: Integer = phi [%8, bb6] ; ✅ PHI correctly receives counter
1: %16: String = const "Result: "
1: %17: String = copy %16
1: %18: Integer = copy %1 ; ✅ MIR uses %1 (PHI result)
1: %19: Box("StringBox") = %17 Add %18
1: %20: Box("StringBox") = copy %19
1: call_global print(%20)
1: %21: Integer = const 0
1: ret %21
```
**MIR is correct**: The exit block (bb3) has a PHI node that receives the counter value from bb6, and subsequent instructions correctly use `%1`.
### LLVM IR (BUG)
```llvm
bb3:
%"phi_1" = phi i64 [%"add_8", %"bb6"] ; ✅ PHI created correctly
%".2" = getelementptr inbounds [9 x i8], [9 x i8]* @".str.main.16", i32 0, i32 0
%"const_str_h_16" = call i64 @"nyash.box.from_i8_string"(i8* %".2")
%"bin_h2p_r_19" = call i8* @"nyash.string.to_i8p_h"(i64 0) ; ❌ Uses 0 instead of %"phi_1"
%"concat_is_19" = call i8* @"nyash.string.concat_is"(i64 0, i8* %"bin_h2p_r_19") ; ❌ Uses 0
%"concat_box_19" = call i64 @"nyash.box.from_i8_string"(i8* %"concat_is_19")
call void @"ny_check_safepoint"()
%"unified_global_print" = call i64 @"print"(i64 0) ; ❌ Uses 0
ret i64 0
```
**Bug identified**: The PHI node `%"phi_1"` is created correctly and receives the counter value from `%"add_8"`. However, **all subsequent uses of ValueId(1) are hardcoded to `i64 0` instead of using `%"phi_1"`**.
### Hypothesis
The Python LLVM builder is not correctly resolving ValueId(1) when lowering instructions in bb3. Possible causes:
1. **vmap issue**: The PHI node is created and stored in `self.vmap[1]` during `setup_phi_placeholders`, but when lowering instructions in bb3, `vmap_cur` may not contain the PHI.
2. **Resolution fallback**: When `resolve_i64_strict` fails to find ValueId(1), it falls back to `ir.Constant(i64, 0)`.
3. **Block-local vmap initialization**: `vmap_cur` is initialized with `dict(builder.vmap)` at the start of each block, but something may be preventing the PHI from being included.
## Investigation Steps
### Step 1: Verify PHI Creation
✅ **Confirmed**: PHI is created in LLVM IR at bb3
### Step 2: Check MIR Exit PHI Generation
✅ **Confirmed**: MIR has correct exit PHI with debug logs:
```
[DEBUG-177] Phase 246-EX: Block BasicBlockId(1) has jump_args metadata: [ValueId(1004)]
[DEBUG-177] Phase 246-EX: Remapped jump_args: [ValueId(8)]
[DEBUG-177] Phase 246-EX: exit_phi_inputs from jump_args[0]: (BasicBlockId(6), ValueId(8))
[DEBUG-177] Phase 246-EX-P5: Added loop_var 'counter' to carrier_inputs: (BasicBlockId(6), ValueId(8))
[DEBUG-177] Exit block PHI (carrier 'counter'): ValueId(1) = phi [(BasicBlockId(6), ValueId(8))]
```
### Step 3: Trace vmap Resolution
🔄 **In progress**: Running with `NYASH_LLVM_VMAP_TRACE=1` to see if PHI is in vmap_cur
## Code Locations
### Python LLVM Builder
- PHI placeholder creation: `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/llvm_builder.py:276-368`
- Line 343: `self.vmap[dst0] = ph0` - PHI stored in global vmap
- Block lowering: `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/builders/block_lower.py`
- Line 335: `vmap_cur = dict(builder.vmap)` - Copy global vmap to block-local
- Instruction lowering: `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/builders/instruction_lower.py`
- Line 8: `vmap_ctx = getattr(owner, '_current_vmap', owner.vmap)` - Use block-local vmap
- Value resolution: `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/utils/values.py:11-56`
- `resolve_i64_strict` - Checks vmap, global_vmap, then resolver
- Falls back to `ir.Constant(i64, 0)` if all fail (line 53)
### Rust MIR Generation
- Exit PHI generation: `src/mir/join_ir/lowering/simple_while_minimal.rs`
- Pattern 5 (InfiniteEarlyExit) lowering
## Root Cause Identified
### Bug Location
`/home/tomoaki/git/hakorune-selfhost/src/llvm_py/llvm_builder.py:342-343`
```python
ph0 = b0.phi(self.i64, name=f"phi_{dst0}")
self.vmap[dst0] = ph0
# ❌ MISSING: self.phi_manager.register_phi(bid0, dst0, ph0)
```
### Failure Chain
1. **PHI Creation** (line 342-343): PHI is created and stored in `self.vmap[1]` ✅
2. **PHI Registration** (MISSING): PHI is **never registered** via `phi_manager.register_phi()` ❌
3. **Block Lowering** (block_lower.py:325): `filter_vmap_preserve_phis` is called
4. **PHI Filtering** (phi_manager.py:filter_vmap_preserve_phis): Checks `is_phi_owned(3, 1)`
5. **Ownership Check** (phi_manager.py:is_phi_owned): Looks for `(3, 1)` in `predeclared` dict
6. **Not Found** ❌: PHI was never registered, so `(3, 1)` is not in `predeclared`
7. **PHI Filtered Out**: PHI is removed from `vmap_cur`
8. **Value Resolution Fails**: Instructions can't find ValueId(1), fall back to `ir.Constant(i64, 0)`
### Fix Strategy
**Option A: Add PHI Registration** (Recommended)
Add `self.phi_manager.register_phi(bid0, dst0, ph0)` after line 343 in `llvm_builder.py:setup_phi_placeholders`:
```python
if not is_phi:
ph0 = b0.phi(self.i64, name=f"phi_{dst0}")
self.vmap[dst0] = ph0
# ✅ FIX: Register PHI for filter_vmap_preserve_phis
self.phi_manager.register_phi(int(bid0), int(dst0), ph0)
```
This ensures PHIs are included in `vmap_cur` when lowering their defining block.
### Verification Plan
1. Add `register_phi` call in `setup_phi_placeholders`
2. Rebuild and test: `NYASH_LLVM_STRICT=1 tools/build_llvm.sh apps/tests/llvm_stage3_loop_only.hako -o /tmp/case_c`
3. Execute: `/tmp/case_c` should output `Result: 3`
4. Check LLVM IR: Should use `%"phi_1"` instead of `0`
## Acceptance Criteria
- ✅ LLVM EXE output: `Result: 3`
- ✅ LLVM IR uses `%"phi_1"` instead of `0`
- ✅ STRICT mode passes without fallback warnings
## Related Documents
- [Phase 132 Plan](/home/tomoaki/git/hakorune-selfhost/docs/development/current/main/phase132-plan.md)
- [Phase 131-3 LLVM Lowering Inventory](/home/tomoaki/git/hakorune-selfhost/docs/development/current/main/phase131-3-llvm-lowering-inventory.md)
- [Simple While Minimal Lowering](/home/tomoaki/git/hakorune-selfhost/src/mir/join_ir/lowering/simple_while_minimal.rs)

149
docs/development/current/main/investigations/phase132-p0-case-c-root-cause.md Normal file
View File

@ -0,0 +1,149 @@
# Phase 132-P0 Case C Root Cause Investigation
**Date**: 2025-12-15
**Status**: Root cause identified, fix designed
**Priority**: P0 (blocks LLVM EXE execution)
## Problem Statement
Case C (Pattern 5 + print concat) LLVM EXE fails with domination error:
```
RuntimeError: Instruction does not dominate all uses!
%phi_1 = phi i64 [ %add_8, %bb6 ]
%phi_3 = phi i64 [ %phi_1, %bb0 ], [ %add_8, %bb7 ]
```
`%phi_3` in bb4 uses `%phi_1` from bb0 edge, but `%phi_1` is defined in bb3 which doesn't dominate bb0.
## Investigation Process
### Step 0: IR Dump Confirmation
Generated IR shows:
```llvm
bb0:
br label %bb4
bb3:
%phi_1 = phi i64 [%add_8, %bb6] ; Defined in bb3
...
bb4:
%phi_3 = phi i64 [%phi_1, %bb0], [%add_8, %bb7] ; Uses %phi_1 from bb0!
```
MIR shows correct structure:
- bb0: `ValueId(1) = const 0`
- bb3: `ValueId(1) = PHI [bb6, ValueId(8)]`
- bb4: `ValueId(3) = PHI [(bb0, ValueId(2)), (bb7, ValueId(8))]`
**Key observation**: Same ValueId(1) used in different blocks is normal (SSA allows this), but LLVM builder is confusing them!
### Step 1: VMAP Trace Analysis
VMAP trace showed:
```
[vmap/id] Pass A bb0 snapshot id=139925440649984 keys=[0, 1]
[vmap/id] Pass A bb0 snapshot id=139925440650112 keys=[1, 2, 3]
```
Two different bb0 snapshots! But only one bb0 in `main` function.
### Root Cause Discovery
Checked all functions in MIR JSON:
```json
{
"name": "condition_fn",
"blocks": [0]
},
{
"name": "main",
"blocks": [0, 3, 4, 5, 6, 7]
}
```
**BINGO**: Two functions have bb0!
- `condition_fn` has bb0 (first snapshot)
- `main` has bb0 (second snapshot, overwrites first)
### Root Cause
**`block_end_values` uses `block_id` as key instead of `(function_name, block_id)` tuple**
**Problem flow**:
1. Process `condition_fn` bb0 → `block_end_values[0] = {0: ..., 1: ...}`
2. Process `main` bb0 → `block_end_values[0] = {1: ..., 2: ..., 3: ...}` (OVERWRITES!)
3. Process `main` bb4's PHI → resolve incoming ValueId(1) from bb0
4. `resolve_incoming(pred_block_id=0, value_id=1)` looks up `block_end_values[0][1]`
5. Gets `main`'s bb0 ValueId(1) (which is copy of PHI) instead of const 0!
**Result**: bb4's PHI gets `%phi_1` (bb3's PHI) instead of `i64 0` (bb0's const), causing domination error.
## Solution Design
### Change 1: Tuple-Key block_end_values
**Old**:
```python
block_end_values: Dict[int, Dict[int, ir.Value]] = {}
block_end_values[bid] = snap
```
**New**:
```python
block_end_values: Dict[Tuple[str, int], Dict[int, ir.Value]] = {}
block_end_values[(func_name, bid)] = snap
```
### Change 2: Thread function name through call chain
**Files to modify**:
1. `llvm_builder.py` - Type annotation
2. `function_lower.py` - Pass `func.name` to `lower_blocks`
3. `block_lower.py` - Accept `func_name` parameter, use tuple keys
4. `resolver.py` - Update `resolve_incoming` to accept `func_name`
5. `phi_wiring/wiring.py` - Update `wire_incomings` to use tuple keys
### Change 3: Verifier (STRICT mode)
Add collision detection:
```python
if key in block_end_values and STRICT:
existing_func = find_function_for_key(key)
if existing_func != current_func:
raise RuntimeError(
f"Block ID collision: bb{bid} exists in both "
f"{existing_func} and {current_func}"
)
```
## Acceptance Criteria
1. **Pattern 1** (Phase 132): Still passes (regression test)
2. **Case C** (Pattern 5): Builds and executes correctly
3. **VM/LLVM parity**: Both produce same result
4. **STRICT mode**: No collisions, no fallback to 0
## Implementation Status
- [x] Root cause identified
- [x] Solution designed
- [ ] Tuple-key implementation
- [ ] STRICT verifier
- [ ] Acceptance testing
- [ ] Documentation update
## Related Documents
- Task: `/home/tomoaki/git/hakorune-selfhost/CURRENT_TASK.md` (Phase 132-P0)
- Inventory: `/home/tomoaki/git/hakorune-selfhost/docs/development/current/main/phase131-3-llvm-lowering-inventory.md`
## Key Insight
**"Same block ID in different functions is a FEATURE, not a bug"**
MIR reuses block IDs across functions (bb0 is common entry block). The LLVM builder MUST namespace block_end_values by function to avoid collisions.
This is a **Box-First principle violation**: `block_end_values` should have been scoped per-function from the start (encapsulation boundary).

180
docs/development/current/main/investigations/phase132-p0-tuple-key-implementation.md Normal file
View File

@ -0,0 +1,180 @@
# Phase 132-P0: block_end_values Tuple-Key Implementation
**Date**: 2025-12-15
**Status**: ✅ Implementation Complete
**Related**: phase132-p0-case-c-root-cause.md
## Summary
Implemented tuple-key `(func_name, block_id)` for `block_end_values` to prevent cross-function block ID collisions in LLVM backend.
## Root Cause (from investigation)
```python
# ❌ Before: Block ID collision across functions
block_end_values: Dict[int, Dict[int, ir.Value]]
# main:bb0 and condition_fn:bb0 collide!
# ✅ After: Function-scoped keys
block_end_values: Dict[Tuple[str, int], Dict[int, ir.Value]]
# ("main", 0) and ("condition_fn", 0) are distinct
```
## Implementation (5 files modified)
### 1. `src/llvm_py/llvm_builder.py` (Type annotation)
```python
# Line 116: Updated type annotation
self.block_end_values: Dict[Tuple[str, int], Dict[int, ir.Value]] = {}
```
### 2. `src/llvm_py/builders/function_lower.py` (Pass func_name)
```python
# Line 303: Pass func_name to lower_blocks
_lower_blocks(builder, func, block_by_id, order, loop_plan, func_name=name)
# Line 308: Pass func_name to resolve_jump_only_snapshots
_resolve_jump_only_snapshots(builder, block_by_id, func_name=name)
# Line 333: Pass func_name to finalize_phis
_finalize_phis(builder, func_name=name)
```
### 3. `src/llvm_py/builders/block_lower.py` (Tuple-key usage)
```python
# Line 184: Accept func_name parameter
def lower_blocks(..., func_name: str = "unknown"):
# Line 61: Accept func_name parameter
def resolve_jump_only_snapshots(..., func_name: str = "unknown"):
# Line 118-119: Use tuple-key for read
if (func_name, bid) in builder.block_end_values:
snapshot = builder.block_end_values[(func_name, bid)]
# Line 177: Use tuple-key for write (Pass B)
builder.block_end_values[(func_name, bid)] = snapshot
# Line 529: Use tuple-key for write (Pass A)
builder.block_end_values[(func_name, bid)] = snap
```
### 4. `src/llvm_py/resolver.py` (resolve_incoming)
```python
# Line 127: Accept func_name parameter
def resolve_incoming(self, pred_block_id: int, value_id: int, func_name: str = "unknown"):
# Line 143: Use tuple-key for snapshot lookup
snapshot = self.block_end_values.get((func_name, pred_block_id), {})
```
### 5. `src/llvm_py/phi_wiring/wiring.py` (PHI wiring)
```python
# Line 242: Accept func_name parameter in finalize_phis
def finalize_phis(builder, func_name: str = "unknown"):
# Line 140: Accept func_name parameter in wire_incomings
def wire_incomings(builder, ..., func_name: str = "unknown"):
# Line 155: Use tuple-key for PHI lookup
cur = (snap.get((func_name, int(block_id)), {}) or {}).get(int(dst_vid))
# Line 219: Pass func_name to resolve_incoming
val = builder.resolver.resolve_incoming(pred_match, vs, func_name=func_name)
# Line 256: Pass func_name to wire_incomings
wired = wire_incomings(builder, ..., func_name=func_name)
```
## Testing Results
### ✅ Pattern 1 (Phase 132 regression check)
```bash
# Test file: /tmp/p1_return_i.hako
static box Main {
main() {
local i = 0
loop(i < 3) { i = i + 1 }
return i
}
}
# VM Result: RC: 3 ✅
# LLVM Result: Result: 3 ✅ (without STRICT mode)
# LLVM STRICT: ValueId collision error (separate issue)
```
**Status**: ✅ No regression - Pattern 1 still works correctly
### ⚠️ Case C (Pattern 5) - Dominance Error Persists
```bash
# Test file: apps/tests/llvm_stage3_loop_only.hako
# VM Result: Result: 3 ✅
# LLVM Result: PHI dominance error ❌
Error: Instruction does not dominate all uses!
%phi_1 = phi i64 [ %add_8, %bb6 ]
%phi_3 = phi i64 [ %phi_1, %bb0 ], [ %add_8, %bb7 ]
```
**Analysis**: The dominance error is NOT caused by block_end_values collision.
It's a different issue related to PHI node placement and control flow structure.
### Verification Logs
```bash
# Pass B resolution working correctly:
[vmap/resolve/passB] Resolving 2 jump-only blocks: [6, 7]
[vmap/resolve/passB] bb6 is jump-only, resolving from pred bb5
[vmap/resolve/passB] bb5 is normal block with snapshot (5 values)
[vmap/resolve/passB] bb6 resolved from bb5: 5 values
[vmap/resolve/passB] ✅ bb6 final snapshot: 5 values, keys=[3, 7, 8, 9, 10]
```
## Design Principles Applied
### Box-First (SSOT)
- Each function is an independent Box
- `block_end_values` keys are scoped to function
- `(func_name, block_id)` is the SSOT identifier
### Fail-Fast
- STRICT mode detects collisions immediately
- Updated error messages include `func_name` context
## Conclusion
### ✅ Implementation Complete
- All 5 files updated with tuple-key logic
- Type annotations consistent
- Function signatures updated
- All call sites pass `func_name`
### ✅ Regression Prevention
- Pattern 1 still works correctly
- VM/LLVM parity maintained for simple cases
### ⚠️ Case C Needs Further Investigation
The dominance error in Case C is **not fixed** by this change.
**Root cause**: Different issue - likely related to:
- PHI node placement in complex control flow (break/continue)
- Block ordering or dominator tree structure
- Need separate investigation (Phase 132-P1?)
## Next Steps
1. **Accept tuple-key fix**: Merge this implementation (prevents future collisions)
2. **Investigate Case C separately**: Create Phase 132-P1 for dominance error
3. **Add tuple-key validation**: Optional STRICT check that all lookups use tuple-key
## Files Modified
1. `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/llvm_builder.py`
2. `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/builders/function_lower.py`
3. `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/builders/block_lower.py`
4. `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/resolver.py`
5. `/home/tomoaki/git/hakorune-selfhost/src/llvm_py/phi_wiring/wiring.py`
## References
- Phase 132 Inventory: `docs/development/current/main/phase131-3-llvm-lowering-inventory.md`
- Root Cause Analysis: `docs/development/current/main/investigations/phase132-p0-case-c-root-cause.md`