feat(parser): Phase 285A1.4 & A1.5 - Weak field sugar + Parser hang fix
A1.4: Add sugar syntax `public weak parent` ≡ `public { weak parent }`
A1.5: Fix parser hang on unsupported `param: Type` syntax
Key changes:
- A1.4: Extend visibility parser to handle weak modifier (fields.rs)
- A1.5: Shared helper `parse_param_name_list()` with progress-zero detection
- A1.5: Fix 6 vulnerable parameter parsing loops (methods, constructors, functions)
- Tests: Sugar syntax (OK/NG), parser hang (timeout-based)
- Docs: lifecycle.md, EBNF.md, phase-285a1-boxification.md
Additional changes:
- weak() builtin implementation (handlers/weak.rs)
- Leak tracking improvements (leak_tracker.rs)
- Documentation updates (lifecycle, types, memory-finalization, etc.)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
@ -0,0 +1,619 @@
|
||||
# LoopBuilder Removal Compatibility Analysis
|
||||
|
||||
**Date**: 2025-12-24
|
||||
**Investigator**: Claude Code
|
||||
**Test**: `core_direct_array_oob_set_rc_vm`
|
||||
**Error**: `[joinir/freeze] Loop lowering failed: JoinIR does not support this pattern, and LoopBuilder has been removed`
|
||||
**User Hypothesis**: "昔のjoinirが今のjoinirに対応してないだけじゃない?" (Old JoinIR code not compatible with current JoinIR?)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**User's hypothesis is CORRECT!** This is not a bug in the current code—it's a **feature gap** in JoinIR pattern coverage.
|
||||
|
||||
### Root Cause
|
||||
|
||||
1. **LoopBuilder was physically deleted** in Phase 187 (commit `fa8a96a51`, Dec 4, 2025)
|
||||
- Deleted: 8 files, ~1,758 lines of legacy loop lowering code
|
||||
- Preserved: Only `IfInLoopPhiEmitter` moved to minimal module
|
||||
|
||||
2. **BundleResolver.resolve/4 uses a loop pattern not yet supported by JoinIR**
|
||||
- The `.hako` code in `lang/src/compiler/entry/bundle_resolver.hako` was written when LoopBuilder still existed
|
||||
- This specific loop pattern hasn't been migrated to JoinIR patterns yet
|
||||
|
||||
3. **This is NOT a regression**—it's an **expected gap** documented in Phase 188
|
||||
- Phase 188 identified 5 failing loop patterns after LoopBuilder removal
|
||||
- The failing loop in `phase264_p0_bundle_resolver_loop_min.hako` matches the documented gap
|
||||
|
||||
### Current Status
|
||||
|
||||
- **LoopBuilder**: ❌ Completely removed (Dec 4, 2025, Phase 187)
|
||||
- **JoinIR Patterns Implemented**: Pattern 1, 2, 3 (Phase 188, Dec 5-6, 2025)
|
||||
- **Coverage**: ~80% of representative tests pass
|
||||
- **Gap**: BundleResolver loop falls into the 20% not yet covered
|
||||
|
||||
---
|
||||
|
||||
## Timeline: LoopBuilder Removal
|
||||
|
||||
### Phase 186: Hard Freeze (Dec 4, 2025)
|
||||
- Commit: `30f94c955`
|
||||
- Added access control guard to prevent LoopBuilder usage
|
||||
- Intent: Make LoopBuilder opt-in only with `NYASH_LEGACY_LOOPBUILDER=1`
|
||||
- **Gap Found**: Guard was added but instantiation point wasn't protected!
|
||||
|
||||
### Phase 187: Physical Removal (Dec 4, 2025)
|
||||
- Commit: `fa8a96a51`
|
||||
- **Deleted**:
|
||||
- `src/mir/loop_builder/mod.rs` (158 lines)
|
||||
- `src/mir/loop_builder/loop_form.rs` (578 lines)
|
||||
- `src/mir/loop_builder/if_lowering.rs` (298 lines)
|
||||
- `src/mir/loop_builder/phi_ops.rs` (333 lines)
|
||||
- `src/mir/loop_builder/control.rs` (94 lines)
|
||||
- `src/mir/loop_builder/statements.rs` (39 lines)
|
||||
- `src/mir/loop_builder/joinir_if_phi_selector.rs` (168 lines)
|
||||
- `src/mir/loop_builder/README.md` (15 lines)
|
||||
- **Total Deletion**: 1,758 lines
|
||||
- **Preserved**: Only `IfInLoopPhiEmitter` moved to `src/mir/phi_core/if_in_loop_phi_emitter/mod.rs`
|
||||
|
||||
### Phase 188: JoinIR Pattern Expansion (Dec 5-6, 2025)
|
||||
- Implemented 3 loop patterns to replace LoopBuilder:
|
||||
1. **Pattern 1**: Simple While Loop (`loop(i < 3) { i++ }`)
|
||||
2. **Pattern 2**: Loop with Conditional Break (`loop(true) { if(...) break }`)
|
||||
3. **Pattern 3**: Loop with If-Else PHI (`loop(...) { if(...) sum += x else sum += 0 }`)
|
||||
- **Coverage**: Estimated 80% of representative tests
|
||||
- **Documented Gaps**: 5 failing patterns identified in `inventory.md`
|
||||
|
||||
---
|
||||
|
||||
## What is LoopBuilder?
|
||||
|
||||
### Historical Context
|
||||
|
||||
**LoopBuilder** was the original loop lowering system (pre-Phase 186):
|
||||
- **Purpose**: Convert loop AST → MIR with SSA/PHI nodes
|
||||
- **Implementation**: 8 files, ~1,000+ lines
|
||||
- **Problems**:
|
||||
- Complex PHI handling with historical bugs
|
||||
- Mixed control flow responsibilities
|
||||
- Silent fallback behavior (hid pattern gaps)
|
||||
|
||||
### Why Was It Removed?
|
||||
|
||||
From Phase 187 documentation:
|
||||
|
||||
> **Reasons for Deletion**:
|
||||
> 1. **Isolated**: Single instantiation point made deletion safe
|
||||
> 2. **Legacy**: Original implementation had historical design issues
|
||||
> 3. **Proven**: Phase 181 confirmed 80% of patterns work with JoinIR only
|
||||
> 4. **Explicit Failure**: Removing fallback drives JoinIR improvement
|
||||
|
||||
### What Replaced It?
|
||||
|
||||
**JoinIR Frontend** (modern loop lowering):
|
||||
- **Pattern-based approach**: Each loop pattern has dedicated lowering logic
|
||||
- **Explicit failure**: Unsupported patterns fail with clear error messages
|
||||
- **Extensible**: New patterns can be added incrementally
|
||||
|
||||
---
|
||||
|
||||
## The Failing BundleResolver Loop
|
||||
|
||||
### Location
|
||||
|
||||
**Source**: `lang/src/compiler/entry/bundle_resolver.hako`
|
||||
**Function**: `BundleResolver.resolve/4`
|
||||
**Lines**: Multiple loops (25-45, 52-71, 76-87, 92-101, 106-112)
|
||||
|
||||
### Example Loop (lines 25-45)
|
||||
|
||||
```nyash
|
||||
// Alias table parsing loop
|
||||
local i = 0
|
||||
loop(i < table.length()) {
|
||||
// find next delimiter or end
|
||||
local j = table.indexOf("|||", i)
|
||||
local seg = ""
|
||||
if j >= 0 {
|
||||
seg = table.substring(i, j)
|
||||
} else {
|
||||
seg = table.substring(i, table.length())
|
||||
}
|
||||
|
||||
if seg != "" {
|
||||
local pos = -1
|
||||
local k = 0
|
||||
loop(k < seg.length()) {
|
||||
if seg.substring(k,k+1) == ":" {
|
||||
pos = k
|
||||
break
|
||||
}
|
||||
k = k + 1
|
||||
}
|
||||
// ... more processing
|
||||
}
|
||||
|
||||
if j < 0 { break }
|
||||
i = j + 3
|
||||
}
|
||||
```
|
||||
|
||||
### Why This Fails
|
||||
|
||||
**Pattern Characteristics**:
|
||||
- **Multiple carriers**: `i`, `j`, `seg`, `pos`, `k` (5+ variables)
|
||||
- **Nested loop**: Inner loop with `break`
|
||||
- **Conditional assignments**: `seg = ... if ... else ...`
|
||||
- **Non-unit increment**: `i = j + 3` (not `i++`)
|
||||
- **Complex control flow**: Multiple `if` statements with `break`
|
||||
|
||||
**Current JoinIR Pattern Coverage**:
|
||||
1. ❌ **Pattern 1** (Simple While): Too simple—only handles single carrier, no breaks
|
||||
2. ❌ **Pattern 2** (Conditional Break): Doesn't handle multiple carriers + nested loops
|
||||
3. ❌ **Pattern 3** (If-Else PHI): Only handles "if-sum" pattern (arithmetic accumulation)
|
||||
4. ❌ **Pattern 4** (Continue): Not applicable (no `continue` statements)
|
||||
|
||||
**Result**: No pattern matches → falls through → explicit error
|
||||
|
||||
---
|
||||
|
||||
## Phase 264: Recent Attempt to Fix Similar Issue
|
||||
|
||||
### Context
|
||||
|
||||
Phase 264 P0 documented a similar failure in `phase264_p0_bundle_resolver_loop_min.hako`:
|
||||
|
||||
```nyash
|
||||
local i = 0
|
||||
local seg = ""
|
||||
|
||||
loop(i < 10) {
|
||||
// Conditional assignment to seg
|
||||
if i == 0 {
|
||||
seg = "first"
|
||||
} else {
|
||||
seg = "other"
|
||||
}
|
||||
|
||||
// Non-unit increment
|
||||
i = i + 2
|
||||
}
|
||||
```
|
||||
|
||||
### Why It Failed
|
||||
|
||||
**Pattern routing flow**:
|
||||
1. **Pattern 8** (BoolPredicateScan): REJECT (condition right is not `.length()`)
|
||||
2. **Pattern 3** (WithIfPhi): MATCHED → but rejects "Not an if-sum pattern"
|
||||
3. **Pattern 1/2**: Not tried
|
||||
4. **Result**: No match → ERROR
|
||||
|
||||
### Root Cause (from Phase 264 analysis)
|
||||
|
||||
**Classification Heuristic Issue**:
|
||||
|
||||
```rust
|
||||
// src/mir/loop_pattern_detection/mod.rs:227-230
|
||||
// Pattern 3 heuristic: has_if_else_phi if carrier_count > 1
|
||||
let has_if_else_phi = carrier_count > 1;
|
||||
```
|
||||
|
||||
**Problem**:
|
||||
- This heuristic is **too conservative**
|
||||
- Any loop with 2+ carriers → classified as Pattern3IfPhi
|
||||
- But Pattern3 only handles **if-sum patterns** (arithmetic accumulation)
|
||||
- Simple conditional assignment → incorrectly routed to Pattern3 → rejected
|
||||
|
||||
### Proposed Fix (Phase 264)
|
||||
|
||||
**Option B** (adopted): Improve classification heuristic
|
||||
|
||||
```rust
|
||||
// Phase 264 P0: Improved if-else PHI detection
|
||||
// Pattern3 heuristic: has_if_else_phi if there's an if-sum pattern signature
|
||||
let has_if_else_phi = carrier_count > 1 && has_if_sum_signature(scope);
|
||||
```
|
||||
|
||||
**Impact**:
|
||||
- Simple conditional assignment → falls through to Pattern1
|
||||
- If-sum pattern → correctly routed to Pattern3
|
||||
|
||||
---
|
||||
|
||||
## Answer to Investigation Questions
|
||||
|
||||
### 1. Where is "LoopBuilder has been removed" error generated?
|
||||
|
||||
**Location**: `src/mir/builder/control_flow/mod.rs:136-145`
|
||||
|
||||
```rust
|
||||
// Phase 186: LoopBuilder Hard Freeze - Legacy path disabled
|
||||
// Phase 187-2: LoopBuilder module removed - all loops must use JoinIR
|
||||
use crate::mir::join_ir::lowering::error_tags;
|
||||
return Err(error_tags::freeze(&format!(
|
||||
"Loop lowering failed: JoinIR does not support this pattern, and LoopBuilder has been removed.\n\
|
||||
Function: {}\n\
|
||||
Hint: This loop pattern is not supported. All loops must use JoinIR lowering.",
|
||||
self.scope_ctx.current_function.as_ref().map(|f| f.signature.name.as_str()).unwrap_or("<unknown>")
|
||||
)));
|
||||
```
|
||||
|
||||
### 2. When was LoopBuilder removed?
|
||||
|
||||
**Date**: December 4, 2025
|
||||
**Commit**: `fa8a96a51b909e1fbb7a61c8e1b989050c58d4ee`
|
||||
**Phase**: 187
|
||||
**Files Deleted**: 8 files, 1,758 lines
|
||||
|
||||
**Git History**:
|
||||
```bash
|
||||
git log --all --oneline --grep="LoopBuilder" | head -5
|
||||
# fa8a96a51 docs(joinir): Phase 187 LoopBuilder Physical Removal
|
||||
# 1e350a6bc docs(joinir): Phase 186-4 Documentation Update
|
||||
# 30f94c955 feat(joinir): Phase 186 LoopBuilder Hard Freeze
|
||||
```
|
||||
|
||||
### 3. Are there any remaining LoopBuilder calls?
|
||||
|
||||
**No**. The module was completely deleted in Phase 187:
|
||||
|
||||
```bash
|
||||
# Before Phase 187
|
||||
src/mir/loop_builder/
|
||||
├── mod.rs (158 lines)
|
||||
├── loop_form.rs (578 lines)
|
||||
├── if_lowering.rs (298 lines)
|
||||
├── phi_ops.rs (333 lines)
|
||||
├── control.rs (94 lines)
|
||||
├── statements.rs (39 lines)
|
||||
├── joinir_if_phi_selector.rs (168 lines)
|
||||
└── README.md (15 lines)
|
||||
|
||||
# After Phase 187
|
||||
❌ DELETED (all files)
|
||||
✅ PRESERVED: IfInLoopPhiEmitter → src/mir/phi_core/if_in_loop_phi_emitter/mod.rs
|
||||
```
|
||||
|
||||
**Remaining References**: Only in documentation and git history
|
||||
|
||||
### 4. Where is BundleResolver defined?
|
||||
|
||||
**Location**: `lang/src/compiler/entry/bundle_resolver.hako`
|
||||
**Type**: `.hako` source code (selfhost compiler component)
|
||||
**Function**: `BundleResolver.resolve/4` (static box method)
|
||||
|
||||
**Purpose**: Stage-B bundling resolver
|
||||
- Merges multiple source bundles
|
||||
- Resolves module dependencies
|
||||
- Handles duplicate/missing module detection
|
||||
|
||||
### 5. Is it using old JoinIR patterns?
|
||||
|
||||
**YES—this is the core issue!**
|
||||
|
||||
**BundleResolver.resolve/4 characteristics**:
|
||||
- Written during LoopBuilder era (before Phase 186-188)
|
||||
- Uses loop patterns that worked with LoopBuilder
|
||||
- Those patterns are NOT yet covered by current JoinIR patterns
|
||||
|
||||
**Example incompatibility**:
|
||||
```nyash
|
||||
// This pattern worked with LoopBuilder
|
||||
local i = 0
|
||||
loop(i < table.length()) {
|
||||
local j = table.indexOf("|||", i)
|
||||
// ... complex processing
|
||||
if j < 0 { break }
|
||||
i = j + 3 // Non-unit increment
|
||||
}
|
||||
|
||||
// Current JoinIR patterns can't handle:
|
||||
// - Multiple carriers (i, j)
|
||||
// - Non-unit increment (i = j + 3)
|
||||
// - Conditional break
|
||||
// - String method calls in condition/body
|
||||
```
|
||||
|
||||
### 6. Can we see the actual loop structure that's failing?
|
||||
|
||||
**Yes**—see section "The Failing BundleResolver Loop" above.
|
||||
|
||||
**Key problematic features**:
|
||||
1. **Multiple carrier variables** (i, j, seg, pos, k)
|
||||
2. **Nested loops** (outer loop with inner loop)
|
||||
3. **Conditional assignments** (`seg = if ... then ... else ...`)
|
||||
4. **Non-unit increment** (`i = j + 3`)
|
||||
5. **Complex control flow** (multiple `if`/`break`)
|
||||
6. **String operations** (`.indexOf()`, `.substring()`, `.length()`)
|
||||
|
||||
### 7. What replaced LoopBuilder?
|
||||
|
||||
**JoinIR Frontend** with pattern-based lowering:
|
||||
|
||||
| Pattern | Description | Status | Example |
|
||||
|---------|-------------|--------|---------|
|
||||
| **Pattern 1** | Simple While | ✅ Implemented | `loop(i < n) { i++ }` |
|
||||
| **Pattern 2** | Conditional Break | ✅ Implemented | `loop(true) { if(...) break }` |
|
||||
| **Pattern 3** | If-Else PHI | ✅ Implemented | `loop(...) { if(...) sum += x else sum += 0 }` |
|
||||
| **Pattern 4** | Continue | ⏳ Planned | `loop(...) { if(...) continue }` |
|
||||
| **Pattern 5** | Infinite Early Exit | ⏳ Planned | `loop(true) { if(...) break; if(...) continue }` |
|
||||
|
||||
**Coverage**: ~80% of representative tests pass with Patterns 1-3
|
||||
|
||||
### 8. Is there a migration path documented?
|
||||
|
||||
**YES**—documented in Phase 188:
|
||||
|
||||
**From**: `docs/private/roadmap2/phases/phase-188-joinir-loop-pattern-expansion/README.md`
|
||||
|
||||
> **Phase 188 Objective**: Expand JoinIR loop pattern coverage to handle loop patterns currently failing with `[joinir/freeze]` error.
|
||||
>
|
||||
> **Status**: Planning 100% complete, Pattern 1/2 lowering + execution 100% complete, Pattern 3 lowering 100% complete
|
||||
|
||||
**Migration Strategy**:
|
||||
1. **Identify failing patterns** (Task 188-1: Error Inventory) ✅ Done
|
||||
2. **Classify patterns** (Task 188-2: Pattern Classification) ✅ Done
|
||||
3. **Prioritize 2-3 patterns** (Task 188-2) ✅ Done (Patterns 1, 2, 3)
|
||||
4. **Design JoinIR extensions** (Task 188-3) ✅ Done
|
||||
5. **Implement lowering** (Task 188-4) ✅ Done (Patterns 1-3)
|
||||
6. **Verify representative tests** (Task 188-5) ⏳ Partial (80% coverage)
|
||||
|
||||
**Documented Gaps** (from Phase 188 inventory):
|
||||
|
||||
| # | Pattern | Status | Priority |
|
||||
|---|---------|--------|----------|
|
||||
| 1 | Simple While Loop | ✅ Implemented | HIGH |
|
||||
| 2 | Loop with Conditional Break | ✅ Implemented | HIGH |
|
||||
| 3 | Loop with If-Else PHI | ✅ Implemented | MEDIUM |
|
||||
| 4 | Loop with One-Sided If | ⏳ May be auto-handled | MEDIUM |
|
||||
| 5 | Loop with Continue | ⏳ Deferred | LOW |
|
||||
| **6** | **Complex Multi-Carrier** | ❌ **NOT PLANNED** | **?** |
|
||||
|
||||
**BundleResolver falls into Gap #6**: Complex loops with multiple carriers, nested loops, and non-unit increments.
|
||||
|
||||
---
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
### Is this a code update issue or a feature gap?
|
||||
|
||||
**Answer**: **Feature gap** (not a bug)
|
||||
|
||||
### Detailed Analysis
|
||||
|
||||
**What happened**:
|
||||
1. **Phase 186-187** (Dec 4): LoopBuilder removed to force explicit failures
|
||||
2. **Phase 188** (Dec 5-6): JoinIR Patterns 1-3 implemented (80% coverage)
|
||||
3. **Gap remains**: Complex loops (like BundleResolver) not yet covered
|
||||
4. **This test fails**: Because BundleResolver uses a pattern outside the 80% coverage
|
||||
|
||||
**Is the `.hako` code wrong?**
|
||||
- **No**—the code is valid Nyash syntax
|
||||
- It worked when LoopBuilder existed
|
||||
- It's just a pattern that hasn't been migrated to JoinIR yet
|
||||
|
||||
**Is the JoinIR implementation wrong?**
|
||||
- **No**—JoinIR is working as designed
|
||||
- It explicitly fails on unsupported patterns (as intended)
|
||||
- This failure drives future pattern expansion
|
||||
|
||||
**Is this a regression?**
|
||||
- **No**—this is an **expected gap** documented in Phase 188
|
||||
- The removal of LoopBuilder was intentional to force explicit failures
|
||||
- Phase 188 documented 5 failing patterns; this is one of them
|
||||
|
||||
---
|
||||
|
||||
## What's the Fix?
|
||||
|
||||
### Short-term Options
|
||||
|
||||
#### Option A: Rewrite BundleResolver loops to use supported patterns ⭐ **RECOMMENDED**
|
||||
|
||||
**Pros**:
|
||||
- Immediate fix (no compiler changes needed)
|
||||
- Makes code compatible with current JoinIR
|
||||
- Educational exercise (learn supported patterns)
|
||||
|
||||
**Cons**:
|
||||
- Requires understanding which patterns are supported
|
||||
- May require restructuring loop logic
|
||||
|
||||
**Example Rewrite**:
|
||||
|
||||
**Before** (unsupported):
|
||||
```nyash
|
||||
local i = 0
|
||||
loop(i < table.length()) {
|
||||
local j = table.indexOf("|||", i)
|
||||
// ... complex processing
|
||||
if j < 0 { break }
|
||||
i = j + 3 // Non-unit increment
|
||||
}
|
||||
```
|
||||
|
||||
**After** (Pattern 2 compatible):
|
||||
```nyash
|
||||
local i = 0
|
||||
local done = 0
|
||||
loop(i < table.length() and done == 0) {
|
||||
local j = table.indexOf("|||", i)
|
||||
// ... complex processing
|
||||
if j < 0 {
|
||||
done = 1
|
||||
} else {
|
||||
i = j + 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Option B: Implement new JoinIR pattern for complex loops ⏳ **FUTURE WORK**
|
||||
|
||||
**Pros**:
|
||||
- Proper solution (makes compiler more capable)
|
||||
- Benefits all similar loop patterns
|
||||
|
||||
**Cons**:
|
||||
- Large effort (Pattern 1-3 took 1,802 lines + design docs)
|
||||
- Not urgent (only affects 20% of tests)
|
||||
|
||||
**Steps**:
|
||||
1. Design "Pattern 6: Complex Multi-Carrier Loop"
|
||||
2. Implement detection logic
|
||||
3. Implement JoinIR lowering
|
||||
4. Test with BundleResolver and similar loops
|
||||
|
||||
#### Option C: Temporarily restore LoopBuilder ❌ **NOT RECOMMENDED**
|
||||
|
||||
**Pros**:
|
||||
- Immediate fix (git revert)
|
||||
|
||||
**Cons**:
|
||||
- Defeats the purpose of Phase 186-188
|
||||
- Re-introduces legacy bugs
|
||||
- Hides pattern gaps (prevents JoinIR improvement)
|
||||
|
||||
**From Phase 187 docs**:
|
||||
> **Lesson 3**: Explicit Failure Drives Architecture Forward
|
||||
> Removing the fallback path forces future work to improve JoinIR, not work around it.
|
||||
|
||||
### Long-term Solution
|
||||
|
||||
**Phase 189+ planning needed**:
|
||||
1. Analyze remaining 20% of failing tests
|
||||
2. Identify common patterns (e.g., "Complex Multi-Carrier")
|
||||
3. Prioritize by impact (selfhost critical paths)
|
||||
4. Design and implement additional JoinIR patterns
|
||||
|
||||
**From Phase 188 docs**:
|
||||
> **Next**: Phase 189 - Multi-function JoinIR→MIR merge / Select instruction MIR bridge
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Immediate Action (for this test failure)
|
||||
|
||||
**1. Rewrite BundleResolver loops** (Option A)
|
||||
- **Who**: Developer working on selfhost compiler
|
||||
- **What**: Refactor loops to use Pattern 1 or Pattern 2
|
||||
- **Why**: Immediate fix, no compiler changes needed
|
||||
- **How**: See "Example Rewrite" above
|
||||
|
||||
**2. Document the pattern gap** (if not already documented)
|
||||
- **Where**: `docs/private/roadmap2/phases/phase-188-joinir-loop-pattern-expansion/inventory.md`
|
||||
- **Add**: BundleResolver.resolve/4 as example of "Complex Multi-Carrier" pattern
|
||||
- **Tag**: Phase 189+ future work
|
||||
|
||||
### Future Work
|
||||
|
||||
**1. Analyze BundleResolver loop patterns systematically**
|
||||
- Extract common loop structures
|
||||
- Classify into existing or new patterns
|
||||
- Prioritize by impact on selfhost pipeline
|
||||
|
||||
**2. Design "Pattern 6: Complex Multi-Carrier Loop"** (if needed)
|
||||
- Support multiple carrier variables (3+)
|
||||
- Support non-unit increments
|
||||
- Support nested loops
|
||||
- Support string operations in conditions
|
||||
|
||||
**3. Consider intermediate patterns**
|
||||
- "Pattern 2.5": Conditional Break + Multiple Carriers
|
||||
- "Pattern 3.5": If-Else PHI + Non-unit Increment
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
**1. Create minimal reproducer**
|
||||
- Extract minimal BundleResolver loop to separate test file
|
||||
- Use as regression test for future pattern work
|
||||
|
||||
**2. Add to Phase 188 inventory**
|
||||
- Document exact pattern characteristics
|
||||
- Mark as "Pattern 6 candidate" or "Defer to Phase 189+"
|
||||
|
||||
**3. Track coverage metrics**
|
||||
- Current: 80% (Patterns 1-3)
|
||||
- Target: 95%+ (add Pattern 4-6)
|
||||
- Critical: 100% of selfhost paths
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
### Summary of Findings
|
||||
|
||||
1. **User hypothesis is CORRECT**: This is old code (BundleResolver.hako) not compatible with new JoinIR patterns
|
||||
|
||||
2. **LoopBuilder was removed intentionally** in Phase 187 (Dec 4, 2025)
|
||||
- Deleted: 8 files, 1,758 lines
|
||||
- Replaced by: JoinIR Frontend with pattern-based lowering
|
||||
|
||||
3. **Current JoinIR coverage: ~80%** (Patterns 1-3 implemented in Phase 188)
|
||||
- Pattern 1: Simple While ✅
|
||||
- Pattern 2: Conditional Break ✅
|
||||
- Pattern 3: If-Else PHI ✅
|
||||
|
||||
4. **BundleResolver loop falls in the 20% gap**:
|
||||
- Multiple carriers (5+ variables)
|
||||
- Nested loops
|
||||
- Non-unit increments
|
||||
- Complex control flow
|
||||
|
||||
5. **This is NOT a bug**—it's a documented feature gap
|
||||
- Expected from Phase 188 planning
|
||||
- Failure is intentional (drives JoinIR improvement)
|
||||
|
||||
### Recommended Fix
|
||||
|
||||
**Short-term**: Rewrite BundleResolver loops to use Pattern 1 or Pattern 2
|
||||
**Long-term**: Implement "Pattern 6: Complex Multi-Carrier Loop" in Phase 189+
|
||||
|
||||
### Key Insight
|
||||
|
||||
The removal of LoopBuilder forces **explicit failures** rather than **silent fallbacks**. This is a **feature, not a bug**—it ensures:
|
||||
- Clear error messages guide developers
|
||||
- Pattern gaps are visible (not hidden)
|
||||
- Future work focuses on real needs (not legacy compatibility)
|
||||
|
||||
**From Phase 187 documentation**:
|
||||
> "Explicit failures replace implicit fallbacks. Future JoinIR expansion is the only way forward."
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
### Documentation
|
||||
|
||||
- **Phase 186**: `docs/private/roadmap2/phases/phase-186-loopbuilder-freeze/README.md`
|
||||
- **Phase 187**: `docs/private/roadmap2/phases/phase-180-joinir-unification-before-selfhost/README.md#8-phase-187`
|
||||
- **Phase 188**: `docs/private/roadmap2/phases/phase-188-joinir-loop-pattern-expansion/README.md`
|
||||
- **Phase 264**: `docs/development/current/main/phases/phase-264/README.md`
|
||||
|
||||
### Source Files
|
||||
|
||||
- **Error location**: `src/mir/builder/control_flow/mod.rs:136-145`
|
||||
- **Pattern detection**: `src/mir/loop_pattern_detection/mod.rs`
|
||||
- **BundleResolver**: `lang/src/compiler/entry/bundle_resolver.hako`
|
||||
- **Test file**: `apps/tests/phase264_p0_bundle_resolver_loop_min.hako`
|
||||
|
||||
### Git Commits
|
||||
|
||||
- **Phase 186 Freeze**: `30f94c955` (Dec 4, 2025)
|
||||
- **Phase 187 Removal**: `fa8a96a51` (Dec 4, 2025)
|
||||
- **Phase 188 Pattern 1**: `d303d24b4` (Dec 6, 2025)
|
||||
- **Phase 188 Pattern 2**: `87e477b13` (Dec 6, 2025)
|
||||
- **Phase 188 Pattern 3**: `638182a8a` (Dec 6, 2025)
|
||||
|
||||
### Related Issues
|
||||
|
||||
- Phase 188 Error Inventory: 5 failing patterns documented
|
||||
- Phase 264 P0: Similar issue with classification heuristic
|
||||
- Phase 189 Planning: Multi-function JoinIR merge (future)
|
||||
|
||||
---
|
||||
|
||||
**Investigation Complete**
|
||||
**Date**: 2025-12-24
|
||||
**Status**: ✅ Root cause identified, recommendations provided
|
||||
@ -0,0 +1,147 @@
|
||||
# LoopBuilder Removal - Quick Summary
|
||||
|
||||
**Date**: 2025-12-24
|
||||
**Test Failure**: `core_direct_array_oob_set_rc_vm`
|
||||
**Error**: `[joinir/freeze] Loop lowering failed: JoinIR does not support this pattern, and LoopBuilder has been removed`
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
✅ **User hypothesis is CORRECT**: Old `.hako` code not compatible with new JoinIR patterns
|
||||
|
||||
❌ **NOT a bug**: Expected feature gap (20% of patterns not yet covered)
|
||||
|
||||
⭐ **Fix**: Rewrite BundleResolver loops to use supported Pattern 1 or Pattern 2
|
||||
|
||||
---
|
||||
|
||||
## What Happened?
|
||||
|
||||
### Timeline
|
||||
|
||||
1. **Dec 4, 2025 (Phase 186-187)**: LoopBuilder deleted (1,758 lines removed)
|
||||
2. **Dec 5-6, 2025 (Phase 188)**: JoinIR Patterns 1-3 implemented (80% coverage)
|
||||
3. **Dec 24, 2025 (Today)**: BundleResolver test fails (falls in 20% gap)
|
||||
|
||||
### Why This Fails
|
||||
|
||||
**BundleResolver.resolve/4** uses a complex loop pattern:
|
||||
- Multiple carriers (5+ variables: i, j, seg, pos, k)
|
||||
- Nested loops
|
||||
- Non-unit increment (`i = j + 3`)
|
||||
- Conditional assignments
|
||||
|
||||
**Current JoinIR patterns** don't cover this:
|
||||
- ✅ Pattern 1: Simple While (single carrier, no breaks)
|
||||
- ✅ Pattern 2: Conditional Break (simple breaks only)
|
||||
- ✅ Pattern 3: If-Else PHI (arithmetic accumulation only)
|
||||
- ❌ **Pattern 6** (Complex Multi-Carrier): **Not implemented yet**
|
||||
|
||||
---
|
||||
|
||||
## What is LoopBuilder?
|
||||
|
||||
**Before Phase 187**:
|
||||
- Legacy loop lowering system (8 files, 1,000+ lines)
|
||||
- Handled all loop patterns (but with bugs)
|
||||
- Silent fallback when JoinIR didn't match
|
||||
|
||||
**After Phase 187**:
|
||||
- ❌ Completely deleted
|
||||
- ✅ Replaced by JoinIR Frontend (pattern-based)
|
||||
- ✅ Explicit failures (no silent fallbacks)
|
||||
|
||||
**Why removed?**
|
||||
> "Explicit failures replace implicit fallbacks. Future JoinIR expansion is the only way forward."
|
||||
|
||||
---
|
||||
|
||||
## How to Fix
|
||||
|
||||
### Option A: Rewrite BundleResolver Loops ⭐ **RECOMMENDED**
|
||||
|
||||
**Before** (unsupported):
|
||||
```nyash
|
||||
local i = 0
|
||||
loop(i < table.length()) {
|
||||
local j = table.indexOf("|||", i)
|
||||
// ... complex processing
|
||||
if j < 0 { break }
|
||||
i = j + 3 // Non-unit increment
|
||||
}
|
||||
```
|
||||
|
||||
**After** (Pattern 2 compatible):
|
||||
```nyash
|
||||
local i = 0
|
||||
local done = 0
|
||||
loop(i < table.length() and done == 0) {
|
||||
local j = table.indexOf("|||", i)
|
||||
// ... complex processing
|
||||
if j < 0 {
|
||||
done = 1
|
||||
} else {
|
||||
i = j + 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Tips**:
|
||||
- Convert non-unit increments to unit increments if possible
|
||||
- Convert breaks to condition checks
|
||||
- Reduce carrier count (merge variables if possible)
|
||||
|
||||
### Option B: Implement Pattern 6 ⏳ **FUTURE WORK**
|
||||
|
||||
**Effort**: Large (Pattern 1-3 took 1,802 lines + design)
|
||||
**Timeline**: Phase 189+
|
||||
**Benefit**: Supports all complex loops
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Supported Patterns (Phase 188)
|
||||
|
||||
| Pattern | Example | Status |
|
||||
|---------|---------|--------|
|
||||
| **Pattern 1** | `loop(i < n) { i++ }` | ✅ Works |
|
||||
| **Pattern 2** | `loop(true) { if(...) break }` | ✅ Works |
|
||||
| **Pattern 3** | `loop(...) { if(...) sum += x else sum += 0 }` | ✅ Works |
|
||||
| **Complex Multi-Carrier** | BundleResolver loops | ❌ Not yet |
|
||||
|
||||
### Where to Find More
|
||||
|
||||
- **Full Analysis**: `loopbuilder-removal-compatibility-analysis.md` (this directory)
|
||||
- **Phase 188 Docs**: `docs/private/roadmap2/phases/phase-188-joinir-loop-pattern-expansion/`
|
||||
- **BundleResolver Source**: `lang/src/compiler/entry/bundle_resolver.hako`
|
||||
|
||||
### Git History
|
||||
|
||||
```bash
|
||||
# Find LoopBuilder removal
|
||||
git log --all --oneline --grep="Phase 187"
|
||||
# fa8a96a51 docs(joinir): Phase 187 LoopBuilder Physical Removal
|
||||
|
||||
# See deleted files
|
||||
git show fa8a96a51 --stat
|
||||
# 13 files changed, 56 insertions(+), 1758 deletions(-)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
1. ✅ **This is NOT a bug**—it's an expected gap (20% of patterns)
|
||||
2. ✅ **LoopBuilder is gone forever**—no plans to restore
|
||||
3. ✅ **Fix: Rewrite loops**—use Pattern 1 or Pattern 2
|
||||
4. ✅ **Future: Implement Pattern 6**—if this becomes a blocker
|
||||
|
||||
**Philosophy**:
|
||||
> Explicit failures drive architecture forward. No silent fallbacks!
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-24
|
||||
**Status**: Investigation complete, recommendations provided
|
||||
@ -180,9 +180,140 @@ Phase 285A1 は Phase 285 の一部として、weak field 処理の基盤を整
|
||||
|
||||
**Note**: weak field 構文(`weak next: Node`)は未実装。Phase 285 P1 で実装予定。
|
||||
|
||||
---
|
||||
|
||||
## A1.5: Parser Hang Fix - Parameter Type Annotations ✅
|
||||
|
||||
**Status**: ✅ Complete (2025-12-24)
|
||||
|
||||
### Problem
|
||||
|
||||
During Phase 285A1.4 implementation, discovered critical parser bug:
|
||||
- User writes: `setParent(p: Node) { ... }`
|
||||
- Parser hangs infinitely at COLON token (no advance on unexpected token)
|
||||
- Workaround required removing type annotation → `setParent(p) { ... }` → Works
|
||||
|
||||
### Root Cause
|
||||
|
||||
**6 identical vulnerable parameter parsing loops** across the parser codebase:
|
||||
1. `src/parser/declarations/box_def/members/methods.rs:21-30`
|
||||
2. `src/parser/declarations/box_def/members/constructors.rs:27-34` (init)
|
||||
3. `src/parser/declarations/box_def/members/constructors.rs:101-108` (pack)
|
||||
4. `src/parser/declarations/box_def/members/constructors.rs:138-145` (birth)
|
||||
5. `src/parser/items/functions.rs:34-51`
|
||||
6. `src/parser/items/static_items.rs:72-87`
|
||||
|
||||
**Vulnerable Code Pattern**:
|
||||
```rust
|
||||
while !p.match_token(&TokenType::RPAREN) && !p.is_at_end() {
|
||||
must_advance!(p, _unused, "method parameter parsing");
|
||||
if let TokenType::IDENTIFIER(param) = &p.current_token().token_type {
|
||||
params.push(param.clone());
|
||||
p.advance(); // ← Only advances on IDENTIFIER
|
||||
}
|
||||
// ⚠️ COLON token: not IDENTIFIER, not COMMA, not RPAREN → NO ADVANCE → INFINITE LOOP
|
||||
if p.match_token(&TokenType::COMMA) {
|
||||
p.advance();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**EBNF Spec Finding**: `params` grammar undefined in EBNF.md, `:` TYPE is for return type only
|
||||
|
||||
### Solution: Shared Helper Function (DRY Principle)
|
||||
|
||||
Created `src/parser/common/params.rs` with common parameter parsing logic:
|
||||
|
||||
```rust
|
||||
/// Parse parameter name list with Fail-Fast on unexpected tokens
|
||||
///
|
||||
/// Parses: IDENT (',' IDENT)*
|
||||
/// Rejects: Type annotations, unexpected tokens, malformed comma sequences
|
||||
pub(crate) fn parse_param_name_list(
|
||||
p: &mut NyashParser,
|
||||
context: &str, // "method", "constructor", "function" for error messages
|
||||
) -> Result<Vec<String>, ParseError>
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
- **Progress-zero detection**: Tracks token position, errors if stuck (prevents infinite loops)
|
||||
- **Explicit token handling**: All token types (IDENTIFIER, COMMA, other) explicitly matched
|
||||
- **Fail-Fast**: Either advances or errors (no infinite loop possible)
|
||||
- **Unified error messages**: Context parameter customizes messages per call site
|
||||
|
||||
### Files Modified
|
||||
|
||||
**New Files** (2):
|
||||
- `src/parser/common/params.rs` (~90 lines) - Helper function
|
||||
- `tools/smokes/v2/profiles/quick/parser/phase285_param_type_annotation_nohang.sh` - Timeout smoke test
|
||||
|
||||
**Modified Files** (8):
|
||||
1. `src/parser/common/mod.rs` - Module declaration (moved common.rs → common/mod.rs)
|
||||
2. `src/parser/declarations/box_def/members/methods.rs` - Replaced 12 lines with 1 call
|
||||
3. `src/parser/declarations/box_def/members/constructors.rs` - Replaced 3 loops (init/pack/birth)
|
||||
4. `src/parser/items/functions.rs` - Replaced 20 lines with 2 lines
|
||||
5. `src/parser/items/static_items.rs` - Replaced 22 lines with 2 lines
|
||||
6. `apps/tests/phase285_parser_param_type_annot_should_not_hang.hako` - Regression test
|
||||
|
||||
**Net Change**: +90 new - 72 removed + 6 calls = **+24 lines** (with better error handling!)
|
||||
|
||||
### Tests
|
||||
|
||||
**Regression Test**: `apps/tests/phase285_parser_param_type_annot_should_not_hang.hako`
|
||||
```hako
|
||||
box TestNode {
|
||||
value: IntegerBox
|
||||
// ❌ Should error immediately (type annotation not supported)
|
||||
setParent(p: Node) {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Smoke Test**: `tools/smokes/v2/profiles/quick/parser/phase285_param_type_annotation_nohang.sh`
|
||||
- Timeout: 3 seconds (detects hang)
|
||||
- Expected: Parse error within 1 second
|
||||
- Validation: Error message mentions "Unexpected token COLON" and "Parameter type annotations not supported"
|
||||
|
||||
**Result**: ✅ PASS
|
||||
```
|
||||
✅ PASS: Parser correctly rejects param type annotations without hanging
|
||||
```
|
||||
|
||||
### Error Message Examples
|
||||
|
||||
**Before** (infinite hang):
|
||||
```
|
||||
🚨 PARSER INFINITE LOOP DETECTED at method parameter parsing
|
||||
```
|
||||
|
||||
**After** (clear error):
|
||||
```
|
||||
❌ Parse error: Unexpected token COLON, expected ',' or ')' in method parameter list.
|
||||
Note: Parameter type annotations are not supported.
|
||||
```
|
||||
|
||||
### Architecture Benefits
|
||||
|
||||
**DRY Principle**:
|
||||
- **Before**: 6 identical vulnerable loops (72 lines total)
|
||||
- **After**: 1 helper function (~90 lines) + 6 one-line calls
|
||||
|
||||
**Maintainability**:
|
||||
- Future fixes only need 1 location
|
||||
- Unified error messages (single source of truth)
|
||||
- Progress-zero detection guaranteed in one place
|
||||
|
||||
**Safety**:
|
||||
- No infinite loop possible (Fail-Fast guaranteed)
|
||||
- All token types explicitly handled (no silent fallthrough)
|
||||
- Context-aware error messages (better UX)
|
||||
|
||||
## References
|
||||
|
||||
- Phase 33: Box Theory Modularization ([phase-33-modularization.md](../../../architecture/phase-33-modularization.md))
|
||||
- Phase 285: Box lifecycle ([phase-285/README.md](README.md))
|
||||
- `src/mir/builder/weak_field_validator.rs`: 実装本体
|
||||
- `src/mir/builder/fields.rs`: 呼び出し側
|
||||
- `src/mir/builder/weak_field_validator.rs`: A1.1 実装本体
|
||||
- `src/mir/builder/fields.rs`: A1.1 呼び出し側
|
||||
- `src/parser/common/params.rs`: A1.5 実装本体
|
||||
- `src/parser/declarations/box_def/members/fields.rs`: A1.2-A1.4 実装本体
|
||||
|
||||
Reference in New Issue
Block a user