d7805e5974
feat(joinir): Phase 213-2 Step 2-2 & 2-3 Data structure extensions
...
Extended PatternPipelineContext and CarrierUpdateInfo for Pattern 3 AST-based generalization.
Changes:
1. PatternPipelineContext:
- Added loop_condition: Option<ASTNode>
- Added loop_body: Option<Vec<ASTNode>>
- Added loop_update_summary: Option<LoopUpdateSummary>
- Updated build_pattern_context() for Pattern 3
2. CarrierUpdateInfo:
- Added then_expr: Option<ASTNode>
- Added else_expr: Option<ASTNode>
- Updated analyze_loop_updates() with None defaults
Status: Phase 213-2 Steps 2-2 & 2-3 complete
Next: Create Pattern3IfAnalyzer to extract if statement and populate update summary
2025-12-10 00:01:53 +09:00
32a91e31ac
feat(joinir): Phase 200-B/C/D capture analysis + Phase 201-A reserved_value_ids infra
...
Phase 200-B: FunctionScopeCaptureAnalyzer implementation
- analyze_captured_vars_v2() with structural loop matching
- CapturedEnv for immutable function-scope variables
- ParamRole::Condition for condition-only variables
Phase 200-C: ConditionEnvBuilder extension
- build_with_captures() integrates CapturedEnv into ConditionEnv
- fn_body propagation through LoopPatternContext to Pattern 2
Phase 200-D: E2E verification
- capture detection working for base, limit, n etc.
- Test files: phase200d_capture_minimal.hako, phase200d_capture_in_condition.hako
Phase 201-A: MirBuilder reserved_value_ids infrastructure
- reserved_value_ids: HashSet<ValueId> field in MirBuilder
- next_value_id() skips reserved IDs
- merge/mod.rs sets/clears reserved IDs around JoinIR merge
Phase 201: JoinValueSpace design document
- Param/Local/PHI disjoint regions design
- API: alloc_param(), alloc_local(), reserve_phi()
- Migration plan for Pattern 1-4 lowerers
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2025-12-09 18:32:03 +09:00
440f8646b1
feat(joinir): Phase 183 LoopBodyLocal role separation + test fixes
...
Phase 183 Implementation:
- Added is_var_used_in_condition() helper for AST variable detection
- Implemented LoopBodyLocal filtering in TrimLoopLowerer
- Created 4 test files for P1/P2 patterns
- Added 5 unit tests for variable detection
Test Fixes:
- Fixed test_is_outer_scope_variable_pinned (BasicBlockId import)
- Fixed test_pattern2_accepts_loop_param_only (literal node usage)
Refactoring:
- Unified pattern detection documentation
- Consolidated CarrierInfo initialization
- Documented LoopScopeShape construction paths
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-08 23:43:26 +09:00
be06365870
feat(joinir): Phase 182-2 Add _parse_number/_atoi to routing whitelist
...
- Add JsonParserBox._parse_number/2 (P2 Break pattern)
- Add JsonParserBox._atoi/1 (P2 Break pattern)
- Fix _match_literal arity: /2 -> /3 (s, pos, literal)
- Prepare for Phase 182 simple loop implementation
2025-12-08 21:36:39 +09:00
95f3aa429e
refactor(joinir): Extract legacy binding path to routing_legacy_binding.rs
...
Phase 179-A Step 2: Separate LoopFrontendBinding JSON construction logic
into dedicated module for better organization.
Changes:
- New file: routing_legacy_binding.rs (223 lines)
- routing.rs: cf_loop_joinir_impl() simplified to 15 lines (delegates to legacy path)
- Routing now clearly separates pattern-based vs. legacy binding paths
Benefits:
- Clear separation of concerns (pattern router vs. legacy whitelist)
- routing.rs reduced from 364 to 146 lines (60% reduction)
- Legacy path isolated for future deprecation
2025-12-08 18:36:13 +09:00
24aa8ced75
feat(joinir): Phase 175 - P5 multi-carrier architecture validation
...
Task 175-1: Analyzed _parse_string carrier candidates
- Identified 3 carriers: pos (position), result (buffer), is_ch_match (promoted)
- Categorized as: required carriers (pos, result), promoted carrier (is_ch_match)
Task 175-2: Validated existing boxes support multi-carrier
- CarrierInfo: Vec<CarrierData> supports arbitrary carriers ✅
- LoopUpdateAnalyzer: Loops over all carriers ✅
- ExitMeta: Vec<(String, ValueId)> supports all exit bindings ✅
- ExitLineReconnector: Reconnects all carriers to variable_map ✅
- No code changes needed - architecture already supports it!
Task 175-3: PoC test revealed Pattern2 limitation
- Test: test_jsonparser_parse_string_min2.hako (pos + result carriers)
- CarrierInfo detected 3 carriers correctly (pos, result, is_ch_match)
- variable_map contains all carriers at pattern2_start
- BUT: Pattern2's Trim optimization only emits pos carrier in MIR
- MIR shows result stays as empty string (no loop update emitted)
- Root cause: Trim pattern focuses on position-only optimization
Task 175-4: Documentation updates
- Created: phase175-multicarrier-design.md (comprehensive analysis)
- Updated: CURRENT_TASK.md (Phase 175 completion)
- Updated: routing.rs (added JsonParserStringTest2 whitelist)
Key Finding:
- Architecture is sound ✅ - all boxes support multi-carrier
- Pattern2 implementation gap ❌ - Trim optimization ignores non-position carriers
- Phase 176 scope: Extend Pattern2 to emit all carrier updates
Next: Phase 176 for escape sequence handling and full multi-carrier emission
2025-12-08 13:34:43 +09:00
309d0803c7
feat(joinir): Phase 174 - JsonParser complex loop P5 extension design
...
- Task 174-1: Complex loop inventory (_parse_string/_parse_array/_parse_object)
- Analyzed remaining JsonParser loops for P5 expansion potential
- Identified _parse_string as most Trim-like structure (99% similarity)
- Documented complexity scores and minimization potential
- Task 174-2: Selected _parse_string as next P5 target (closest to Trim)
- Reason: LoopBodyLocal 'ch' usage matches Trim pattern exactly
- Structure: loop(pos < len) + substring + char comparison + break
- Minimization: Remove escape/buffer/continue → identical to Trim
- Task 174-3: Design doc for P5B extension (TrimLoopHelper reuse strategy)
- File: docs/development/current/main/phase174-jsonparser-p5b-design.md
- Strategy: Reuse existing TrimLoopHelper without modifications
- Proven: Pattern applies to any character comparison, not just whitespace
- Task 174-4: Minimal PoC (_parse_string without escape) successful
- Test: local_tests/test_jsonparser_parse_string_min.hako
- Result: [pattern2/trim] Safe Trim pattern detected ✅
- Detection: Trim with literals=['"'] (quote instead of whitespace)
- Routing: Added whitelist entries for JsonParserStringTest methods
- Task 174-5: Documentation updates
- Updated CURRENT_TASK.md with Phase 174 summary
- Updated joinir-architecture-overview.md with P5 generality proof
- Created phase174-jsonparser-loop-inventory-2.md (detailed analysis)
- Created phase174-jsonparser-p5b-design.md (implementation strategy)
Success Criteria Met:
✅ _parse_string minimized version runs on P5 pipeline
✅ TrimLoopHelper works with '"' (non-whitespace character)
✅ Proven: Trim pattern is character-comparison-generic, not whitespace-specific
✅ Two new design docs (inventory + design)
✅ Phase 175+ roadmap established (multi-carrier, escape sequences)
Technical Achievement:
The P5 Trim pipeline successfully handled a quote-detection loop with zero code changes,
proving the architecture's generality beyond whitespace trimming.
2025-12-08 13:08:44 +09:00
290e97c54c
feat(joinir): Phase 173 - JsonParser P5 expansion with _skip_whitespace
...
- Task 173-1: JsonParser loop inventory recheck
- Observed 6 loops: _trim (2 loops, already working), _skip_whitespace,
_parse_string, _parse_array, _unescape_string
- Created comprehensive observation report with Trim similarity ratings
- Discovered that JsonParser._trim already uses P5 pipeline successfully
- Task 173-2: Selected _skip_whitespace as Trim-equivalent pattern
- Perfect structural match with Trim (100% identical)
- Independent helper method, easy to test
- Frequently used in JsonParser (7 call sites)
- Task 173-3: Design doc for P5 pipeline extension to JsonParser
- Confirmed existing TrimLoopHelper works without modification
- No charAt() support needed (_skip_whitespace uses substring())
- Documented data flow and risk analysis
- Task 173-4: Successfully converted _skip_whitespace to JoinIR
- Created test case: local_tests/test_jsonparser_skip_whitespace.hako
- Added routing whitelist: JsonParserTest._skip_whitespace/3
- Pattern detection SUCCESS:
* LoopBodyLocal 'ch' detected
* Carrier promotion to 'is_ch_match'
* Trim pattern recognized
* JoinIR generation successful
- Verified P5 pipeline works for both static box methods and helper methods
- Task 173-5: Documentation updates
- phase173-jsonparser-loop-recheck.md: Loop observation report
- phase173-jsonparser-p5-design.md: P5 extension design
- phase173-jsonparser-p5-impl.md: Implementation results
- CURRENT_TASK.md: Phase 173 completion record
Key achievement: Proven that Trim P5 pipeline is fully generic -
works for both TrimTest (static box method) and JsonParser (helper method).
LoopBodyLocal carrier promotion is production-ready for Trim-like patterns.
Changes:
- src/mir/builder/control_flow/joinir/routing.rs: Add JsonParserTest whitelist
- docs/development/current/main/*173*.md: 3 new documentation files
- CURRENT_TASK.md: Phase 173 completion entry
2025-12-08 10:13:34 +09:00
7a9ebf7fc1
feat(joinir): Phase 170-4 Structure-based routing opt-in
...
Add NYASH_JOINIR_STRUCTURE_ONLY=1 environment variable to bypass
function name whitelist and route purely based on loop structure.
When enabled:
- Skips hardcoded function name whitelist (13 entries)
- Routes directly to pattern detection (LoopPatternContext)
- Falls back to legacy LoopBuilder if no pattern matches
This is Phase 1 of structure-based routing migration:
- Phase 1: Opt-in via env flag (this commit) ✅
- Phase 2: Make structure-based default (future)
- Phase 3: Remove whitelist entirely (future)
Verified:
- test_loop_return.hako → RC: 2 ✅
- test_trim_loop.hako → RC: 3 ✅
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-07 13:03:43 +09:00
7470bebb0b
fix(joinir): Phase 170-B Break condition delegation to condition_to_joinir
...
Remove hardcoded break condition (i >= 2) from loop_with_break_minimal.rs.
Now delegates to condition_to_joinir for dynamic break condition lowering.
Changes:
- Add extract_break_condition() to ast_feature_extractor.rs
- Pattern2 lowerer extracts break condition AST and passes to lowerer
- loop_with_break_minimal.rs uses BoolExprLowerer instead of hardcoded values
- Add TrimTest.main/0 to JoinIR routing whitelist
Box Theory compliance:
- Single responsibility: Pattern2 handles structure, condition_to_joinir handles lowering
- Zero hardcoding: All break conditions now dynamic
Verified:
- test_loop_return.hako (i >= 2) → RC: 2 ✅
- test_trim_loop.hako (i >= 3) → RC: 3 ✅
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-07 12:59:33 +09:00
a09ce0cbff
feat(joinir): Phase 33-14 JoinFragmentMeta for expr/carrier separation
...
Introduces JoinFragmentMeta to distinguish between loop expression results
and carrier variable updates, fixing SSA correctness issues.
## Changes
### New: JoinFragmentMeta struct (carrier_info.rs)
- `expr_result: Option<ValueId>` - Loop as expression (return loop(...))
- `exit_meta: ExitMeta` - Carrier updates for variable_map
- Helper methods: with_expr_result(), carrier_only(), empty()
### Pattern 2 Lowerer Updates
- loop_with_break_minimal.rs: Returns (JoinModule, JoinFragmentMeta)
- pattern2_with_break.rs: Sets boundary.expr_result from fragment_meta
### instruction_rewriter.rs
- Phase 33-14: Only add to exit_phi_inputs when boundary.expr_result is Some
- Phase 33-13: MergeResult struct with carrier_inputs map
### JoinInlineBoundary (inline_boundary.rs)
- New field: expr_result: Option<ValueId>
- All constructors updated with expr_result: None default
## Design Philosophy
Previously, exit_phi_inputs mixed expr results with carrier updates, causing:
- PHI inputs referencing undefined remapped values
- SSA-undef errors in VM execution
With JoinFragmentMeta:
- expr_result → exit_phi_inputs (generates PHI for expr value)
- exit_meta → carrier_inputs (updates variable_map via carrier PHIs)
## Test Results
- Pattern 1 (carrier-only): Works correctly (no exit_phi_inputs)
- Pattern 2 (expr result): Design complete, SSA-undef fix deferred to Phase 33-15
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-07 05:07:28 +09:00
e30116f53d
feat(joinir): Phase 171-fix ConditionEnv/ConditionBinding architecture
...
Proper HOST↔JoinIR ValueId separation for condition variables:
- Add ConditionEnv struct (name → JoinIR-local ValueId mapping)
- Add ConditionBinding struct (HOST/JoinIR ValueId pairs)
- Modify condition_to_joinir to use ConditionEnv instead of builder.variable_map
- Update Pattern2 lowerer to build ConditionEnv and ConditionBindings
- Extend JoinInlineBoundary with condition_bindings field
- Update BoundaryInjector to inject Copy instructions for condition variables
This fixes the undefined ValueId errors where HOST ValueIds were being
used directly in JoinIR instructions. Programs now execute (RC: 0),
though loop variable exit values still need Phase 172 work.
Key invariants established:
1. JoinIR uses ONLY JoinIR-local ValueIds
2. HOST↔JoinIR bridging is ONLY through JoinInlineBoundary
3. condition_to_joinir NEVER accesses builder.variable_map
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-07 01:45:03 +09:00
abfe0b198b
feat(joinir): Phase 195 - Unified JoinLoopTrace for all JoinIR debug output
...
Created centralized tracing module for JoinIR loop lowering operations,
consolidating scattered eprintln! calls into a single SSOT interface.
# Implementation
1. **Created trace.rs module** (~233 lines)
- JoinLoopTrace struct with env var controls
- Unified API: pattern(), varmap(), joinir_stats(), phi(), merge(), etc.
- Global singleton via trace() function
- Supports 5 env vars: NYASH_TRACE_VARMAP, NYASH_JOINIR_DEBUG,
NYASH_OPTION_C_DEBUG, NYASH_JOINIR_MAINLINE_DEBUG, NYASH_LOOPFORM_DEBUG
2. **Updated debug.rs** - Delegates trace_varmap() to JoinLoopTrace
3. **Updated routing.rs** - All eprintln! replaced with trace calls (10 instances)
4. **Updated pattern routers** - All 3 patterns now use unified trace
- pattern1_minimal.rs: 6 replacements
- pattern2_with_break.rs: 6 replacements
- pattern3_with_if_phi.rs: 6 replacements
- router.rs: 2 replacements
5. **Updated merge/block_allocator.rs** - 6 eprintln! → trace calls
# Benefits
- **Single Source of Truth**: All trace control through environment variables
- **Consistent Format**: Unified [trace:*] prefix for easy filtering
- **Zero Overhead**: No output when env vars unset
- **Easy Extension**: Add new trace points via existing API
- **Better Debug Experience**: Structured output with clear categories
# Testing
✅ cargo build --release - Success
✅ NYASH_TRACE_VARMAP=1 - Shows varmap traces only
✅ NYASH_JOINIR_DEBUG=1 - Shows joinir + blocks + routing traces
✅ No env vars - No debug output
✅ apps/tests/loop_min_while.hako - All tests pass
# Related
- Phase 191-194 groundwork (modular merge structure)
- NYASH_TRACE_VARMAP added today for variable_map debugging
- Consolidates ~80 scattered eprintln! calls across JoinIR
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-05 22:23:51 +09:00
67e2bfada4
feat(joinir): Phase 194 - Table-driven loop pattern router
...
Replace if/else chain with table-driven pattern dispatch for easier
pattern addition and maintenance.
# Changes
## New Files
- router.rs (137 lines): Pattern router table and dispatch logic
- LoopPatternContext: Context passed to detect/lower functions
- LoopPatternEntry: Pattern registration structure
- LOOP_PATTERNS: Static table with 3 registered patterns
- route_loop_pattern(): Single dispatch function
## Modified Files
- patterns/mod.rs (+8 lines): Export router module
- pattern1_minimal.rs (+19 lines): Added can_lower() + lower() wrapper
- pattern2_with_break.rs (+17 lines): Added can_lower() + lower() wrapper
- pattern3_with_if_phi.rs (+22 lines): Added can_lower() + lower() wrapper
- routing.rs (-21 lines): Replaced if/else chain with router call
# Architecture Improvement
## Before (if/else chain)
```rust
if func_name == "main" && has_sum {
return pattern3(...);
} else if func_name == "main" {
return pattern1(...);
} else if func_name == "JoinIrMin.main/0" {
return pattern2(...);
}
```
## After (table-driven)
```rust
let ctx = LoopPatternContext::new(...);
route_loop_pattern(self, &ctx)?
```
# Adding New Patterns (Now Trivial!)
1. Create pattern4_your_name.rs
2. Implement can_lower() + lower()
3. Add entry to LOOP_PATTERNS table
That's it! No routing logic changes needed.
# Testing
✅ Pattern 1 (loop_min_while.hako): PASSED
✅ Pattern 2 (joinir_min_loop.hako): PASSED
✅ Pattern 3 (loop_if_phi.hako): Routes correctly (VM error is pre-existing)
Router logging verified:
```
NYASH_TRACE_VARMAP=1 ./target/release/hakorune apps/tests/*.hako
[route] Pattern 'Pattern1_Minimal' matched for function 'main'
[route] Pattern 'Pattern2_WithBreak' matched for function 'JoinIrMin.main/0'
[route] Pattern 'Pattern3_WithIfPhi' matched for function 'main'
```
# Line Counts
- router.rs: 137 lines (new)
- Total pattern files: 491 lines (3 patterns)
- routing.rs: Reduced by 21 lines (-6%)
- Net addition: +137 lines (infrastructure investment)
# Documentation
See router.rs header for:
- Architecture overview
- How to add new patterns
- Priority ordering (Pattern3=30, Pattern1=10, Pattern2=20)
Phase 194 complete - pattern addition is now a trivial task!
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-12-05 22:11:39 +09:00
f018eeeba2
refactor: Extract JoinIR routing logic from control_flow.rs (Phase 3)
...
- Created joinir/routing.rs
- Moved try_cf_loop_joinir() and cf_loop_joinir_impl()
- Updated joinir/mod.rs to include routing module
- Removed ~320 lines from main control_flow.rs
- Zero breaking changes, all tests pass
Phase 1-3 complete:
- control_flow.rs: 1,632 → ~900 lines (45% reduction)
- Extracted 3 modules: debug, patterns (3 files), routing
- All functionality preserved and verified
2025-12-05 20:48:31 +09:00
