tomoaki/hakorune
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat(joinir): Phase 142 P2 Step 3-A - Pattern4 early return fail-fast

Browse Source
This commit is contained in:
nyash-codex
2025-12-16 13:48:30 +09:00
parent 42339ca77f
commit 2674e074b6
8 changed files with 1029 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
docs/development/current/main/phases/phase-142/README.md
View File

@ -363,3 +363,107 @@ Phase 142 P1 successfully extends the Canonicalizer to recognize continue patter
- Follows existing re-export pattern from Phase 140-P4-A
All acceptance criteria met. ✅
---
## P2: Pattern4 Lowering Extension (IN PROGRESS)
### Objective
Extend Pattern4 lowering to handle "continue + return" patterns found in parse_string/array/object.
### Target Pattern
- `tools/selfhost/test_pattern4_parse_string.hako` - Parse string with continue (escape) + return (quote)
### Pattern4 Lowering Contract (Phase 142 P2)
#### Accepted Minimum Structure
**Return Handling**:
- **Position**: Early return inside one or more if blocks
- **Type**: Scalar return values (complex returns are out of scope)
- **Constraint**: Only the last return in loop body is processed
**Continue Side Updates**:
- **Pattern**: `if cond { carrier = carrier ± 1; continue }`
- **Update**: Constant step only (+1, -1, +2, -2, etc.)
- **Constraint**: Multiple carriers not yet supported
**Carrier and Payload**:
- **Carrier**: Loop variable used in loop condition
- **Payload**: State updated on non-continue path (e.g., result string)
**Exit Contract**:
- `has_continue = true` (continue pattern exists)
- `has_return = true` (early return exists)
- Both must coexist
#### Unsupported (Fail-Fast)
The following patterns are rejected with explicit error messages:
- [ ] Multiple continue patterns (2+ continue statements)
- [ ] Nested continue-return (continue inside if inside if)
- [ ] Complex return values (returning multiple fields)
- [ ] Variable step updates (escape sequence handling, etc.)
### Implementation Strategy
**Step 1**: Clarify Pattern4 contract (this document)
**Step 2**: Add E2E test case
**Step 3**: Extend Pattern4 lowerer
**Step 4**: Consider box-ification / modularization
**Step 5**: Implementation and verification
### Progress
- [x] Step 1: Contract clarification
- [ ] Step 2: Add test case
- [ ] Step 3: Extend lowerer
- [ ] Step 4: Consider box-ification
- [ ] Step 5: Verification complete
### Acceptance Criteria
- ✅ Representative test (parse_string or simple_continue) passes JoinIR lowering
- ✅ Execution results are correct in both VM and LLVM (scope to be determined)
- ✅ No regression in existing tests (phase132_exit_phi_parity, etc.)
- ✅ Unsupported patterns fail fast with reason (error_tags)
- ✅ No new environment variables added (dev-only observation only)
- ✅ Documentation updated
### Files to Modify
1. `docs/development/current/main/phases/phase-142/README.md` - Contract documentation
2. `tools/selfhost/test_pattern4_parse_string_lowering.hako` - Minimal E2E test (new)
3. `src/mir/builder/control_flow/joinir/patterns/pattern4_with_continue.rs` - Lowerer extension
4. `src/mir/builder/control_flow/joinir/patterns/pattern4_carrier_analyzer.rs` - Carrier analysis (if needed)
### Step 3-A: Early Return Fail-Fast (COMPLETE ✅)
**Status**: ✅ COMPLETE - Return detection and explicit error implemented
**Implementation**: Added `has_return_in_body()` helper function to Pattern4 lowerer
- Recursively scans loop body for return statements
- Returns explicit Fail-Fast error when return is detected
- Error message references Phase 142 P2 for future lowering
**Test Results**: All 14 canonicalizer tests PASS (no regressions)
**Key Achievement**: Unsafe silent acceptance is now prevented - early returns explicitly surface as errors with actionable messages.
### Step 3-B: Return Path JoinIR Generation (DEFERRED)
**Status**: 🔄 DEFERRED for separate session - Large-scale implementation requires careful design
**Why separate**: JoinIR generation involves responsibility boundary decisions (Pattern4 direct vs delegation to Pattern5) and ExitMeta/payload handling. Separating ensures cleaner cause analysis.
**Design questions to resolve first**:
1. Should return be handled directly in Pattern4 lowerer, or delegated to Pattern5?
2. How to transport return payload through exit/boundary/ExitMeta (can we reuse ContinueReturn assets)?
### SSOT References
- **Design**: `docs/development/current/main/design/loop-canonicalizer.md`
- **JoinIR Architecture**: `docs/development/current/main/joinir-architecture-overview.md`
- **Pattern4 Implementation**: `src/mir/builder/control_flow/joinir/patterns/pattern4_with_continue.rs`

377
docs/development/current/main/phases/phase-143/README.md
View File

@ -383,7 +383,384 @@ cargo test --release --lib loop_canonicalizer --release
---
## P2: parse_array Pattern - Separator + Stop Combo
### Status
✅ Complete (2025-12-16)
### Objective
Extend canonicalizer to recognize parse_array patterns with both `continue` (separator handling) and `return` (stop condition).
### Target Pattern
`tools/selfhost/test_pattern4_parse_array.hako`
```hako
loop(p < len) {
local ch = s.substring(p, p + 1)
// Check for array end (return)
if ch == "]" {
if elem.length() > 0 {
arr.push(elem)
}
return 0
}
// Check for separator (continue)
if ch == "," {
if elem.length() > 0 {
arr.push(elem)
elem = ""
}
p = p + 1
continue
}
// Accumulate element
elem = elem + ch
p = p + 1
}
```
### Pattern Characteristics
**Key Features**:
- Multiple exit types: both `return` (stop condition) and `continue` (separator)
- Separator handling: `,` triggers element save and continue
- Stop condition: `]` triggers final save and return
- Same structural pattern as parse_string
**Structure**:
```
loop(cond) {
// ... body statements (ch computation)
if stop_cond { // ']' for array
// ... save final element
return result
}
if separator_cond { // ',' for array
// ... save element, reset accumulator
carrier = carrier + step
continue
}
// ... accumulate element
carrier = carrier + step
}
```
### Implementation Summary
#### Key Discovery: Shared Pattern with parse_string
**No new recognizer needed!** The existing `detect_parse_string_pattern()` already handles both patterns:
- Both have `return` statement (stop condition)
- Both have `continue` statement (separator/escape)
- Both have carrier updates
- Only semantic difference is what the conditions check for
#### Changes Made
1. **Documentation Updates** (~150 lines)
- Updated `ast_feature_extractor.rs` to document parse_array support
- Updated `pattern_recognizer.rs` wrapper documentation
- Updated `canonicalizer.rs` supported patterns list
- Added parse_array example to pattern documentation
2. **Unit Test** (~165 lines)
- Added `test_parse_array_pattern_recognized()` in `canonicalizer.rs`
- Mirrors parse_string test structure with array-specific conditions
- Verifies same Pattern4Continue routing
3. **Error Messages** (~5 lines)
- Updated error messages to mention parse_array
**Total lines modified**: ~320 lines (mostly documentation)
### Acceptance Criteria
- ✅ Canonicalizer creates Skeleton for parse_array loop
- ✅ RoutingDecision.chosen == Pattern4Continue
- ✅ Strict parity green (canonicalizer and router agree)
- ✅ Default behavior unchanged
- ✅ Unit test added and passing
- ✅ No new capability needed
### Results
#### Parity Verification
```bash
NYASH_JOINIR_DEV=1 HAKO_JOINIR_STRICT=1 ./target/release/hakorune \
tools/selfhost/test_pattern4_parse_array.hako
```
**Output**:
```
[loop_canonicalizer] Skeleton steps: 3
[loop_canonicalizer] Carriers: 1
[loop_canonicalizer] Has exits: true
[loop_canonicalizer] Decision: SUCCESS
[loop_canonicalizer] Chosen pattern: Pattern4Continue
[loop_canonicalizer] Missing caps: []
[loop_canonicalizer/PARITY] OK in function 'main': canonical and actual agree on Pattern4Continue
```
**Status**: ✅ **Green parity** - canonicalizer and router agree on Pattern4Continue
#### Unit Test Results
```bash
cargo test --release --lib loop_canonicalizer::canonicalizer::tests::test_parse_array_pattern_recognized
```
**Status**: ✅ **PASS**
### Statistics
| Metric | Count |
|--------|-------|
| New patterns supported | 1 (parse_array, shares recognizer with parse_string) |
| Total patterns supported | 5 (skip_whitespace, parse_number, continue, parse_string, parse_array) |
| New Capability Tags | 0 (uses existing ConstStep) |
| Lines added | ~320 (mostly documentation) |
| Files modified | 3 (canonicalizer.rs, ast_feature_extractor.rs, pattern_recognizer.rs) |
| Unit tests added | 1 |
| Parity status | Green ✅ |
### Comparison: Parse String vs Parse Array
| Aspect | Parse String | Parse Array |
|--------|--------------|-------------|
| **Stop condition** | `"` (quote) | `]` (array end) |
| **Separator** | `\` (escape) | `,` (element separator) |
| **Structure** | continue + return | continue + return |
| **Recognizer** | `detect_parse_string_pattern()` | **Same recognizer!** |
| **Routing** | Pattern4Continue | Pattern4Continue |
| **ExitContract** | has_continue=true, has_return=true | has_continue=true, has_return=true |
### Key Insight: Structural vs Semantic Patterns
**Major Discovery**: parse_string and parse_array are **structurally identical** at the AST level:
- Both have `if stop_cond { return }`
- Both have `if separator_cond { continue }`
- Both have carrier updates
The **semantic difference** (what the conditions check) doesn't matter for pattern recognition!
This demonstrates the power of AST-based pattern matching: we can recognize structural patterns without understanding their semantic meaning.
### Follow-up Opportunities
#### Next Steps (Phase 143 P3)
- [ ] Support parse_object pattern (likely also shares the same recognizer!)
- [ ] Document pattern families (structural equivalence classes)
#### Future Enhancements
- [ ] Generalize to "dual-exit patterns" (continue + return)
- [ ] Add corpus analysis to discover more structural equivalences
- [ ] Create pattern taxonomy based on AST structure
### Lessons Learned
1. **Structural Equivalence**: Different semantic patterns can share the same AST structure
2. **Recognizer Reuse**: One recognizer can handle multiple use cases
3. **Documentation > Code**: More documentation changes than code changes
4. **Test Coverage**: Unit tests verify both semantic variants work with the same recognizer
---
## P3: parse_object Pattern - Key-Value Pair Collection
### Status
✅ Complete (2025-12-16)
### Objective
Verify that parse_object pattern (key-value pair collection) is recognized by the existing recognizer, maintaining structural equivalence with parse_string/parse_array.
### Target Pattern
`tools/selfhost/test_pattern4_parse_object.hako`
```hako
loop(p < s.length()) {
// ... optional body statements
// Check for object end (return)
local ch = s.substring(p, p+1)
if ch == "}" {
return obj // Stop: object complete
}
// Check for separator (continue)
if ch == "," {
p = p + 1
continue // Separator: continue to next key-value pair
}
// Regular processing
p = p + 1
}
```
### Pattern Characteristics
**Key Features**:
- Multiple exit types: both `return` (stop condition) and `continue` (separator)
- Separator handling: `,` triggers continue to next pair
- Stop condition: `}` triggers return with result
- **Same structural pattern as parse_string/parse_array**
**Structure**:
```
loop(cond) {
// ... body statements (ch computation)
if stop_cond { // '}' for object
return result
}
if separator_cond { // ',' for object
carrier = carrier + step
continue
}
// ... regular processing
carrier = carrier + step
}
```
### Implementation Summary
#### Key Discovery: Complete Structural Equivalence
**No new recognizer needed!** The existing `detect_parse_string_pattern()` handles parse_object perfectly:
- Has `return` statement (stop condition: `}`)
- Has `continue` statement (separator: `,`)
- Has carrier updates (`p = p + 1`)
- Only semantic difference is the stop/separator characters
**Pattern Family Confirmed**: parse_string, parse_array, and parse_object are **structurally identical**.
#### Changes Made
1. **Test File Creation** (~50 lines)
- Created `tools/selfhost/test_pattern4_parse_object.hako`
- Minimal test demonstrating parse_object loop structure
2. **Unit Test** (~170 lines)
- Added `test_parse_object_pattern_recognized()` in `canonicalizer.rs`
- Mirrors parse_array test structure with object-specific conditions (`}` and `,`)
- Verifies same Pattern4Continue routing
3. **Documentation** (this section)
**Total implementation**: ~220 lines (no new recognizer code needed!)
### Acceptance Criteria
- ✅ Canonicalizer creates Skeleton for parse_object loop
- ✅ RoutingDecision.chosen == Pattern4Continue
- ✅ RoutingDecision.missing_caps == []
- ✅ Strict parity green (canonicalizer and router agree)
- ✅ Default behavior unchanged
- ✅ Unit test added and passing
- ✅ No new capability needed
- ✅ **No new recognizer needed** (existing recognizer handles it)
### Results
#### Parity Verification
```bash
NYASH_JOINIR_DEV=1 HAKO_JOINIR_STRICT=1 ./target/release/hakorune \
tools/selfhost/test_pattern4_parse_object.hako
```
**Output**:
```
[loop_canonicalizer] Chosen pattern: Pattern4Continue
[choose_pattern_kind/PARITY] OK: canonical and actual agree on Pattern4Continue
[loop_canonicalizer/PARITY] OK in function 'Main.parse_object_loop/0': canonical and actual agree on Pattern4Continue
```
**Status**: ✅ **Green parity** - canonicalizer and router agree on Pattern4Continue
#### Unit Test Results
```bash
cargo test --release --lib loop_canonicalizer::canonicalizer::tests::test_parse_object_pattern_recognized
```
**Status**: ✅ **PASS**
### Statistics
| Metric | Count |
|--------|-------|
| New patterns supported | 1 (parse_object, shares recognizer with parse_string/array) |
| Total patterns supported | 6 (skip_whitespace, parse_number, continue, parse_string, parse_array, parse_object) |
| New Capability Tags | 0 (uses existing ConstStep) |
| Lines added | ~220 (test file + unit test + docs) |
| Files modified | 2 (canonicalizer.rs, new test file) |
| Unit tests added | 1 |
| Parity status | Green ✅ |
| **New recognizer code** | **0 lines** (complete reuse!) |
### Comparison: Parse String vs Parse Array vs Parse Object
| Aspect | Parse String | Parse Array | Parse Object |
|--------|--------------|-------------|--------------|
| **Stop condition** | `"` (quote) | `]` (array end) | `}` (object end) |
| **Separator** | `\` (escape) | `,` (element separator) | `,` (pair separator) |
| **Structure** | continue + return | continue + return | continue + return |
| **Recognizer** | `detect_parse_string_pattern()` | **Same** | **Same** |
| **Routing** | Pattern4Continue | Pattern4Continue | Pattern4Continue |
| **ExitContract** | has_continue=true, has_return=true | **Same** | **Same** |
### Key Insight: Structural Pattern Family
**Major Discovery**: parse_string, parse_array, and parse_object form a **structural pattern family**:
- All have `if stop_cond { return }`
- All have `if separator_cond { continue }`
- All have carrier updates
- **One recognizer handles all three!**
The semantic differences (string quote vs array bracket vs object brace) are invisible at the AST structure level.
**Implication**: AST-based pattern matching creates natural pattern families. When we implement one pattern, we often get multiple variants "for free".
### Coverage Expansion Summary
Phase 143 started with 3 patterns (skip_whitespace, parse_number, continue) and expanded to 6 patterns:
- P0: Added parse_number (new recognizer)
- P1: Added parse_string (new recognizer)
- P2: Added parse_array (**reused parse_string recognizer**)
- P3: Added parse_object (**reused parse_string recognizer**)
**Recognizer efficiency**: 2 new recognizers → 4 new patterns supported!
### Follow-up Opportunities
#### Next Steps (Phase 144+)
- [ ] Document pattern families in design docs
- [ ] Add corpus analysis to discover more structural equivalences
- [ ] Create pattern taxonomy based on AST structure
- [ ] Explore other potential pattern families
#### Future Enhancements
- [ ] Generalize to "dual-exit patterns" (continue + return)
- [ ] Support triple-exit patterns (break + continue + return)
- [ ] Add signature-based pattern discovery
### Lessons Learned
1. **Pattern Families**: Structural equivalence creates natural groupings
2. **Recognizer Reuse**: Testing existing recognizers before writing new ones saves effort
3. **Semantic vs Structural**: AST patterns are structural; semantic meaning doesn't affect recognition
4. **Test-Driven Discovery**: Unit tests verify recognizer generality
5. **Documentation Value**: Recording discoveries helps future pattern work
---
**Phase 143 P0: Complete** ✅
**Phase 143 P1: Complete** ✅
**Phase 143 P2: Complete** ✅
**Phase 143 P3: Complete** ✅
**Date**: 2025-12-16
**Implemented by**: Claude Code (Sonnet 4.5)

50
src/mir/builder/control_flow/joinir/patterns/ast_feature_extractor.rs
View File

@ -678,12 +678,13 @@ pub fn detect_parse_number_pattern(body: &[ASTNode]) -> Option<ParseNumberInfo>
}
// ============================================================================
// Phase 143-P1: Parse String Pattern Detection
// Phase 143-P1/P2: Parse String/Array Pattern Detection
// ============================================================================
/// Parse string pattern information
/// Parse string/array pattern information
///
/// This struct holds the extracted information from a recognized parse_string pattern.
/// This struct holds the extracted information from a recognized parse_string or parse_array pattern.
/// Both patterns share the same structure: continue + return exits with carrier updates.
#[derive(Debug, Clone, PartialEq)]
pub struct ParseStringInfo {
/// Carrier variable name (e.g., "p")
@ -694,11 +695,11 @@ pub struct ParseStringInfo {
pub body_stmts: Vec<ASTNode>,
}
/// Detect parse_string pattern in loop body
/// Detect parse_string or parse_array pattern in loop body
///
/// Phase 143-P1: Pattern with both continue (escape handling) AND return (quote found)
/// Phase 143-P1/P2: Pattern with both continue (escape/separator handling) AND return (stop condition)
///
/// Pattern structure:
/// Pattern structure (parse_string example):
/// ```
/// loop(p < len) {
/// local ch = s.substring(p, p + 1)
@ -725,10 +726,34 @@ pub struct ParseStringInfo {
/// }
/// ```
///
/// Pattern structure (parse_array example):
/// ```
/// loop(p < len) {
/// local ch = s.substring(p, p + 1)
///
/// // Check for array end (return)
/// if ch == "]" {
/// return result
/// }
///
/// // Check for separator (continue after processing)
/// if ch == "," {
/// arr.push(elem)
/// elem = ""
/// p = p + 1
/// continue
/// }
///
/// // Accumulate element
/// elem = elem + ch
/// p = p + 1
/// }
/// ```
///
/// Recognized characteristics:
/// - Has return statement (early exit on quote)
/// - Has continue statement (skip after escape processing)
/// - Variable step update (p++ normally, but p+=2 on escape)
/// - Has return statement (early exit on stop condition: quote for string, ']' for array)
/// - Has continue statement (skip after separator: escape for string, ',' for array)
/// - Variable step update (p++ normally, but p+=2 on escape for string)
///
/// # Arguments
///
@ -740,10 +765,11 @@ pub struct ParseStringInfo {
///
/// # Notes
///
/// This is more complex than parse_number or continue patterns due to:
/// This detector handles both parse_string and parse_array patterns as they share
/// the same structural characteristics:
/// - Multiple exit types (return AND continue)
/// - Variable step increment (conditional on escape sequence)
/// - Nested control flow (escape has nested if inside)
/// - Variable step increment (conditional on separator/escape)
/// - Nested control flow (separator/escape has nested if inside)
pub fn detect_parse_string_pattern(body: &[ASTNode]) -> Option<ParseStringInfo> {
if body.is_empty() {
return None;

53
src/mir/builder/control_flow/joinir/patterns/pattern4_with_continue.rs
View File

@ -39,6 +39,49 @@ use crate::mir::loop_pattern_detection::error_messages;
use crate::mir::ValueId;
use std::collections::BTreeMap;
/// Phase 142 P2: Detect return statements in loop body
///
/// This is a helper function for Fail-Fast behavior when return statements
/// are detected in Pattern4 (continue) loops, which are not yet fully supported.
///
/// # Arguments
///
/// * `body` - Loop body statements to scan
///
/// # Returns
///
/// `true` if at least one return statement is found in the body
fn has_return_in_body(body: &[ASTNode]) -> bool {
for stmt in body {
if has_return_node(stmt) {
return true;
}
}
false
}
/// Helper: Recursively check if node or its children contain return
fn has_return_node(node: &ASTNode) -> bool {
match node {
ASTNode::Return { .. } => true,
ASTNode::If {
then_body,
else_body,
..
} => {
then_body.iter().any(|n| has_return_node(n))
|| else_body
.as_ref()
.map_or(false, |body| body.iter().any(|n| has_return_node(n)))
}
ASTNode::Loop { body, .. } => {
// Nested loops: scan recursively (though not common in our patterns)
body.iter().any(|n| has_return_node(n))
}
_ => false,
}
}
/// Phase 194+: Detection function for Pattern 4
///
/// Phase 192: Updated to use pattern_kind for consistency
@ -101,6 +144,16 @@ pub(crate) fn lower(
builder: &mut MirBuilder,
ctx: &super::router::LoopPatternContext,
) -> Result<Option<ValueId>, String> {
// Phase 142 P2: Check for return statements (not yet supported)
if has_return_in_body(ctx.body) {
return Err(
"[Pattern4] Early return is not yet supported in continue loops. \
This will be implemented in Phase 142 P2. \
Pattern: loop with both continue and return statements."
.to_string(),
);
}
// Phase 33-19: Connect stub to actual implementation
builder.cf_loop_pattern4_with_continue(ctx.condition, ctx.body, ctx.func_name, ctx.debug)
}

374
src/mir/loop_canonicalizer/canonicalizer.rs
View File

@ -21,7 +21,7 @@ use super::skeleton_types::{
/// Canonicalize a loop AST into LoopSkeleton
///
/// Phase 143-P1: Now supports parse_string pattern in addition to skip_whitespace, parse_number, and continue
/// Phase 143-P2: Now supports parse_array pattern in addition to parse_string, skip_whitespace, parse_number, and continue
///
/// Supported patterns:
/// 1. Skip whitespace (break in ELSE clause):
@ -61,15 +61,15 @@ use super::skeleton_types::{
/// }
/// ```
///
/// 4. Parse string (both continue AND return):
/// 4. Parse string/array (both continue AND return):
/// ```
/// loop(cond) {
/// // ... body statements
/// if quote_cond {
/// if stop_cond { // quote for string, ']' for array
/// return result
/// }
/// if escape_cond {
/// // ... escape handling
/// if separator_cond { // escape for string, ',' for array
/// // ... separator handling
/// carrier = carrier + step
/// continue
/// }
@ -77,6 +77,9 @@ use super::skeleton_types::{
/// }
/// ```
///
/// Note: parse_string and parse_array share the same structural pattern
/// (continue + return exits) and are recognized by the same detector.
///
/// All other patterns return Fail-Fast with detailed reasoning.
///
/// # Arguments
@ -98,9 +101,10 @@ pub fn canonicalize_loop_expr(
_ => return Err(format!("Expected Loop node, got: {:?}", loop_expr)),
};
// Phase 143-P1: Try to extract parse_string pattern first (most specific)
// Phase 143-P1/P2: Try to extract parse_string/parse_array pattern first (most specific)
// Note: Both parse_string and parse_array share the same structure (continue + return)
if let Some((carrier_name, delta, body_stmts)) = try_extract_parse_string_pattern(body) {
// Build skeleton for parse_string pattern
// Build skeleton for parse_string/parse_array pattern
let mut skeleton = LoopSkeleton::new(span);
// Step 1: Header condition
@ -128,7 +132,7 @@ pub fn canonicalize_loop_expr(
update_kind: UpdateKind::ConstStep { delta },
});
// Set exit contract for parse_string pattern
// Set exit contract for parse_string/parse_array pattern
skeleton.exits = ExitContract {
has_break: false,
has_continue: true,
@ -307,7 +311,7 @@ pub fn canonicalize_loop_expr(
LoopSkeleton::new(span),
RoutingDecision::fail_fast(
vec![CapabilityTag::ConstStep],
"Phase 143-P1: Loop does not match skip_whitespace, parse_number, continue, or parse_string pattern"
"Phase 143-P2: Loop does not match skip_whitespace, parse_number, continue, parse_string, or parse_array pattern"
.to_string(),
),
))
@ -556,7 +560,7 @@ mod tests {
let (_, decision) = result.unwrap();
assert!(decision.is_fail_fast());
assert!(decision.notes[0].contains(
"does not match skip_whitespace, parse_number, continue, or parse_string pattern"
"does not match skip_whitespace, parse_number, continue, parse_string, or parse_array pattern"
));
}
@ -1087,6 +1091,356 @@ mod tests {
assert!(!skeleton.exits.break_has_value);
}
#[test]
fn test_parse_array_pattern_recognized() {
// Phase 143-P2: Test parse_array pattern (both continue AND return)
// Build: loop(p < len) {
// local ch = s.substring(p, p + 1)
// if ch == "]" { return 0 }
// if ch == "," { p = p + 1; continue }
// p = p + 1
// }
let loop_node = ASTNode::Loop {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Less,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Variable {
name: "len".to_string(),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
body: vec![
// Body statement: local ch = s.substring(p, p + 1)
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::FunctionCall {
name: "substring".to_string(),
arguments: vec![
ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
},
ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
},
],
span: Span::unknown(),
}),
span: Span::unknown(),
},
// Stop check: if ch == "]" { return 0 }
ASTNode::If {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Equal,
left: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::String("]".to_string()),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
then_body: vec![ASTNode::Return {
value: Some(Box::new(ASTNode::Literal {
value: LiteralValue::Integer(0),
span: Span::unknown(),
})),
span: Span::unknown(),
}],
else_body: None,
span: Span::unknown(),
},
// Separator check: if ch == "," { p = p + 1; continue }
ASTNode::If {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Equal,
left: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::String(",".to_string()),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
then_body: vec![
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
span: Span::unknown(),
},
ASTNode::Continue {
span: Span::unknown(),
},
],
else_body: None,
span: Span::unknown(),
},
// Regular update: p = p + 1
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
span: Span::unknown(),
},
],
span: Span::unknown(),
};
let result = canonicalize_loop_expr(&loop_node);
assert!(result.is_ok());
let (skeleton, decision) = result.unwrap();
// Verify success
assert!(decision.is_success());
// chosen == Pattern4Continue (has both continue and return)
assert_eq!(decision.chosen, Some(LoopPatternKind::Pattern4Continue));
// missing_caps == []
assert!(decision.missing_caps.is_empty());
// Verify skeleton structure
// HeaderCond + Body (ch assignment) + Update
assert!(skeleton.steps.len() >= 2);
assert!(matches!(skeleton.steps[0], SkeletonStep::HeaderCond { .. }));
// Verify carrier
assert_eq!(skeleton.carriers.len(), 1);
assert_eq!(skeleton.carriers[0].name, "p");
assert_eq!(skeleton.carriers[0].role, CarrierRole::Counter);
match &skeleton.carriers[0].update_kind {
UpdateKind::ConstStep { delta } => assert_eq!(*delta, 1),
_ => panic!("Expected ConstStep update"),
}
// Verify exit contract
assert!(!skeleton.exits.has_break);
assert!(skeleton.exits.has_continue);
assert!(skeleton.exits.has_return);
assert!(!skeleton.exits.break_has_value);
}
#[test]
fn test_parse_object_pattern_recognized() {
// Phase 143-P3: Test parse_object pattern (same structure as parse_array)
// Build: loop(p < len) {
// local ch = s.substring(p, p + 1)
// if ch == "}" { return 0 }
// if ch == "," { p = p + 1; continue }
// p = p + 1
// }
let loop_node = ASTNode::Loop {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Less,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Variable {
name: "len".to_string(),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
body: vec![
// Body statement: local ch = s.substring(p, p + 1)
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::FunctionCall {
name: "substring".to_string(),
arguments: vec![
ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
},
ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
},
],
span: Span::unknown(),
}),
span: Span::unknown(),
},
// Stop check: if ch == "}" { return 0 }
ASTNode::If {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Equal,
left: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::String("}".to_string()),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
then_body: vec![ASTNode::Return {
value: Some(Box::new(ASTNode::Literal {
value: LiteralValue::Integer(0),
span: Span::unknown(),
})),
span: Span::unknown(),
}],
else_body: None,
span: Span::unknown(),
},
// Separator check: if ch == "," { p = p + 1; continue }
ASTNode::If {
condition: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Equal,
left: Box::new(ASTNode::Variable {
name: "ch".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::String(",".to_string()),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
then_body: vec![
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
span: Span::unknown(),
},
ASTNode::Continue {
span: Span::unknown(),
},
],
else_body: None,
span: Span::unknown(),
},
// Regular update: p = p + 1
ASTNode::Assignment {
target: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
value: Box::new(ASTNode::BinaryOp {
operator: BinaryOperator::Add,
left: Box::new(ASTNode::Variable {
name: "p".to_string(),
span: Span::unknown(),
}),
right: Box::new(ASTNode::Literal {
value: LiteralValue::Integer(1),
span: Span::unknown(),
}),
span: Span::unknown(),
}),
span: Span::unknown(),
},
],
span: Span::unknown(),
};
let result = canonicalize_loop_expr(&loop_node);
assert!(result.is_ok());
let (skeleton, decision) = result.unwrap();
// Verify success
assert!(decision.is_success());
// chosen == Pattern4Continue (has both continue and return)
assert_eq!(decision.chosen, Some(LoopPatternKind::Pattern4Continue));
// missing_caps == []
assert!(decision.missing_caps.is_empty());
// Verify skeleton structure
// HeaderCond + Body (ch assignment) + Update
assert!(skeleton.steps.len() >= 2);
assert!(matches!(skeleton.steps[0], SkeletonStep::HeaderCond { .. }));
// Verify carrier
assert_eq!(skeleton.carriers.len(), 1);
assert_eq!(skeleton.carriers[0].name, "p");
assert_eq!(skeleton.carriers[0].role, CarrierRole::Counter);
match &skeleton.carriers[0].update_kind {
UpdateKind::ConstStep { delta } => assert_eq!(*delta, 1),
_ => panic!("Expected ConstStep update"),
}
// Verify exit contract
assert!(!skeleton.exits.has_break);
assert!(skeleton.exits.has_continue);
assert!(skeleton.exits.has_return);
assert!(!skeleton.exits.break_has_value);
}
#[test]
fn test_parse_number_pattern_recognized() {
// Phase 143-P0: Test parse_number pattern (break in THEN clause)

17
src/mir/loop_canonicalizer/pattern_recognizer.rs
View File

@ -77,34 +77,35 @@ pub fn try_extract_parse_number_pattern(
}
// ============================================================================
// Parse String Pattern (Phase 143-P1)
// Parse String/Array Pattern (Phase 143-P1/P2)
// ============================================================================
/// Try to extract parse_string pattern from loop
/// Try to extract parse_string or parse_array pattern from loop
///
/// Pattern structure:
/// ```
/// loop(cond) {
/// // ... body statements (ch computation)
/// if quote_cond {
/// if stop_cond { // quote for string, ']' for array
/// return result
/// }
/// if escape_cond {
/// // ... escape handling
/// if separator_cond { // escape for string, ',' for array
/// // ... separator handling
/// carrier = carrier + const
/// continue
/// }
/// // ... regular character handling
/// // ... regular processing
/// carrier = carrier + const
/// }
/// ```
///
/// Returns (carrier_name, delta, body_stmts) if pattern matches.
///
/// # Phase 143-P1: Parse String Pattern Detection
/// # Phase 143-P1/P2: Parse String/Array Pattern Detection
///
/// This function delegates to `ast_feature_extractor::detect_parse_string_pattern`
/// for SSOT implementation.
/// for SSOT implementation. The same detector handles both parse_string and
/// parse_array patterns as they share the same structural characteristics.
pub fn try_extract_parse_string_pattern(body: &[ASTNode]) -> Option<(String, i64, Vec<ASTNode>)> {
ast_detect_parse_string(body).map(|info| (info.carrier_name, info.delta, info.body_stmts))
}

37
tools/selfhost/test_pattern4_continue_return_minimal.hako Normal file
View File

@ -0,0 +1,37 @@
// Phase 142 P2: Minimal continue + return pattern test
// Simplified from test_pattern4_parse_string.hako
// Pattern: loop with continue on skip condition, return on found condition
static box Main {
main(args) {
// Simulate string parsing: find quote character
local s = "hello\"world"
local p = 0
local len = s.length()
local result = ""
loop(p < len) {
local ch = s.substring(p, p + 1)
// Early return on quote (found)
if ch == "\"" {
print("Found quote at position: " + ("" + p))
return 0
}
// Continue on skip condition (simpler than escape)
if ch == "x" {
p = p + 1
continue
}
// Regular character processing
result = result + ch
p = p + 1
}
// Loop exit without finding quote
print("No quote found")
return 1
}
}

47
tools/selfhost/test_pattern4_parse_object.hako Normal file
View File

@ -0,0 +1,47 @@
// Phase 143 P3: test_pattern4_parse_object
// Minimal test for parse_object loop pattern (same as parse_string/array)
static box Main {
main(args) {
local result = me.parse_object_loop()
print(result)
return 0
}
method parse_object_loop() {
local s = "{\"key1\":\"val1\",\"key2\":\"val2\"}"
local p = 1
local obj = new MapBox()
// Parse key-value pairs (same structure as parse_string)
loop(p < s.length()) {
// Skip whitespace (simplified)
// Parse key (must be string)
local ch = s.substring(p, p+1)
if ch != '"' { return null }
// Parse value (simplified)
local key = "key"
local value = "val"
obj.set(key, value)
p = p + 1
// Check for object end or separator
local next_ch = s.substring(p, p+1)
if next_ch == "}" {
return obj // Stop: object complete
}
if next_ch == "," {
p = p + 1
continue // Separator: continue to next key-value pair
}
return null // Error
}
return null
}
}