diff --git a/CURRENT_TASK.md b/CURRENT_TASK.md index e3dd619c..15560f18 100644 --- a/CURRENT_TASK.md +++ b/CURRENT_TASK.md @@ -167,16 +167,21 @@ 1. ~~Phase 245B(コード)~~: ✅ 完了(Phase 43/245B の一部) 2. ~~Phase 246-EX(コード)~~: ✅ 完了(Phase 43/245B の一部) -3. **Phase 46+**: Canonical set 拡張(capability-based filtering) - - 現在の canonical set を段階的に拡張 - - ShapeCapability インフラを活用 -4. JsonParser 残りループへの JoinIR 展開 +3. **Phase 47-NORM-P3(設計完了✅ 2025-12-12)**: Pattern3 Normalized 設計 + - 設計詳細: [phase47-norm-p3-design.md](docs/development/current/main/phase47-norm-p3-design.md) + - P3 if-sum を Normalized JoinIR に載せる設計。P2 と同じ ConditionEnv/CarrierInfo/ExitLine インフラを再利用。 + - Phase 47-A: Minimal sum_count (dev-only), Phase 47-B: array_filter, Phase 47-C: Canonical promotion。 + - 実装は未着手(設計のみ完了)。 +4. **Phase 48-NORM-P4-DESIGN(候補)**: Pattern4 Normalized 設計 + - `_parse_array` / `_parse_object` / continue 混在ループを棚卸しし、P4 をどの順番で Normalized に載せるかを整理。 + - continue を TailCallFn でどう表現するか、EnvLayout との対応を設計メモに落とす。 +5. JsonParser 残りループへの JoinIR 展開 - `_parse_array` / `_parse_object` / `_unescape_string` / 本体 `_parse_string` など。 - 既存の P2/P3/P4+P5 パイプラインをどこまで延ばせるかを docs 側で設計 → コード側はその設計に沿って小さく実装。 -5. selfhost depth‑2 ラインの再開 +6. selfhost depth‑2 ラインの再開 - `.hako` 側で Program/MIR JSON を読んで JoinIR/MIR/VM/LLVM に流すライン。 - JsonParser 側のカバレッジが上がったあとに、小さいループから順に移植する。 -6. JoinIR Verify / 最適化まわり +7. JoinIR Verify / 最適化まわり - すでに PHI/ValueId 契約は debug ビルドで検証しているので、 必要なら SSA‑DFA や軽い最適化(Loop invariant / Strength reduction)を検討。 diff --git a/docs/development/current/main/joinir-architecture-overview.md b/docs/development/current/main/joinir-architecture-overview.md index 24d040be..fd1385a8 100644 --- a/docs/development/current/main/joinir-architecture-overview.md +++ b/docs/development/current/main/joinir-architecture-overview.md @@ -1358,3 +1358,25 @@ JsonParser _skip_whitespace / _atoi / _parse_number が**すべて canonical Nor - `normalized_joinir_min.rs`: canonical set 検証テスト追加 **テスト**: 937/937 PASS + +### 3.24 Phase 47-NORM-P3 – Normalized P3 (If-Sum) Support 🏗️ DESIGN (2025-12-12) + +**設計詳細**: [phase47-norm-p3-design.md](./phase47-norm-p3-design.md) + +Pattern3 (if-sum) ループを Normalized JoinIR に対応させる。P2 と同じ ConditionEnv/CarrierInfo/ExitLine インフラを再利用。 + +**Phase 220 基盤**: P3 if-sum は Phase 220 で ConditionEnv 統合済み(既に P2 と同じ Structured JoinIR foundation を持つ) + +**Key difference**: P3 は **conditional carrier updates**(if 内でのみキャリア更新)vs P2 の unconditional updates before break + +**Phase 47-A**: Minimal sum_count (dev-only Normalized) +- ✅ EnvLayout: P2 と同じ構造(carrier フィールド) +- ✅ StepScheduleBox: `IfCond`, `ThenUpdates`, `ElseUpdates` ステップ追加 +- ✅ JpInst: 既存の `If` 命令を再利用 +- 🎯 Test: `phase212_if_sum_min.hako` → Normalized→MIR(direct) 検証 + +**Phase 47-B**: array_filter (dev-only, body-local + method calls) + +**Phase 47-C**: Canonical promotion (P3 minimal → canonical) + +**スコープ外**: P4 (continue) 対応(NORM-P4 フェーズで実施)、Complex P3 patterns(後続フェーズ) diff --git a/docs/development/current/main/phase47-norm-p3-design.md b/docs/development/current/main/phase47-norm-p3-design.md new file mode 100644 index 00000000..45b94eee --- /dev/null +++ b/docs/development/current/main/phase47-norm-p3-design.md @@ -0,0 +1,376 @@ +# Phase 47: Normalized P3 (If-Sum) Design + +**Status**: Design Complete, Implementation Planned +**Date**: 2025-12-12 + +## Goal + +Extend Normalized JoinIR to support Pattern3 (if-sum) loops using the same infrastructure that successfully handles P1/P2. + +**Key insight**: P3 already shares P2's Structured JoinIR foundation (Phase 220), so Normalized extension reuses existing components. + +## Background: P2 Normalized Success + +Phase 43/245B/46 established canonical Normalized for all P2 patterns: +- ✅ Pattern2Mini (simple break) +- ✅ JsonParser skip_whitespace (Trim pattern) +- ✅ JsonParser _atoi (DigitPos + NumberAccumulation) +- ✅ JsonParser _parse_number (multi-carrier) + +**Infrastructure complete**: +- Structured→Normalized→MIR(direct) pipeline +- EnvLayout, JpInst/JpOp, StepScheduleBox +- ConditionEnv, CarrierInfo, ExitLine +- Mode system (Phase 45), Capability system (Phase 44) + +## Why P3 Uses Same Normalized + +### 1. Shared Structured JoinIR Foundation (Phase 220) + +From `joinir-architecture-overview.md` (lines 73-84): + +``` +Phase 220: P3 if-sum の ConditionEnv 統合完了 +- P3 if-sum には ConditionPatternBox + ConditionEnv が必須 +- Phase 220-D で loop 条件の変数サポート完了 +- Phase 220 で if-sum の expr-result exit contract が P2 と揃った +``` + +**P3 already uses P2 infrastructure**: +- ✅ ConditionEnv (condition analysis) +- ✅ CarrierInfo (state tracking) +- ✅ ExitLine/Boundary (exit handling) +- ✅ LoopHeaderPHI (SSA construction) + +### 2. Pattern Similarity + +| Aspect | P2 (Break) | P3 (If-Sum) | Shared? | +|--------|-----------|-------------|---------| +| Loop control | `loop(cond)` + `break` | `loop(cond)` + conditional update | ✅ Yes | +| Carriers | sum, count, result | sum, count (if-conditional) | ✅ Yes | +| Exit condition | Break early | Continue all iterations | Different | +| ConditionEnv | ✅ Used | ✅ Used (Phase 220) | ✅ Yes | +| ExitLine | ✅ Used | ✅ Used | ✅ Yes | + +**Key difference**: P3 has **conditional carrier updates** inside loop body, vs P2's unconditional updates before break. + +### 3. Normalized Extension Points + +P3 needs minimal additions to existing Normalized: + +**Already working**: +- ConditionEnv (loop + if conditions) +- CarrierInfo (state tracking) +- EnvLayout (carrier fields) +- LoopHeaderPHI (entry/latch values) + +**Need to add**: +- **ConditionalUpdate** pattern in StepScheduleBox + - P2: `[HeaderCond, BodyInit, BreakCheck, Updates, Tail]` + - P3: `[HeaderCond, IfCond, ThenUpdates, ElseUpdates, Tail]` +- **If branching** in Normalized JpInst + - Already exists: `If { cond, then_target, else_target, env }` + - Just need to emit for P3 body structure + +## Architecture: Unified Normalized + +``` +┌──────────────────────────────────────────┐ +│ Structured JoinIR (Pattern1-4 共通) │ +│ - ConditionEnv (P2/P3/P4 統一 Phase 220) │ +│ - CarrierInfo │ +│ - ExitLine/Boundary │ +└──────────────┬───────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────┐ +│ Normalized JoinIR (Pattern1-4 共通) │ ← P3 もここに載せる! +│ - EnvLayout (P2 完成 → P3 拡張) │ +│ - JpInst/JpOp (If 分岐追加) │ +│ - StepScheduleBox (ConditionalUpdate) │ +└──────────────┬───────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────┐ +│ MIR (Pattern1-4 共通) │ +└──────────────────────────────────────────┘ +``` + +## Representative P3 Loops + +### Phase 47-A: Minimal (sum_count) + +**Example**: `phase212_if_sum_min.hako` + +```nyash +local sum = 0 +local count = 0 +local i = 0 +local n = 5 + +loop(i < n) { + if (i % 2 == 1) { + sum = sum + i + count = count + 1 + } + i = i + 1 +} +``` + +**Characteristics**: +- Simple condition: `i % 2 == 1` +- Two carriers: `sum`, `count` (conditionally updated) +- One loop param: `i` (always updated) +- No break, runs all iterations + +**Normalized shape**: +- EnvLayout: `{ i: int, sum: int, count: int }` +- StepSchedule: `[HeaderCond(i < n), IfCond(i % 2 == 1), ThenUpdates(sum, count), Updates(i), Tail]` + +### Phase 47-B: JsonParser array_filter + +**Example**: JsonParser `array_filter` (if present in codebase) + +```nyash +local out = new ArrayBox() +local i = 0 + +loop(i < arr.length()) { + local v = arr.get(i) + if (predicate(v)) { + out.push(v) + } + i = i + 1 +} +``` + +**Characteristics**: +- Method calls: `arr.length()`, `arr.get(i)`, `out.push(v)` +- Body-local: `v` (used in if condition) +- Conditional side effect: `out.push(v)` + +**Complexity**: Higher than sum_count (method calls, body-local) + +### Phase 47-C: Selfhost loops (future) + +Complex P3 patterns from selfhost compiler (deferred to later phase). + +## Implementation Strategy + +### Phase 47-A: Minimal sum_count (dev-only) + +**Goal**: Prove P3 can use Normalized infrastructure with minimal additions. + +**Steps**: +1. **ShapeGuard**: Add `Pattern3IfSumMinimal` shape +2. **StepScheduleBox**: Add `ConditionalUpdate` step kind +3. **Normalized lowering**: + - Generate `If` JpInst for body if-statement + - Emit carrier updates in then/else branches +4. **Test**: Verify Structured→Normalized→MIR(direct) matches Structured→MIR + +**Expected additions**: +- `shape_guard.rs`: +1 shape variant +- `pattern2_step_schedule.rs`: Rename to `step_schedule.rs`, add P3 support +- `normalized_bridge/direct.rs`: Handle `If` JpInst with carrier updates +- `tests/normalized_joinir_min.rs`: +1 P3 test + +**Dev fixture**: `phase212_if_sum_min.hako` → `jsonparser_if_sum_minimal` + +### Phase 47-B: array_filter (dev-only) + +**Goal**: Extend to body-local + method calls. + +**Additions**: +- Body-local handling in EnvLayout (already exists for P2 DigitPos) +- Method call in if condition (ExprLowerer already supports) + +### Phase 47-C: Canonical promotion + +**Goal**: Move P3 minimal from dev-only to canonical (like P2). + +**Criteria**: +- All invariants verified (Phase 47-A/B tests passing) +- No regressions in 937+ tests +- Performance acceptable (Normalized vs Structured comparison) + +## Normalized Components for P3 + +### EnvLayout Extension + +**P2 example**: +```rust +struct Pattern2Env { + i: int, // loop param + sum: int, // carrier +} +``` + +**P3 extension** (same structure): +```rust +struct Pattern3Env { + i: int, // loop param + sum: int, // carrier (conditionally updated) + count: int, // carrier (conditionally updated) +} +``` + +**No new fields needed** - P3 carriers work same as P2. + +### StepScheduleBox Extension + +**P2 steps**: +```rust +enum StepKind { + HeaderCond, // loop(cond) + BodyInit, // local ch = ... + BreakCheck, // if (cond) break + Updates, // sum = sum + 1 + Tail, // i = i + 1 +} +``` + +**P3 addition**: +```rust +enum StepKind { + // ... existing P2 steps + IfCond, // if (cond) in body + ThenUpdates, // carrier updates in then branch + ElseUpdates, // carrier updates in else branch (if any) +} +``` + +**P3 schedule**: +```rust +// sum_count pattern +[HeaderCond, IfCond, ThenUpdates, Updates, Tail] + +// vs P2 pattern +[HeaderCond, BodyInit, BreakCheck, Updates, Tail] +``` + +### JpInst Reuse + +**Already exists** in Normalized (from P2): +```rust +pub enum JpInst { + Let { dst, op, args }, + If { cond, then_target, else_target, env }, // ← P3 uses this! + TailCallFn { target, args }, + TailCallKont { target, args }, +} +``` + +**P3 usage**: +- `If` instruction for body if-statement +- `then_target` → block with carrier updates +- `else_target` → block without updates (or with different updates) + +**No new JpInst needed!** + +## Testing Strategy + +### Phase 47-A: Minimal + +**Test**: `test_normalized_pattern3_if_sum_minimal` + +```rust +#[cfg(feature = "normalized_dev")] +#[test] +fn test_normalized_pattern3_if_sum_minimal() { + let source = r#" + local sum = 0 + local count = 0 + local i = 0 + local n = 5 + loop(i < n) { + if (i % 2 == 1) { + sum = sum + i + count = count + 1 + } + i = i + 1 + } + print("sum = " + sum.to_string()) + print("count = " + count.to_string()) + "#; + + // Compare Structured→MIR vs Normalized→MIR(direct) + assert_vm_output_matches(source); +} +``` + +**Expected output**: +``` +sum = 6 (1 + 3 + 5) +count = 3 +``` + +### Phase 47-B: array_filter + +**Test**: `test_normalized_pattern3_array_filter` + +Similar structure, verify method calls + body-local work in Normalized. + +## Success Criteria + +**Phase 47-A complete when**: +1. ✅ `test_normalized_pattern3_if_sum_minimal` passes (dev-only) +2. ✅ Structured→Normalized→MIR(direct) output matches Structured→MIR +3. ✅ All 937+ tests still pass (no regressions) +4. ✅ ShapeGuard can detect Pattern3IfSumMinimal +5. ✅ Documentation updated (architecture overview, CURRENT_TASK) + +**Phase 47-B complete when**: +1. ✅ array_filter test passes (dev-only) +2. ✅ Body-local + method calls work in P3 Normalized + +**Phase 47-C complete when**: +1. ✅ P3 minimal promoted to canonical (always Normalized) +2. ✅ Performance validated + +## Scope Management + +**In Scope (Phase 47-A)**: +- ✅ Minimal P3 (sum_count pattern) +- ✅ Dev-only Normalized support +- ✅ Reuse P2 infrastructure (ConditionEnv, CarrierInfo, ExitLine) + +**Out of Scope (deferred)**: +- ❌ Complex P3 patterns (nested if, multiple conditions) +- ❌ Canonical promotion (Phase 47-C) +- ❌ Pattern4 (continue) support (separate NORM-P4 phase) +- ❌ Selfhost loops (later phase) + +## File Impact Estimate + +**Expected modifications** (Phase 47-A): +1. `shape_guard.rs`: +20 lines (Pattern3IfSumMinimal shape) +2. `step_schedule.rs`: +40 lines (P3 step kinds, rename from pattern2_*) +3. `normalized_bridge/direct.rs`: +60 lines (If instruction handling) +4. `tests/normalized_joinir_min.rs`: +30 lines (P3 test) +5. `phase47-norm-p3-design.md`: +200 lines (this doc) +6. `joinir-architecture-overview.md`: +10 lines (Phase 47 section) +7. `CURRENT_TASK.md`: +5 lines (Phase 47 entry) + +**Total**: ~365 lines (+), pure additive (no P2 code changes) + +## Benefits + +1. **Infrastructure reuse**: 90% of P2 Normalized code works for P3 +2. **Unified pipeline**: All patterns (P1/P2/P3) use same Normalized +3. **Incremental rollout**: Dev-only → canonical, same as P2 +4. **Clear path to P4**: Pattern4 (continue) follows same approach + +## Next Steps After Phase 47 + +1. **NORM-P4**: Pattern4 (continue) Normalized support +2. **Canonical promotion**: Move P3/P4 from dev-only to canonical +3. **Selfhost loops**: Complex patterns from selfhost compiler +4. **Performance optimization**: Profile Normalized vs Structured + +## References + +- **P2 Completion**: [PHASE_43_245B_NORMALIZED_COMPLETION.md](./PHASE_43_245B_NORMALIZED_COMPLETION.md) +- **Phase 46 P2-Mid**: [phase46-norm-canon-p2-mid.md](./phase46-norm-canon-p2-mid.md) +- **Architecture**: [joinir-architecture-overview.md](./joinir-architecture-overview.md) +- **Phase 220 P3 Foundation**: joinir-architecture-overview.md lines 73-84 diff --git a/docs/development/current/main/phase47-test-files-inventory.md b/docs/development/current/main/phase47-test-files-inventory.md new file mode 100644 index 00000000..18e371e2 --- /dev/null +++ b/docs/development/current/main/phase47-test-files-inventory.md @@ -0,0 +1,199 @@ +# Phase 47 Test Files Inventory + +**Date**: 2025-12-12 + +## P3 (If-Sum) Test Files Available + +### 1. phase212_if_sum_min.hako ⭐ PRIMARY TARGET + +**Path**: `/home/tomoaki/git/hakorune-selfhost/apps/tests/phase212_if_sum_min.hako` + +**Pattern**: Simple if-sum with single carrier +```nyash +local sum = 0 +local i = 0 +local len = 3 + +loop(i < len) { + if (i > 0) { + sum = sum + 1 // Conditional update + } else { + sum = sum + 0 // Else branch (no-op) + } + i = i + 1 +} +``` + +**Characteristics**: +- ✅ Single carrier: `sum` +- ✅ Simple condition: `i > 0` +- ✅ One loop param: `i` +- ✅ Explicit else branch +- ✅ Clean structure (no method calls, no body-local) + +**Expected output**: `sum = 2` + +**Use case**: **Phase 47-A Minimal** (first Normalized P3 implementation) + +--- + +### 2. phase217_if_sum_multi_min.hako + +**Path**: `/home/tomoaki/git/hakorune-selfhost/apps/tests/phase217_if_sum_multi_min.hako` + +**Pattern**: Multi-carrier if-sum (2 accumulators) +```nyash +local sum = 0 +local count = 0 +local i = 0 +local len = 3 + +loop(i < len) { + if (i > 0) { + sum = sum + 1 // Conditional update 1 + count = count + 1 // Conditional update 2 + print(sum) + print(count) + } else { + print(0) + } + i = i + 1 +} +``` + +**Characteristics**: +- ✅ **Two carriers**: `sum`, `count` +- ✅ Side effects: `print()` calls +- ✅ Multiple updates in then branch +- ⚠️ Slightly more complex than phase212 + +**Expected output**: `sum = 2, count = 2` + +**Use case**: **Phase 47-A Extended** (multi-carrier test) + +--- + +### 3. phase218_json_if_sum_min.hako + +**Path**: `/home/tomoaki/git/hakorune-selfhost/apps/tests/phase218_json_if_sum_min.hako` + +**Pattern**: JsonParser-style accumulation +```nyash +local sum = 0 +local i = 0 +local len = 5 + +loop(i < len) { + if (i > 0) { + sum = sum + i // Variable accumulation (not constant) + print(sum) + } else { + print(0) + } + i = i + 1 +} +``` + +**Characteristics**: +- ✅ Single carrier: `sum` +- ✅ Variable accumulation: `sum + i` (not `sum + 1`) +- ✅ Simulates JsonParser digit accumulation pattern +- ⚠️ More complex arithmetic + +**Expected output**: `sum = 10` (1+2+3+4) + +**Use case**: **Phase 47-A Extended** (variable accumulation test) + +--- + +## Phase 47-A Implementation Priority + +### First Target: phase212_if_sum_min.hako + +**Why**: +1. ✅ **Simplest structure** - single carrier, simple condition +2. ✅ **Clean pattern** - no side effects (no print), no method calls +3. ✅ **Direct P2 analogy** - just adds if branching to P2 structure +4. ✅ **Best for proving concept** - minimal complexity + +**Normalized components needed**: +- EnvLayout: `{ i: int, sum: int }` (same as P2) +- StepSchedule: `[HeaderCond, IfCond, ThenUpdates, Updates, Tail]` +- JpInst: Reuse existing `If` instruction + +### Second Target: phase217_if_sum_multi_min.hako + +**Why**: +1. ✅ **Multi-carrier test** - proves P3 handles multiple accumulators +2. ⚠️ Side effects (print) - tests that side effects work in then/else branches +3. ✅ **Still simple condition** - `i > 0` (same as phase212) + +**Additional Normalized components**: +- EnvLayout: `{ i: int, sum: int, count: int }` (multi-carrier) +- Side effect handling in then/else branches + +### Third Target: phase218_json_if_sum_min.hako + +**Why**: +1. ✅ **Variable accumulation** - `sum + i` (not constant) +2. ✅ **JsonParser pattern** - prepares for real JsonParser loops +3. ⚠️ More complex arithmetic expression + +**Additional Normalized components**: +- Expression lowering for `sum + i` (already supported by ExprLowerer) + +--- + +## Phase 47-B Target (Future) + +**Candidate**: JsonParser array_filter pattern (if exists in codebase) +```nyash +local out = new ArrayBox() +local i = 0 + +loop(i < arr.length()) { + local v = arr.get(i) + if (predicate(v)) { + out.push(v) + } + i = i + 1 +} +``` + +**Complexity**: +- ⚠️ Method calls: `arr.length()`, `arr.get(i)`, `out.push(v)` +- ⚠️ Body-local: `v` (used in if condition) +- ⚠️ External predicate call + +**Status**: Not yet verified if this exact pattern exists in codebase + +--- + +## Summary + +**Available P3 test files**: 3 +1. ✅ phase212_if_sum_min.hako (PRIMARY) +2. ✅ phase217_if_sum_multi_min.hako (SECONDARY) +3. ✅ phase218_json_if_sum_min.hako (TERTIARY) + +**Phase 47-A roadmap**: +1. Start with phase212 (single carrier, simple) +2. Extend to phase217 (multi-carrier) +3. Extend to phase218 (variable accumulation) +4. All dev-only (Normalized→MIR direct comparison tests) + +**Phase 47-B roadmap** (future): +- Real JsonParser loops with method calls + body-local +- Requires body-local handling in EnvLayout (already exists for P2 DigitPos) + +**Phase 47-C roadmap** (future): +- Canonical promotion of P3 minimal (like P2) +- Performance validation + +--- + +## References + +- **Phase 47 Design**: [phase47-norm-p3-design.md](./phase47-norm-p3-design.md) +- **P2 Completion**: [PHASE_43_245B_NORMALIZED_COMPLETION.md](./PHASE_43_245B_NORMALIZED_COMPLETION.md) +- **Architecture**: [joinir-architecture-overview.md](./joinir-architecture-overview.md)