fix(joinir): Phase 241-EX - Remove hardcoded 'sum' check from Pattern3

Remove legacy hardcoded 'sum' carrier validation that was blocking array_filter patterns with different accumulator names (e.g., 'out'). Before: Pattern3 required carrier named 'sum' to exist After: Pattern3 uses carrier_info generically (any carrier name works) Test results: - phase49_joinir_array_filter_smoke: PASS ✅ - phase49_joinir_array_filter_fallback: PASS ✅ - phase49_joinir_array_filter_ab_comparison: PASS ✅ - Full suite: 909/909 PASS, 0 FAIL Also: Archive old roadmap documentation (67k lines moved to docs/archive/) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 00:48:42 +09:00
parent a7dbc15878
commit 811dfebf98
387 changed files with 106 additions and 5551 deletions
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/CURRENT_TASK.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/CURRENT_TASK.md
@ -0,0 +1,41 @@
+# CURRENT TASK – Phase 11.7（JIT Complete / Semantics Layer）
+
+Focus Window: Semantics導入＋jit-direct安定化の確定報告 → GC/Sync/非同期のJIT降下着手
+
+合意事項（要約）
+
+- LLVM AOT は一旦クローズ。Windows 依存と重量を回避し、Cranelift に集中。
+- 単一意味論層（Semantics）を導入し、Nyashスクリプト/VM/JIT(exe)を同一動作に揃える。
+- VM は参照実装。JIT は実行/生成を担い、VM→JITのランタイムフォールバックは行わない。
+
+現状ステータス（2025-09-01）
+
+- jit-direct 分岐/PHI 合流：単一出口＋BlockParam合流で安定化を確認。
+  - テスト: `mir-branch-ret`, `mir-phi-min`, `mir-branch-multi`, `mir-nested-branch`, `mir-phi-two` で VM/JIT 一致（tag=201/200 一致）。
+- Semantics 層：`src/semantics/{mod.rs, eval.rs}` にトレイトとPoCインタプリタの骨組みを追加済（未配線）。
+- C ABI（NyRT）：`crates/nyrt` の `libnyrt.a` に必要シンボル実装済（console/array/string/plugin_invoke/checkpoint/gc_barrier 等）。
+- VM 側：Safepoint/書込バリア/簡易スケジューラ（SingleThread）連携は稼働。
+- JIT 側：Safepoint/バリア/await はまだスタブまたは未emit（要降下）。
+
+直近タスク（このフェーズでやること）
+
+1) Semantics 実用化配線（VM/JITの動作一致の“芯”）
+   - `SemanticsVM`（VM実行での実装）と `SemanticsClif`（LowerCore+IRBuilder委譲）を用意。
+   - `semantics::MirInterpreter` で両者を同一MIRへ適用し、差分検出の土台を作る。
+2) JIT へ GC/Sync/非同期の降下
+   - Safepoint: `I::Safepoint` を `nyash.rt.checkpoint` emit。`nyrt` 側で `gc.safepoint()` と `scheduler.poll()` に橋渡し。
+   - Write Barrier: Array/Map の set/push 等をlowerする箇所で `nyash.gc.barrier_write` を emit（CountingGc で検証）。
+   - Await: PoC として FutureBox の同期 get にlower（動作一致優先）。
+3) パリティ検証
+   - `NYASH_GC_COUNTING=1` で VM/JIT ともに safepoint/barrier カウントが増えることを確認。
+   - 既存 smokes（分岐/PHI/配列/外部呼び出し）で一致を継続監視。
+
+実行メモ
+
+- Build（JIT）: `cargo build --release --features cranelift-jit`
+- jit-direct: `NYASH_JIT_THRESHOLD=1 ./target/release/nyash --jit-direct <app>`
+- 追跡: `NYASH_JIT_TRACE_RET/SEL/BLOCKS=1`、GC: `NYASH_GC_COUNTING=1`（必要時）
+
+備考
+
+- LLVM AOT のドキュメント/ツールは維持するが、Windows 前提の依存導入は行わない。Cranelift で“がっちり作る”。
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/MEETING_NOTES.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/MEETING_NOTES.md
@ -0,0 +1,46 @@
+# Design Notes – Phase 11.7 JIT Complete (Meeting Summary)
+
+Date: 2025-09-01
+
+Key Decisions
+
+- Single Semantics Source: Introduce a MIR semantics layer (trait) as the single source of truth. All backends (VM/Cranelift/LLVM/WASM) implement this interface.
+- No Runtime Fallback: Remove VM→JIT fallback complexity. VM becomes the reference executor; codegen backends handle execution/AOT. JIT is compile-only/AOT‑assist when needed.
+- Handle‑First ABI: Unify handle/i64/ptr conversions, tag classification, concat/extern/boxcall via shared helpers; call into NyRT shims from backends.
+- GC Hooks: Insert sync barriers and async safepoints as MIR‑level hooks that each backend lowers appropriately.
+- Backends Roadmap: LLVM AOT は一旦アーカイブ。Cranelift を主線（JIT/軽量AOT）とし、WASM は同一セマンティクスで後段。Windows DX を軽く保つ。
+
+Architecture Sketch
+
+- MIR → Semantics<E>
+  - VmSem: executes values (reference)
+  - ClifSem: builds Cranelift IR (JIT/AOT)
+  - LlvmSem: builds LLVM IR (AOT)
+  - WasmSem: builds Wasm (future)
+- Shared ABI utilities: handle↔i64/ptr, to_bool, compare, tags, invoke variants, NyRT shims.
+
+Implementation Plan (Progressive)
+
+- Phase 1: Skeleton + minimal lowering (Const/Return/Add) → echo-lite returns 0 via Cranelift JIT skeleton.
+- Phase 2: Control (Jump/Branch/Phi), Load/Store, Compare, String concat via NyRT, Box/Extern by-id (fixed/vector).
+- Phase 3: GC barriers/safepoints; parity with VM（CountingGc での観測を含む）。
+- Phase 4: Stability, logs, strict/legacy guards; optional AOT via cranelift-object + link scripts.
+
+Notes from Review
+
+- Using Semantics trait enables zero‑cost abstractions with static dispatch.
+- Add optional debug hooks (location/value) and optimization hints (likely_branch, pure_function_hint) later.
+- Testing: MockSemantics for unit tests; parity tests VM vs CLIF.
+
+Action Items
+
+- Land Semantics trait + minimal MirInterpreter (skeleton added).
+- Implement ClifSem minimal lowering; wire Runner `--backend cranelift`.
+- Centralize ABI helpers; migrate existing scattered logic to shared module.
+- Emit `nyash.rt.checkpoint` and `nyash.gc.barrier_write` from LowerCore at appropriate sites; wire Await (PoC: blocking get).
+- Add build_cl scripts for AOT when ready.
+
+Links
+
+- PLAN.md – milestones and coverage
+- CURRENT_TASK.md – immediate focus
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/PLAN.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/PLAN.md
@ -0,0 +1,81 @@
+# Phase 11.7 – JIT Complete Plan (Cranelift)
+
+Goal
+
+- Ship a complete JIT backend (Cranelift) for MIR Core‑15 with Semantics layer as the single source of truth, GC sync/async support, and full Box/Extern integration. Keep DX simple and cross‑platform.
+
+Milestones
+
+1) Bootstrap (Infra + Skeleton)
+- Add backend module: `src/backend/cranelift/{mod.rs,context.rs,lower.rs,jit.rs,object.rs}`
+- Context: host ISA, FunctionBuilder, Module (JIT/Object) setup helpers
+- Runner: `--backend cranelift` execution path; feature flag reuse (`cranelift-jit`)
+- Minimal ops: Const i64/f64/bool/null/string → CLIF values; Return; straight‑line add/sub
+- Smoke: `apps/tests/ny-echo-lite` returns 0 via JIT
+
+2) Core MIR‑15 Lowering (Parity with VM)
+- Control: Jump/Branch/Phi, Load/Store (alloca on entry; i1↔i64 widen/narrow)
+- Unary/Binary/Compare: int/float/ptr eq/ne; logical and/or via to_bool
+- TypeOp (when needed) + pointer/int casts (handle→i64/i8* via ABI helpers)
+- Strings: concat via NyRT shims (`nyash.string.concat_*`)
+- BoxCall (by‑id): fixed/tagged args (<=4) + vector path; handle‑first returns (i64/ptr)
+- ExternCall: `env.console.*`, `env.debug.trace`, `console.readLine` via NyRT shims
+- Smokes: array/map/vinvoke(size/ret), echo; compare outputs with VM
+
+3) GC Cooperation
+- Sync barriers: insert read/write barriers at Load/Store & NewBox as per VM semantics
+- Async safepoints: at call sites/loop backedges; NyRT entry glue to yield if required
+- Tests: targeted barrier smoke (array/map mutations) + perf sanity (no excessive barriers)
+
+4) Parity + Stability
+- UnsupportedLegacyInstruction: maintain strict mode; allow env override for debug
+- Error reporting: source op pretty‑print, MIR value ids in error messages
+- Logging: `NYASH_CLI_VERBOSE=1` shows JIT compile stages + object sizes (optional)
+- Doc/Doctor script: `tools/doctor.ps1`/`.sh` for quick env checks (optional nice‑to‑have)
+
+5) AOT (Optional within 11.7 if time)
+- `cranelift-object` emit: `.o` for main; link with NyRT → exe (`tools/build_cl.*`)
+- Windows: clang/lld link; Linux: cc link; parity with LLVM’s scripts
+
+Instruction Coverage (MIR Core‑15)
+
+- Const, UnaryOp, BinOp, Compare, TypeOp
+- Load, Store, Phi, Jump, Branch, Return
+- Call, NewBox, BoxCall, ExternCall
+
+ABI & Shims
+
+- Handle‑first: receiver/values normalized to i64; ptr/int casts via helpers
+- NyRT shims used: `nyash.console.*`, `nyash.debug.trace`, `nyash.console.readline`,
+  `nyash_string_new`, `nyash.string.concat_*`, `nyash_array_*_h`, `nyash.instance.*_h`,
+  plugin invoke (by‑id tagged, vector variants)
+
+Semantics Integration (new in 11.7)
+
+- Add `Semantics` trait as unified MIR semantics API.
+- Provide `SemanticsVM` (exec) and `SemanticsClif` (lower) so the same MIR walks yield identical behavior across VM and JIT.
+- Use `semantics::MirInterpreter` for parity tests; prefer zero‑cost abstractions with static dispatch.
+
+Status Notes (2025‑09‑01)
+
+- LLVM AOT: closed for now due to Windows dependency weight; Cranelift is the mainline.
+- VM: safepoint/barrier/scheduler wired and observable (CountingGc).
+- JIT: `nyash.rt.checkpoint`/`nyash.gc.barrier_write` symbols are available via NyRT; LowerCore needs to emit safepoints and barriers; Await lowering pending.
+
+Deliverables
+
+- Code: Cranelift backend files + runner integration
+- Tools: build/run scripts for JIT/AOT; updated smokes to exercise CL route
+- Docs: this plan + CURRENT_TASK.md; brief README how to run JIT
+
+Risks & Mitigations
+
+- Pointer/int mismatch → normalize via i64/ptr helpers in one place; add asserts
+- Barrier placement overhead → start with conservative placement; measure; trim if safe
+- Windows toolchain variance → Cranelift avoids external LLVM; keep MSVC only
+
+Timeline (indicative)
+
+- Week 1: Milestone 1 + 2 (most ops) → basic smokes green
+- Week 2: GC barriers + safepoints, full parity sweep; docs/tools polish
+- Optional: AOT via cl‑object emit & link scripts
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/README.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/README.md
@ -0,0 +1,33 @@
+# Phase 11.7 – JIT Complete (MIR-15 + Semantics + Sync/Async GC)
+
+Scope: Implement a production‑ready JIT backend for Nyash that fully supports the MIR Core‑15 instruction set, synchronous/async GC cooperation, and delivers a simple, reproducible developer experience across platforms (WSL/Linux/Windows).
+
+Why now:
+- LLVM AOT は一度実装を進めたが、Windows 依存が重くサイズも大きい。Cranelift に回帰して、純Rustで“Just Works”のDXを取り戻し、高速な反復開発を実現する（AOT/LLVMは資料として保持）。
+
+Outcomes (Definition of Done):
+- All MIR‑15 ops compile and execute via JIT with behavioral parity to VM.
+- BoxCall/ExternCall are handled through NyRT shims (handle‑first ABI) safely.
+- Sync GC barriers in place (read/write), async safepoints wired at call/loop edges.
+- Smokes: echo/array/map/vinvoke/extern pass; parity checks vs VM/JIT (logs included).
+- 1‑command setup and run on WSL + Windows Dev PowerShell; no external LLVM needed.
+
+Backends Strategy:
+- LLVM AOT はアーカイブ（参照は可）。主線は Cranelift（JIT/軽量AOT）。
+- JIT 既定は Cranelift（feature: `cranelift-jit`）。AOT は必要に応じ `cranelift-object` を併用。
+
+This folder contains the living plan (PLAN.md) and the rolling snapshot of the current task focus (CURRENT_TASK.md). Semantics 層の導入により、Nyash スクリプト／VM／JIT（exe）の動作を一致させる。
+
+## JIT Single-Exit Policy and TRACE
+
+- Single-Exit: JIT は関数終端で単一の ret ブロックに合流する方針。分岐合流は BlockParam（最小PHI）で表現し、`end_function` で最終 seal を行う。
+- Branch Fast-Path: then/else がともに i64 定数を即時 return する場合、`select(cond, K_then, K_else)` → `return` に縮約（常時有効）。
+- TRACE 環境変数（必要時のみON）:
+  - `NYASH_JIT_DUMP=1` …… Lower の要約/CFGライトダンプを表示
+  - `NYASH_JIT_TRACE_BLOCKS=1` … ブロック入場ログ（`[JIT-BLOCK] enter=<idx>`）
+  - `NYASH_JIT_TRACE_BR=1` …… br_if の cond 有無ログ
+  - `NYASH_JIT_TRACE_SEL=1` … select の cond/then/else 値（tag=100/101/102）
+  - `NYASH_JIT_TRACE_RET=1` … return 値ログ（tag=201=直前, 200=合流）
+
+Notes:
+- 旧フラグ `NYASH_JIT_FASTPATH_SELECT` は不要になりました（存在しても無視）。
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/PLAN.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/PLAN.md
@ -0,0 +1,59 @@
+# Async Task System — Phased Plan (P1–P3)
+
+## Phase 1: Foundations (stabilize + timeouts)
+
+- FutureBox: switch to Mutex+Condvar (done).
+- Await: poll scheduler + timeout gate in VM and JIT (done; unify to Result.Err in P3).
+- env.future.spawn_instance: enqueue via Scheduler; fallback sync if no scheduler (done).
+- Safepoint: ensure env.runtime.checkpoint is emitted around await (Lowerer rule). 
+- Smokes: async-await-min; async-spawn-instance (safe form, no env).
+- Acceptance:
+  - No hangs (await respects timeout); CPU near idle while waiting.
+  - VM/JIT pass basic smokes; lingering processes do not remain.
+
+## Phase 2: TaskGroup & CancellationToken
+
+- Types:
+  - CancellationToken { cancel(), is_cancelled() } idempotent; parent→child propagation only.
+  - TaskGroup Box { spawn(fn)->Future, cancelAll(), joinAll() }, owns token; fini enforces cancel→join.
+- API:
+  - nowait sugar targets current TaskGroup.
+  - Unsafe detached spawn hidden behind unsafe_spawn_detached() and verifier checks.
+- VM implementation:
+  - Extend scheduler to accept token; tasks periodically check token or are cancelled at await.
+  - Group registry per scope; insert fini hooks in function epilogues and Main.
+- JIT/EXE:
+  - NyRT shims accept optional token handle; if missing, use current group’s token.
+- Smokes:
+  - spawn_cancel_on_scope_exit; nested_groups; lifo_join_order.
+- Acceptance:
+  - Parent exit cancels and joins children deterministically (LIFO); no leaks per leak tracker.
+
+## Phase 3: Error Semantics & Unification
+
+- Future.await returns Result<T, Err> (Timeout/Cancelled/Panic) consistently (VM/JIT).
+- Remove 0/None fallbacks; map shims to Result at Nyash layer.
+- Lowerer verifies checkpoints around await; add verifier rule.
+- Observability: minimal counters and optional traces.
+- Smokes:
+  - await_timeout distinct from cancel; panic_propagation; wakeup_race (no double resolve).
+- Acceptance:
+  - Consistent error surface; result handling identical across VM/JIT/EXE.
+
+## Test Matrix & CI
+
+- Backends: {vm, jit, aot} × Modes: {default, strict}.
+- Smokes kept minimal; time‑bounded via timeout(1) wrapper.
+- CPU spin test: ensure idle waiting; measured via time/ps (best‑effort).
+
+## Migration & Compatibility
+
+- Keep env.future.spawn_instance during transition; TaskGroup.spawn preferred.
+- nowait sugar remains; mapped to TaskGroup.spawn.
+- Document flags: NYASH_AWAIT_MAX_MS, NYASH_SCHED_*.
+
+## Files & Ownership
+
+- Spec & Plan live here; updates linked from CURRENT_TASK.md.
+- Code changes limited to runtime/{scheduler,global_hooks}, boxes/future, jit/extern/async, lowerer await rules.
+
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/README.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/README.md
@ -0,0 +1,14 @@
+# Async Task System (Structured Concurrency) — Overview
+
+Goal: A safe, structured, and portable async task system that runs end‑to‑end across Nyash code → MIR → VM → JIT/EXE.
+
+- Default is safe: tasks are scoped to an owning group; when the owner ends, children cancel and join.
+- Everything is Box: TaskGroup and Future are Boxes; user APIs are Box methods; MIR uses BoxCall.
+- No new MIR ops required: use BoxCall/PluginInvoke consistently; safepoints are inserted around await.
+- Deterministic exits: parent exit triggers cancelAll → joinAll on children (LIFO), eliminating leaks.
+
+This folder contains the spec, phase plan, and test plan:
+
+- SPEC.md: User API, Box contracts, MIR/VM/JIT mapping, ABI, error semantics.
+- PLAN.md: Phased rollout (P1–P3), acceptance gates and checklists.
+
--- a/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/SPEC.md
+++ b/docs/archive/roadmap/phases/phase-11.7_jit_complete/async_task_system/SPEC.md
@ -0,0 +1,83 @@
+# Async Task System — SPEC
+
+Scope: Define a structured concurrency model for Nyash with TaskGroup and Future as Boxes. Implementable across VM and JIT/EXE without adding new MIR instructions.
+
+## User‑Facing API (Everything is Box)
+
+Box TaskGroup
+- spawn(fn: () -> T) -> Future<T>
+- cancelAll() -> void
+- joinAll() -> void
+- fini: must run cancelAll() then joinAll() (LIFO order) to ensure structured shutdown.
+
+Box Future<T>
+- await(timeout_ms?: int) -> Result<T, Err>
+  - Ok(value)
+  - Err(Timeout) | Err(Cancelled) | Err(Panic)
+- cancel() -> void (idempotent)
+
+Sugar
+- nowait v = expr is sugar for: let g = current_group(); v = g.spawn(lambda: expr)
+
+Default Ownership
+- An implicit TaskGroup is created per function scope and for Main. It owns tasks spawned in that scope.
+- Leaving the scope triggers cancelAll→joinAll on its group (LIFO), unless tasks were moved to a longer‑lived group explicitly.
+
+Detachment (discouraged)
+- unsafe_spawn_detached(fn) only in advanced modules. Verifier should disallow use in normal code paths.
+
+## MIR Mapping
+
+- No new MIR instructions. Use existing BoxCall/PluginInvoke forms.
+- TaskGroup.spawn → BoxCall on TaskGroup Box, returns Future Box.
+- Future.await → BoxCall on Future Box with optional timeout parameter.
+- Lowerer inserts safepoint around await: ExternCall env.runtime.checkpoint before and after the await call.
+
+Example Lowering (high level)
+- AST: nowait fut = arr.length()
+- MIR (normalized):
+  - recv = … (arr)
+  - mname = Const("length")
+  - fut = ExternCall iface="env.future", method="spawn_instance", args=[recv, mname]
+  - v = BoxCall Future.await(fut, timeout_ms?)
+
+Note: In the final API, TaskGroup.spawn replaces env.future.spawn_instance, but the MIR contract remains BoxCall/ExternCall‑based.
+
+## VM Semantics
+
+- Scheduler: SingleThreadScheduler initially; spawn enqueues closure in FIFO. safepoint_and_poll() runs due tasks.
+- Future: implemented with Mutex+Condvar; set_result notifies; await waits with optional timeout; on cancel/timeout, returns Result.Err.
+- CancellationToken: parent→child propagation only, idempotent cancel().
+- TaskGroup: holds token and child registry; fini enforces cancelAll→joinAll (LIFO).
+
+## JIT/EXE Semantics
+
+- NyRT C‑ABI Shims:
+  - nyash.future.spawn_method_h(type_id, method_id, argc, recv_h, vals*, tags*) -> i64 (Future handle)
+  - nyash.future.spawn_instance3_i64(a0, a1, a2, argc) -> i64 (Future handle, by name/first args)
+  - nyash.future.await_h(handle, timeout_ms?) -> i64/handle (Result encoding handled at Nyash layer)
+- Await shim must poll safepoints and honor timeout; returns 0 or sentinel; Nyash layer maps to Result.Err.*
+
+## Errors & Results
+
+- Distinguish Timeout vs Cancelled vs Panic.
+- Logging: concise, off by default; env flags can enable traces.
+- No silent fallback: unimplemented paths error early with clear labels.
+
+## GC & Safepoints
+
+- Lowerer must emit env.runtime.checkpoint immediately before and after await calls.
+- Scheduler.poll runs at checkpoints; long loops should explicitly insert checkpoints.
+
+## Configuration
+
+- NYASH_AWAIT_MAX_MS (default 5000) — global default timeout for await when not specified.
+- NYASH_SCHED_POLL_BUDGET — tasks per poll, default 1.
+- NYASH_SCHED_TRACE — prints poll/move/ran counts when 1.
+
+## Security & Determinism
+
+- Structured shutdown prevents orphan tasks after parent exit.
+- LIFO joinAll reduces deadlock surfaces.
+- Detached tasks are explicit and rare.
+