Phase 10.7 - JIT統計とイベント機能の完成

主要な実装: - PHI(b1)統計追跡: phi_total_slots/phi_b1_slotsをJSON出力 - 関数単位統計API: JitStatsBox.perFunction()で詳細統計取得 - JITイベントシステム: compile/execute/fallback/trapをJSONL形式で記録 - Store/Load命令対応: ローカル変数を含む関数のJIT実行が可能に新しいBox: - JitStatsBox: JIT統計の取得 - JitConfigBox: JIT設定の管理（将来用） - JitEventsBox: イベントのJSONL出力（将来用） - JitPolicyBox: 実行ポリシー管理（将来用） CLI拡張: - --jit-exec, --jit-stats, --jit-dump等のフラグ追加 - --jit-directモードでの独立JIT実行 - NYASH_JIT_*環境変数によるきめ細かい制御ドキュメント: - Phase 10.7実装計画の詳細化 - Phase 10.9 (ビルトインBox JIT) の計画追加 - JIT統計JSONスキーマ v1の仕様化 ChatGPT5との共同開発により、JIT基盤が大幅に強化されました。次はPhase 10.9でビルトインBoxのJIT対応を進め、 Python統合（Phase 10.1）への道を開きます。 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-28 09:26:58 +09:00
parent 99e59e24e2
commit e54561e69f
64 changed files with 4311 additions and 189 deletions
--- a/docs/development/roadmap/phases/phase-10/phase_10_7_master_plan.txt
+++ b/docs/development/roadmap/phases/phase-10/phase_10_7_master_plan.txt
@ -0,0 +1,135 @@
+Phase 10.7 — JIT CFG/PHI/ABI Hardening (Master Plan)
+
+Intent
+- Deliver a practical, observable JIT path for control flow and minimal data flow while keeping VM as the safe fallback.
+- Decompose into small, shippable sub-phases with flags and examples, minimizing blast radius.
+
+Related docs
+- 10.7a details: phase_10_7a_jit_phi_cfg_and_abi_min.txt
+- Current work log: docs/development/current/CURRENT_TASK.md
+- Cranelift JIT backend notes: phase_10_cranelift_jit_backend.md
+
+Flags and CLI (common across 10.7)
+- Env: NYASH_JIT_EXEC, NYASH_JIT_STATS, NYASH_JIT_STATS_JSON, NYASH_JIT_DUMP, NYASH_JIT_THRESHOLD, NYASH_JIT_PHI_MIN, NYASH_JIT_HOSTCALL
+- CLI: --jit-exec --jit-stats --jit-stats-json --jit-dump --jit-threshold N --jit-phi-min --jit-hostcall
+
+Examples to validate
+- examples/jit_branch_demo.nyash
+- examples/jit_loop_early_return.nyash
+- examples/jit_phi_demo.nyash
+- examples/jit_array_param_call.nyash, jit_map_param_call.nyash (when hostcall is enabled)
+
+Sub-phases (10.7a → 10.7h)
+
+10.7a — Minimal branch/PHI/ABI (Done or in-flight)
+- Branch wiring with b1 and i64!=0 normalization
+- Minimal PHI: single-value diamond via block param
+- Independent ABI (JitValue) with integer-first calling
+- Panic-safe dispatch (catch_unwind) and VM fallback
+- Minimal hostcall bridge (Array/Map) behind NYASH_JIT_HOSTCALL
+- Observability: unified JIT summary/JSON, lower summary (argc/phi_min), CFG edge dump (phi_min)
+
+Acceptance: see 10.7a doc
+
+10.7b — PHI generalization and block-parameterization
+- Support multiple PHIs in a merge block; pass N values via block params
+- LowerCore: gather PHI inputs per successor, produce explicit arg lists; handle straight merges and simple loops
+- IRBuilder API formalization:
+  - ensure_block_params_i64(index, count)
+  - br_if_with_args(then_idx, else_idx, then_n, else_n)
+  - jump_with_args(target_idx, n)
+- CraneliftBuilder: append N I64 params per block, materialize current block params onto stack when requested
+- Validation: multi-PHI diamonds, if-else chains, early returns; keep loops as simple as possible in 10.7b
+
+10.7c — Independent host-handle registry (JIT↔Host decoupling)
+- Introduce JitHandleRegistry: u64 ↔ Arc<dyn NyashBox>
+- JIT hostcalls accept/return only handles and PODs; VMValue↔JitValue boundary converts BoxRef <-> Handle via registry
+- Rooting policy: enter/pin roots for handles crossing the boundary; leave on return
+- Flags: NYASH_JIT_HOSTCALL=1 (still gated); add NYASH_JIT_HANDLE_DEBUG for dumps
+- Deliverables: array/map operations via handles, no direct VMValue access on JIT side
+
+10.7d — Side-effect boundary and hostcall coverage expansion
+- Keep side-effecting ops (print/IO) on VM for now; document deopt path
+- Expand safe hostcalls (read-only or confined):
+  - String.len / slice basics (POD returns)
+  - Map.size; Map.get with integer and string keys (key boxing handled at boundary)
+- Record effect categories on lowered calls for future optimizer (metadata only)
+
+10.7e — CFG diagnostics and visualization
+- Add DOT export for CLIF CFG with block params and PHI bindings
+- Env: NYASH_JIT_DOT=path.dot (produces per-function graph)
+- Optional: ASCII CFG summary for CI logs
+- Cross-check with VM-side MIR printer for block and PHI consistency
+
+10.7f — JitConfigBox (configuration as a Box)
+- Encapsulate all JIT toggles into a runtime-managed box (JitConfigBox)
+- Harmonize env/CLI with programmatic overrides for tests
+- Feed config into JIT engine and lowerers (no direct env reads inside hot paths)
+- Serialization: dump/load config JSON for reproducible runs
+
+10.7g — Stability, tests, and benchmarks
+- Golden tests: ensure JIT/VM outputs match on curated programs
+- Fallback ratio regression guard: alert when fallback_rate spikes beyond threshold
+- JSON schema stability for stats; include version field
+- Microbenchmarks: branch-heavy, phi-heavy, and hostcall-heavy cases; gated perf checks
+
+10.7h — Type widening for native ABI
+- Add native F64 and Bool parameter/return paths in CLIF
+- Condition handling: keep b1 where possible; map Bool to b1 cleanly
+- Conversions: explicit i64<->f64 ops in lowerer where needed (no silent truncation)
+- Update adapter and closure trampoline to select proper function signatures (arity×type shapes)
+
+Out of scope for 10.7 (shift to 10.8/10.9)
+- Global register allocation strategies or codegen-level optimizations
+- Deoptimization machinery beyond simple VM fallback
+- Advanced exception propagation across JIT/VM boundary
+
+Risks and mitigation
+- PHI N-arity correctness: introduce targeted unit tests over synthetic MIR
+- Handle registry leaks: counting diagnostics, strict mode; tie roots to scope regions
+- CLIF block-param misuse: deterministic block order + seal discipline + assertions in builder
+
+Verification checklist (roll-up)
+- cargo check (± --features cranelift-jit)
+- Run examples with JIT flags; diff with pure VM runs
+- Inspect: unified JIT summary/JSON, lower logs, CFG dumps/DOT
+- Leak/roots checks when NYASH_GC_TRACE=1/2/3 and strict barrier mode is on
+
+Suggested timeline (tentative)
+- 10.7a: Minimal branch/PHI/ABI (done / in-flight)
+- 10.7b: PHI generalization + builder API formalization (1–2 days)
+- 10.7c: Host-handle registry PoC (1–2 days)
+- 10.7d/e: Hostcall coverage + CFG DOT (2–3 days)
+- 10.7f: JitConfigBox + integration (1 day)
+- 10.7g/h: QA + type widening for f64/bool (2–4 days)
+
+10.7z — Follow-ups and Open Items (post-10.7)
+- b1 PHI tagging robustness
+  - Problem: Provenance can be obscured by Load/Store and multi-step copies; (b1) tag may be missed in dumps.
+  - Action:
+    - Add a lightweight boolean-lattice analysis over MIR to classify boolean-producing values independent of path shape.
+    - Extend dump to include phi_summary JSON or structured rows when NYASH_JIT_STATS_JSON is on.
+  - Placement: 10.7g (stability/tests) — does not block 10.7 close.
+
+- VM f64 arithmetic/compare parity
+  - Problem: VM backend currently errors on f64 BinOp/Compare; JIT (Cranelift) supports f64 when enabled.
+  - Action: Implement f64 ops in VM or add consistent auto-promotion; add golden tests.
+  - Placement: 10.8 (VM parity/perf) — out-of-scope for 10.7.
+
+- Native b1 ABI in function signatures
+  - Problem: Toolchain capability for b1 return/params is currently disabled.
+  - Action: Keep centralized switch; add CI probe to flip automatically when supported; wire return path fully.
+  - Placement: 10.7h or later (gated by toolchain).
+
+- Stats/diagnostics polish
+  - Action: Version the unified JIT JSON schema; expose phi(b1) slot counts in JSON; enrich JitStatsBox with summary/topN (partially done).
+  - Placement: 10.7g.
+
+- Build warnings (unexpected cfg: llvm)
+  - Problem: Warning noise from unused/unknown cfg.
+  - Action: Declare feature flags in Cargo.toml or gate code behind existing features; optionally silence for non-supported builds.
+  - Placement: 10.7g (cleanup) — non-blocking.
+
+- Documentation sync
+  - Action: Add a "JIT quick flags" section with common env/CLI combos; ensure CURRENT_TASK and examples remain aligned.
+  - Placement: 10.7e (docs) — non-blocking.
--- a/docs/development/roadmap/phases/phase-10/phase_10_7a_jit_phi_cfg_and_abi_min.txt
+++ b/docs/development/roadmap/phases/phase-10/phase_10_7a_jit_phi_cfg_and_abi_min.txt
@ -0,0 +1,92 @@
+Phase 10.7a — JIT Branch/PHI/Independent ABI: Minimal Path Hardening (Plan)
+
+Purpose
+- Solidify the first functional slice of the JIT control-flow path:
+  - Branch wiring (b1 and i64!=0 normalization)
+  - Minimal PHI via block parameters (single-value diamond)
+  - Independent ABI (JitValue) and safe dispatch + fallback
+  - Observable, toggleable, and easy to verify via flags + examples
+
+Scope (10.7a only)
+1) Minimal PHI handoff
+   - LowerCore: detect single-PHI merge; pass i64 value via block param
+   - CraneliftBuilder: ensure block param (I64), push/pop on branch/jump with args
+   - Flag gate: NYASH_JIT_PHI_MIN=1
+
+2) Branch wiring and condition normalization
+   - Keep compare result on stack as b1 when possible
+   - If top-of-stack is I64, normalize by icmp_imm != 0
+   - Implement br_if/jump to pre-created blocks (deterministic order)
+
+3) Independent ABI (minimum viable)
+   - JitValue(I64/F64/Bool/Handle) in/out; normalize non-i64 args to i64 for now
+   - TLS split: legacy VM args (for hostcalls) and JIT args
+   - Engine.execute_handle uses catch_unwind; panic → VM fallback with stats
+
+4) Minimal hostcall bridge (safe; off by default)
+   - Symbols: nyash.array.{len,get,set,push}, nyash.map.{size}
+   - Gate: NYASH_JIT_HOSTCALL=1
+   - Only integer indices/values (PoC); other types map to handles later
+
+5) Observability and ergonomics
+   - Flags: NYASH_JIT_EXEC, NYASH_JIT_STATS, NYASH_JIT_STATS_JSON, NYASH_JIT_DUMP, NYASH_JIT_THRESHOLD, NYASH_JIT_PHI_MIN, NYASH_JIT_HOSTCALL
+   - CLI: --jit-exec --jit-stats --jit-stats-json --jit-dump --jit-threshold N --jit-phi-min --jit-hostcall
+   - Unified JIT summary on VM exit (sites/compiled/hits/exec_ok/trap/fallback_rate)
+   - Lower log includes argc/phi_min + CFG light dump (phi edges) when NYASH_JIT_DUMP=1
+
+Non-Goals (later 10.7b+)
+- Full PHI generalization (multiple values, loops, complex CFG forms)
+- Non-i64 native path (true F64/Bool return/params in CLIF)
+- Side-effect instruction lowering (print, IO) — keep in VM path
+- Host handle registry for real object bridging (u64↔Arc<dyn NyashBox>)
+
+Deliverables
+- Working minimal PHI + branch JIT execution on curated examples:
+  - examples/jit_branch_demo.nyash
+  - examples/jit_loop_early_return.nyash
+  - examples/jit_phi_demo.nyash (single-PHI diamond)
+- Fallback correctness: traps/panic → VM path; results match VM
+- Configurable via CLI flags; metrics visible via JIT summary/JSON
+
+Acceptance Criteria
+- With: --backend vm --jit-exec --jit-stats --jit-threshold 1
+  - For branch/phi examples, JIT executes without panic
+  - VM fallback occurs only for unsupported ops (logged)
+  - JIT summary shows exec_ok > 0 and reasonable fallback_rate
+- With: --jit-phi-min
+  - CFG dump lists phi_min edges and blocks count
+  - Results match the same program without JIT enabled
+
+Risk & Mitigation
+- Mismatch between VMValue and JitValue adapters
+  - Mitigation: normalize non-i64 to i64 defensively; expand tests on adapters
+- Cranelift block parameter misuse
+  - Mitigation: deterministic block order and explicit ensure_block_param_i64()
+- Panic in hostcall stubs (unexpected Box types)
+  - Mitigation: gated by NYASH_JIT_HOSTCALL=1; default off; fallback to VM on panic
+
+Verification Checklist
+- cargo check (w/ and w/o --features cranelift-jit)
+- Run examples with JIT flags; compare outputs with pure VM
+- Inspect logs: lower summary + CFG dump + unified summary/JSON
+
+Timeline (10.7a)
+Day 1:
+- Finalize ABI normalization and branch wiring; add unified stats (done)
+- Wire CLI flags and JSON stats (done)
+Day 2:
+- Harden minimal PHI path and CFG dump (done)
+- Curate examples and sanity-run on flags
+Day 3:
+- Stabilize logging format, trim rough edges, doc polish
+→ Then roll into 10.7b (multi-PHI, multi-arg block params, real handle registry)
+
+Follow-ups (10.7b/10.7c seeds)
+- Host handle registry (u64↔Arc) and type-safe bridging
+- True F64/Bool native ABI, multi-arg params/returns
+- CFG visualization improvements (dot export) and JitConfigBox
+
+Refs
+- docs/development/current/CURRENT_TASK.md (10_7 items)
+- src/jit/{lower,engine,manager,abi,rt}/
+- examples: jit_* demos
--- a/docs/development/roadmap/phases/phase-10/phase_10_7h_native_abi_types.md
+++ b/docs/development/roadmap/phases/phase-10/phase_10_7h_native_abi_types.md
@ -0,0 +1,45 @@
+# Phase 10.7h — Native ABI Types (F64/Bool)
+
+Goal
+- Extend the minimal i64-only JIT ABI to support f64 and bool as native parameter/return types in CLIF.
+
+Principles
+- Keep JIT independent from VM internals (use JitValue + adapters at boundary)
+- Avoid silent truncation; perform explicit conversions in the lowerer
+- Maintain safety-first fallback to VM for unsupported ops
+
+Plan
+1) JitValue widening
+   - JitValue already has I64/F64/Bool/Handle — keep this as the ABI surface
+   - Adapter: refine to/from VMValue mappings (no lossy coercion by default)
+
+2) CLIF signature selection
+   - Augment CraneliftBuilder to build signatures based on (arity × type shape)
+   - Start with small shapes: (I64|F64|Bool)* → I64|F64|Bool
+   - Closure trampoline: transmute to matching extern "C" fn type; dispatch by shape id
+
+3) Condition handling
+   - Bool: prefer b1 in IR; allow i64!=0 normalization when comparing integers
+   - Comparisons yield b1; lower branch consumes b1 directly
+
+4) Conversions in lowerer (explicit only)
+   - add_const_f64, add_convert_{i64_to_f64, f64_to_i64}
+   - prohibit implicit int<->float coercion in arithmetic; gate conversions via explicit MIR ops or intrinsics
+
+5) Observability and flags
+   - NYASH_JIT_NATIVE_F64=1 / NYASH_JIT_NATIVE_BOOL=1 to enable paths
+   - Dump: show chosen signature shape and conversions when NYASH_JIT_DUMP=1
+
+6) Rollout
+   - Phase A: const/binop/ret for f64; comparisons yield b1
+   - Phase B: mixed-type ops via explicit converts
+   - Phase C: HostCall bridging for f64/bool PODs (read-only first)
+
+Risks / Mitigation
+- Signature explosion: start with a few common shapes; fallback to i64 path
+- Platform ABI mismatches: rely on Cranelift default call conv; e2e-perf and correctness first
+
+Acceptance
+- Examples with pure f64 pipelines run under JIT with matching results vs VM
+- No silent lossy conversions; conversions visible in MIR/Lower logs
+
--- a/docs/development/roadmap/phases/phase-10/phase_10_9_builtin_box_jit_support.md
+++ b/docs/development/roadmap/phases/phase-10/phase_10_9_builtin_box_jit_support.md
@ -0,0 +1,175 @@
+# Phase 10.9 - ビルトインBox JITサポート
+
+## 🎯 目的
+ビルトインBoxをJITで使えるようにし、Python統合（Phase 10.1）への道を開く。
+
+## 📦 対象Box（優先順位順）
+
+### 第1段階：読み取り専用メソッド
+```nyash
+// StringBox
+str.length()      // → i64
+str.isEmpty()     // → bool
+str.charAt(idx)   // → String（新Box生成）
+
+// ArrayBox  
+arr.length()      // → i64
+arr.isEmpty()     // → bool
+arr.get(idx)      // → Box（既存参照）
+
+// IntegerBox/FloatBox
+int.toFloat()     // → f64
+float.toInt()     // → i64
+```
+
+### 第2段階：Box生成
+```nyash
+// new演算子のJIT化
+new StringBox("hello")     // → Handle
+new IntegerBox(42)         // → Handle（または直接i64）
+new ArrayBox()             // → Handle
+```
+
+### 第3段階：書き込みメソッド
+```nyash
+// 状態変更を伴う操作
+arr.push(item)             // Mutex操作必要
+arr.set(idx, value)        // 境界チェック必要
+map.set(key, value)        // ハッシュ操作
+```
+
+## 🔧 実装戦略
+
+### 1. HandleRegistry活用
+```rust
+// 既存のHandleRegistry（80%実装済み）を拡張
+pub fn jit_get_box_method(handle: u64, method: &str) -> Option<MethodPtr> {
+    // ハンドル → Box → メソッドポインタ
+}
+```
+
+### 2. HostCall拡張
+```rust
+// 現在の限定的なHostCallを段階的に拡張
+enum HostCallKind {
+    // 既存
+    ArrayIsEmpty,
+    StringLength,
+    
+    // Phase 10.9で追加
+    StringIsEmpty,
+    StringCharAt,
+    ArrayGet,
+    IntToFloat,
+    FloatToInt,
+    
+    // new演算子サポート
+    NewStringBox,
+    NewIntegerBox,
+    NewArrayBox,
+}
+```
+
+### 3. 型安全性の確保
+```rust
+// JIT時の型チェック
+match method {
+    "length" => {
+        // StringBox/ArrayBoxのみ許可
+        verify_box_type(handle, &[BoxType::String, BoxType::Array])?
+    }
+    "isEmpty" => {
+        // より多くのBoxで使用可能
+        verify_box_type(handle, &[BoxType::String, BoxType::Array, BoxType::Map])?
+    }
+}
+```
+
+## 📊 成功指標
+
+### 機能面
+- [ ] StringBox.length() がJITで実行可能
+- [ ] ArrayBox.isEmpty() がJITで実行可能
+- [ ] new StringBox() がJITで生成可能
+- [ ] 型チェックが正しく動作
+
+### 性能面
+- [ ] HostCall経由でも10倍以上高速化
+- [ ] Handle解決のオーバーヘッド最小化
+- [ ] Mutex競合の回避（読み取り専用）
+
+### Python統合への貢献
+- [ ] PythonParserBoxの基本メソッドが使用可能
+- [ ] MirBuilderBoxへのデータ受け渡し可能
+- [ ] 最小限のPython→Nyash変換が動作
+
+## 🚧 技術的課題
+
+### 1. Arc<Mutex>パターンとの整合性
+```rust
+// 読み取り専用でもMutexロックが必要？
+// → 読み取り専用APIを別途用意？
+```
+
+### 2. Box生成時のメモリ管理
+```rust
+// JIT内でのArc生成
+// → HandleRegistryで一元管理
+```
+
+### 3. エラーハンドリング
+```rust
+// パニックしない設計
+// → Result型での丁寧なエラー伝播
+```
+
+## 📈 実装ロードマップ
+
+### Week 1：基盤整備
+- HandleRegistry拡張
+- HostCallインターフェース設計
+- 型チェック機構
+
+### Week 2：読み取りメソッド実装
+- StringBox：length, isEmpty, charAt
+- ArrayBox：length, isEmpty, get
+- 数値変換：toInt, toFloat
+
+### Week 3：Box生成サポート
+- new演算子のMIR→JIT変換
+- コンストラクタ呼び出し
+- HandleRegistry登録
+
+### Week 4：テストと最適化
+- E2Eテストスイート
+- パフォーマンス測定
+- Python統合の動作確認
+
+## 🎉 期待される成果
+
+```nyash
+// これが高速に動く！
+static box FastPython {
+    main() {
+        local py = new PythonParserBox()     // JITで生成！
+        local code = "def add(a, b): return a + b"
+        local ast = py.parse(code)           // JITで実行！
+        
+        local builder = new MirBuilderBox()  // JITで生成！
+        local mir = builder.build(ast)       // JITで実行！
+        
+        // Python関数がネイティブ速度で動く！
+        return "Python is now Native!"
+    }
+}
+```
+
+## 🚀 次のステップ
+
+→ Phase 10.10：プラグインBox JITサポート
+→ Phase 10.1：Python統合（いよいよ実現！）
+
+---
+
+作成者：Claude（Nyashくんの要望により）  
+目的：「うるさい、Nyashつかえ」を真に実現するため