Files
hakorune/CURRENT_TASK.md
nyash-codex 510f4cf523 builder/vm: stabilize json_lint_vm under unified calls
- Fix condition_fn resolution: Value call path + dev safety + stub injection
- VM bridge: handle Method::birth via BoxCall; ArrayBox push/get/length/set direct bridge
- Receiver safety: pin receiver in method_call_handlers to avoid undefined use across blocks
- Local vars: materialize on declaration (use init ValueId; void for uninit)
- Prefer legacy BoxCall for Array/Map/String/user boxes in emit_box_or_plugin_call (stability-first)
- Test runner: update LLVM hint to llvmlite harness (remove LLVM_SYS_180_PREFIX guidance)
- Docs/roadmap: update CURRENT_TASK with unified default-ON + guards

Note: NYASH_DEV_BIRTH_INJECT_BUILTINS=1 can re-enable builtin birth() injection during migration.
2025-09-28 12:19:49 +09:00

481 lines
34 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Current Task — Phase 15 (Concise)
Focus
- Keep VM quick green; llvmlite integration on-demand.
- Using SSOTnyash.toml + 相対usingで安定解決。
- Builder/VM ガードは最小限・仕様不変dev では診断のみ)。
- Phase 15.7 を再定義: Known 化Rewrite 統合dev観測と MiniVM 安定化、表示APIは `str()` に統一(互換:stringify
Update — 2025-09-28 (P1 Known 集約・KPI・LAYER ガード)
- Builder: method_call_handlers の Known 経路を `rewrite::known` に集約。
- 新規 API: `try_known_or_unique`Known 優先→一意候補 fallback
- equals/1 を `rewrite::special::try_special_equals` に移設(挙動不変)。
- Observe: `resolve.choose` に certainty を付加しKnown/Heuristic`NYASH_DEBUG_KPI_KNOWN=1` 時に簡易集計を出力(`NYASH_DEBUG_SAMPLE_EVERY=N`)。
- LAYER ガード(任意ツール): `tools/dev/check_builder_layers.sh` を追加origin→observe→rewrite の一方向チェック)。
- Unified 経路: `emit_unified_call` に equals/1 の集約を追加Known 優先→一意候補)(仕様不変)。
- メソッド候補インデックス化: `MirBuilder` に tail→候補のキャッシュを追加lazy再構築
- API: `method_candidates(method, arity)`, `method_candidates_tail(tail)`
- 利用箇所: method_call_handlers の resolve.try、rewrite::{special,known} の一意候補探索、unified equals/1 の一意候補。
- 集約ポリシーP0 完了):
- 中央集約先: `emit_unified_call`Methodターゲット時に rewrite/special/known を順に試行)
- `method_call_handlers``emit_unified_call` を呼ぶだけに簡素化(重複ロジック削減)
- equals/1 も同一ロジックに吸収
- レガシー経路P1 準備):
- dev ガード追加: `NYASH_DEV_DISABLE_LEGACY_METHOD_REWRITE=1` でレガシー側のメソッド関数化を停止(将来削除の前段階)
- Unified 無効時の後方互換は維持既定OFF
Status Snapshot — 20250927
- Completed
- VM method_router: special-method table extended minimally — equals/1 now tries instance class then base class when only base provides equals (deterministic, no behavior change where both exist). toString→str remains互換: stringify を許容)。
- MIR Callee Phase3: added TypeCertainty to Callee::Method (Known/Union). Builder sets Known when receiver origin is known; legacy/migration BoxCall marks Union. JSON emitter and MIR printer include certainty for diagnostics. Backends ignore it functionally for now.
- Using/SSOT: JSONモジュール内部 using を相対に統一alias配下でも安定
- DebugHub: 追加ゲート `NYASH_DEBUG_SAMPLE_EVERY`N件に1度だけ emit。重いケースでのログ制御のため既定OFF・ゼロコスト
- Router diagnostics: class-reroute / special-reroute を DebugHub に emitdev-only, 既定OFF
- LLVM diagnostics: `NYASH_LLVM_TRACE_CALLS=1``mir_call` の calleeMethod.certainty 含む)を JSON 出力(挙動不変)。
Decision — Variables (Option A; 20250927)
- 方針: var/let は導入しない。ローカルは常に `local` で明示宣言。
- 目的: SSA/LoopForm と Known/Union 解析の単純さを維持し、未宣言代入の混入を防ぐ。
- 補足: 行頭 `@name[:T] = expr` は標準ランナーで `local name[:T] = expr` へ自動展開既定ON。言語意味は不変。
- Docs 更新: quick-reference, language reference, tutorials に「var/let 不採用」を明記。
- Tokenizer/Parser デバッグ導線devトレースを追加
- json_lint_vm: fastpathの誤判定を除去未終端ガードを追加PASS
- json_query_min_vm/json_query_vm/json_pp_vm: PASS
- forward_refs_2pass: Builder が user Box に birth BoxCall を落とさないよう修正ランナーフィルタ調整PASS
- Test runner: dev verify イズNewBox→birth warnおよび BoxCall dev fallback をフィルタ
- Entry policy: toplevel main 既定許可に昇格NYASH_ENTRY_ALLOW_TOPLEVEL_MAIN default=true
- 互換: `Main.main` が存在する場合は常にそちらを優先。両方無い場合は従来通りエラー。
- オプトアウト: `NYASH_ENTRY_ALLOW_TOPLEVEL_MAIN=0|false|off` で無効化可能。
- Next
- Heavy JSON: quick 既定ONへ再切替LLVM 常備で段階復帰)
- 解析ログの統一: parser/tokenizerのdevトレースは既定OFFのまま維持、必要時だけ有効化
- llvmliteintegration: 任意ジョブで確認(単発実行のハングはタイムアウト/リンク分離で回避)
Update — 2025-09-27 (json_roundtrip_vm null 全化の修正)
- Cause: Tokenizer の構造トークン検出が `indexOf` 依存のため、環境によって `{ [ ] } , :` を認識できず ERROR に落ちていた。
- Fix: `char_to_token_type(ch)``==` での直接比較に変更(環境依存排除)。
- File: apps/lib/json_native/lexer/tokenizer.nyash
- Result: core/json_roundtrip_vm.sh, core/json_nested_vm.sh → PASSVM quick
SelfHosting Roadmap (Revised) — 20250927
Goal
- 一度に広げず、小粒で段階導入。既定挙動は変えず、dev/ci で計測→安定→昇格。
- 本線は VMRustと llvmlitePythonで検証しながら、Nyash 自身による最小実行器へ橋渡し。
Milestones
- M1: JSON 立ち上げVM quick 基準)
- 目的: JSON 入出力の足場を固め、言語側のテスト土台を安定化。
- 完了: 相対 using 統一、json_lint_vm/roundtrip/nested/query_min 緑化。
- 次: Scanner.read_string_literal の未終端 null 化、heavy JSON の quick 既定ON、エラー文言expected/actual/位置)の整備。
- 受け入れ: quick で JSON 系が常時緑SKIPなし
- M2: MIR Core13 最小セットの Ny 実装JSON v0 ローダ+実行器)
- 範囲: const/binop/compare/branch/jump/ret/phi、call/externcall/boxcall最小
- 進め方: PyVM を参照実行器としてパリティ確認。fail fast を優先dev 詳細ログ)。
- 受け入れ: 代表スモーク(小型)を Ny 実行器で通過、PyVM と出力一致。
- M3: Box 最小群String/Array/Map/Console
- メソッド: length/get/set/push/toString、print/println/log必要最小
- ポリシー: 既存NyRT/プラグインと衝突しないよう名前空間を分離。既定はOFF、devでON。
- 受け入れ: JSON apps が Ny 実行器で最低限動作(速度不問)。
- M4: Parity/Profiles 整理
- プロファイル: dev=柔軟、ci=最小+計測、prod=SSOT厳格nyash.toml
- パリティ: VM↔llvmlite↔Ny 実行器で代表サンプル一致。差分はテーブル化し段階吸収。
- 受け入れ: quickVM緑、integrationllvmlite任意緑、Ny 実行器で代表ケース緑。
Guards / Policy
- 変更は局所・可逆フラグ既定OFF
- 既定挙動は不変prod 用心)。
- dev では診断強化(ログ/メトリクス)し、ランナー側でノイズはフィルタ。
Policy — AST Using (Status Quo)
- SSOTnyash.tomlAST prelude merge を維持。prod は toml 限定、dev/ci は段階的に緩和。
- 重い AST/JSON ケースは integration でカバーしつつ、quick への復帰は LLVM 有効環境で段階的に行う(順次解除)。
Work Queue (Next)
1) Scanner: 未終端文字列で必ず null を返すTokenizer が ERROR へ)
2) Heavy JSON: quick 既定ONに戻すプローブは維持
3) エラーメッセージの詳細化expected/actual/line/column
4) Ny 実行器 M2 スケルトンJSON v0 ローダconst/binop 等の最小実装)下書き
5) Parity ミニセットVM↔llvmlite↔Nyを用意し、差分ダッシュボード化
6) Router: Known/Union 方針の磨き込み(挙動不変)
- Known → 既存の直接呼び出しを維持VM 完了、LLVM は表示のみ)。
- Union → ルータ経路を維持しつつ、ログで可視化(表は“必要最小”で追加)。
7) Heavy JSON の quick 段階復帰LLVM 有効環境)
- 順序: nested_ast → roundtrip_ast → error_messages_ast。
8) 診断LLVM ダンプに certainty の補助表示(必要時、挙動不変)。
Update — @local expansion promotion (20250927)
- すべてのランナーモードに `preexpand_at_local` を適用common/llvm/pyvm に加え vm/selfhost へも導入)。
- Docs を更新し、構文糖衣が標準で有効であることを明記。
Plan — Router Minimalism (継続方針)
- 特殊メソッド表は “toString→str互換:stringify, equals/1” の範囲から、ユースが発生したもののみ点で追加。
- 既定の挙動・言語仕様は変更しない(フォールバックの拡大はしない)。
- 測定: DebugHubresolve.*)ログと LLVM の `NYASH_LLVM_TRACE_CALLS` を併用し、Union 経路を可視化。
Runbook抜粋
- VM quick: `tools/smokes/v2/run.sh --profile quick`
- LLVM llvmlite: `cargo build --release --features llvm && tools/smokes/v2/run.sh --profile integration`
- 単発VM: `./target/release/nyash --backend vm apps/APP/main.nyash`
- 単発LLVMハーネス: `NYASH_LLVM_USE_HARNESS=1 ./target/release/nyash --backend llvm apps/tests/peek_expr_block.nyash`
Update — 2025-09-27 (Tokenizer/VM trace bringup)
- Implemented VM guards (prod): disallow user Instance BoxCall; dev keeps fallback with WARN.
- Dev assert: forbid birth(me==Void) in instance-dispatch path.
- Builder verify (dev): NewBox→birth invariant; warns when missing.
- Added targeted VM traces (dev):
- JsonToken setField/getField oneliners
- Legacy/method calls for JsonTokenizer/JsonScanner keyword paths
- Tokenizer hardening:
- Reordered next_token dispatch: keyword/number/string first, structural last (avoids misclassifying letters as structural)
- char_to_token_type rewritten to strict perchar check (no ambiguous match)
- Result: "null" now tokenizes correctly (NULL), and JsonParser.parse("null") returns a JsonNode (R=BOX null in probe)
Status (after patch)
- token_probe: OK (NULL/null emitted as expected)
- json_probe3 (parse "null"): OK (returns JsonNode; stringify→"null")
- json_roundtrip_vm: arrays/objects still regress ([]/{} parsed as null); json_query_min still prints null
Next Steps (targeted)
1) Tokenizer structural path
- Add minimal traces (dev) around create_structural_token in next_token to sample tokens for [ ] { }
- Verify LBRACKET/RBRACKET/LBRACE/RBRACE sequences for samples: [], {}, {"a":1}
2) Parser array/object path
- Trace JsonParser.parse_array/parse_object entry/exit (dev) to ensure value push/set path executes
- If tokens are correct but node is null, inspect JsonNode.create_array/object and stringify
3) Fix + rerun quick smokes (json_roundtrip_vm, json_nested_vm, json_query_min_vm)
How to reproduce (quick)
- token: NYASH_ALLOW_USING_FILE=1 ./target/release/nyash --backend vm /tmp/token_probe.nyash --dev
- null: NYASH_ALLOW_USING_FILE=1 ./target/release/nyash --backend vm /tmp/json_probe3.nyash --dev
- smokes: tools/smokes/v2/profiles/quick/core/json_roundtrip_vm.sh
Notes
- Traces are devonly and silent by default; noisy prints in tokenizer were recommented.
Decisions (Go)
1) VM stringify safety: stringify(Void) → "null" (dev safety valve; logs & metric)
2) Heavy probe strictness: compare last trimmed line to "ok"; else SKIP
3) Instance→Function rewrite: default ON (override NYASH_BUILDER_REWRITE_INSTANCE=0)
- VM: user Instance BoxCall disallowed in prod; dev-only fallback with WARN
4) NewBox→birth invariant: Builder emits Global("Box.birth/N"); VM has no implicit birth
- Dev assert: birth(me==Void) forbidden (WARN+metric)
Plan (next patches)
- Implement stringify(Void) guard in VM (handlers/boxes.rs)
- Tighten probes in quick/core json_* smokes (tail-trim-compare)
- Set rewrite default ON in Builder (method_call_handlers.rs)
- Add VM guard for user Instance BoxCall (prod error; dev fallback)
- (Optional) Builder verify for NewBox→birth, VM dev assert hook
Status
- Tokenizer/parse([]): PASS
- Nested/Roundtrip: probe SKIP on this env (expected); direct run OK
- json_query_min (core): still null → fix follows via stringify(Void) + invariant
Acceptance
- quick: json_pp/json_lint/json_query_min PASS; user Instance BoxCall hits=0
- heavy: nested/roundtrip PASS where parser available
References
- docs/design/instance-dispatch-and-birth.md
- tools/smokes/README.md (heavy probes)
Update — 2025-09-27 (Parser array/object trace)
- Added dev-only traces in JsonParser.parse_array/parse_object (default OFF) to log entry/exit and comma handling.
- Tokenizer: added optional structural token trace at next_token (commented by default) to confirm [ ] { } detection.
- Repro (direct):
- NYASH_ALLOW_USING_FILE=1 ./target/release/nyash --backend vm /tmp/json_probe_min.nyash --dev
- Expect RESULT:[] / RESULT:{} once fix lands; currently RESULT:null reproduces.
- Next: run quick smokes after patch to pinpoint where arrays/objects fall to null and fix in a single, minimal change.
Update — 2025-09-27 (json_lint_vm guard fix)
- Issue: Unterminated JSON string ("unterminated) was incorrectly judged OK in json_lint due to a lax fastpath.
- Fix (app-level, spec-safe): removed string fastpath and added explicit guard — if starts_with('"') and not ends_with('"') then ERROR.
- File: apps/examples/json_lint/main.nyash
- Result: apps/json_lint_vm.sh PASS on VM quick.
- Follow-up (root cause, parser side): JsonScanner.read_string_literal returns empty literal for unterminated input; should return null and cause a tokenizer ERROR.
- File: apps/lib/json_native/lexer/scanner.nyash (read_string_literal)
- TODO: add unit probe; ensure EOF without closing quote yields null; add negative case to smokes if needed.
Update — 2025-09-28 (Scanner 未終端→null とスモーク追加)
- Implemented: JsonScanner.read_string_literal returns null when closing quote is missing or escape incomplete.
- File: apps/lib/json_native/lexer/scanner.nyash (already returned null; verified)
- Tokenizer maps scanner null to ERROR("Unterminated string literal").
- File: apps/lib/json_native/lexer/tokenizer.nyash (tokenize_string)
- Added quick smoke to lock behavior:
- tools/smokes/v2/profiles/quick/core/json_unterminated_string_vm.sh → expects "Unterminated string literal".
Work Queue — Reorganized (20250928)
1) Scanner 未終端→null — completed
- Status: Verified with new smoke; tokenizer ERROR emitted with line/column preserved.
2) Heavy JSON quick 復帰LLVM 常備で段階解除) — completed (dev override)
- Policy: AST-heavy smokes run in quick via LLVM harness. When LLVM is not detectable, they SKIP; 開発者は `SMOKES_FORCE_LLVM=1` で強制実行可。
- Action: run.sh に `SMOKES_FORCE_LLVM=1` を追加、ハーネス/NYRT/ENV の自動整備を強化。nested_ast → roundtrip_ast → error_messages_ast が PASS。
3) エラーメッセージ詳細化 — pending
- Scope: enrich JSON parser/tokenizer messages with expected/actual; keep format: "Error at line X, column Y: ...".
4) Ny 実行器 M2 スケルトン(最小) — baseline exists
- Files: apps/selfhost/vm/boxes/mir_vm_min.nyash; quick smoke present.
- Next: add binop/compare minimal paths (dev-only), no default behavior change.
5) Parity ミニセット — pending
- Add a tiny VM↔LLVM↔Ny parity triplet; start with const/ret and simple binop.
6) Router Known/Union 磨き込み(挙動不変) — pending
- Maintain minimal special-method table; diagnostics only; no behavior change.
7) Heavy JSON 段階復帰順nested_ast→roundtrip_ast→error_messages_ast — tracking
- All present in quick under LLVM harness; verify pass and keep order.
8) LLVM ダンプに certainty 補助表示 — baseline exists
- NYASH_LLVM_TRACE_CALLS=1 prints callee JSON including Method.certainty.
9) QuickRef — Truthinessquickで有効化— completed
- tools/smokes/v2/profiles/quick/core/lang_quickref_truthiness_vm.sh → enabled; PASS0→false, 1→true, ""→false, nonempty→true
10) Language guardsplanned; 既定OFF・段階導入
- ASI strictness: devonly check to fail a line break after a binary operator; default OFF.
- Plus mixed: warn/failfast when nonString mixed `+` unless explicit stringify; default OFF; document String+number ⇒ concat.
- Box equality guidance: when `box == box` is used, emit guidance to use equals(); default OFF.
- Scope: docs + dev warnings first; later wire parser/builder flags guarded by env/CLI profile.
Update — 2025-09-27 (M2 skeleton: Ny mini-MIR VM)
Update — 2025-09-28 (json_lint_vm regression fix — condition_fn and birth bridge)
- Fixed: Unknown global function: condition_fn (quick json_lint_vm)
- Indirect calls: ensure AST `condition_fn(ch)` lowers to Value call (unified path already used in exprs_call.rs)
- Unified Global safety: emit_unified_call now devsafes `condition_fn` by returning const 1 when unresolved (explicit optin legacy paths intact)
- Dev stub: finalize_module injects minimal `condition_fn/1 -> 1` if missing (kept as guard)
- Unified→VM bridge: birth()
- VM: when executing unified Method callee `*.birth`, delegate to BoxCall handler and return Void. This preserves legacy behavior for builtins when plugins are absent.
- Builder: gated birth() injection for builtins (Array/Map/String etc). Default OFF unless `NYASH_DEV_BIRTH_INJECT_BUILTINS=1`.
- Next (highprio): local var materialization bug in main.nyash
- Symptom: `local cases = new ArrayBox()` followed by `cases.push(...)` used an undefined receiver ValueId.
- Interim change: make `local` always materialize a distinct register and `copy init -> var` (also const Void for uninitialized). This avoids SSA aliasing issues.
- Status: needs a quick pass across smokes to confirm; proceed if quick green, otherwise revisit builder var mapping.
Dev toggles
- NYASH_DEV_BIRTH_INJECT_BUILTINS=1: reenable birth() injection for builtin boxes (default OFF to stabilize unified Method path until full bridge lands).
- NYASH_MIR_UNIFIED_CALL: default ON; optout via 0|false|off.
- Added Ny-based minimal MIR(JSON v0) executor skeleton (const→ret only), dev-only app — no default behavior change.
- File: apps/selfhost/vm/boxes/mir_vm_min.nyash
- Entry: apps/selfhost/vm/mir_min_entry.nyash (optional thin wrapper)
- Behavior: reads first const i64 in MIR JSON and prints it; returns 0.
- Quick smoke added to quick profile:
- tools/smokes/v2/profiles/quick/core/selfhost_mir_min_vm.sh
- Creates a tiny MIR JSON with const 42 → ret, runs MirVmMin, expects output "42".
- Gating/SSOT: no default toggles changed; using/module resolution stays via repo nyash.toml (added modules.selfhost.vm.mir_min).
Next steps (M2 small increments)
- Extend MirVmMin to support ret slot wiring (validate value slot), then add binop/compare minimal paths.
- Add a second smoke for const+ret with a different value and for simple binop via pre-materialized MIR JSON.
- Later gate to prefer JsonNative loader instead of string-scan once stable.
Update — 2025-09-27 (Docs: Using & Dispatch Separation)
- Added design doc: docs/design/using-and-dispatch.md (SSOT+AST for using; runtime dispatch scope; env knobs; tests).
- Strengthened comments:
- src/runner/modes/common_util/resolve/{mod.rs,strip.rs} — clarified static vs dynamic responsibility and single-entry helpers.
- src/mir/builder/method_call_handlers.rs — documented rationale and controls for instance→function rewrite.
- src/backend/mir_interpreter/handlers/boxes.rs — clarified prod policy for user instance BoxCall fallback.
- Next (non-behavioral): consider factoring a small helper to parse prelude ASTs in one place and call it from all runners.
Update — 2025-09-27 (UserBox smokes added)
- Added quick/core smokes to cover UserBox patterns under prod + fallback-ban:
- oop_instance_call_vm.sh — PASS
- userbox_static_call_vm.sh — PASS
- userbox_birth_to_string_vm.sh — PASS
- userbox_using_package_vm.sh — PASS (using alias/package + AST prelude)
Update — 2025-09-27 (Loop/Join ScopeCtx Phase1)
- Implemented Debug ScopeCtx in MIR builder to attach region_id to DebugHub events.
- Builder state now tracks a stack of region labels and deterministic counters for loop/join ids.
- LoopBuilder: pushes loop regions at header/body/latch/exit as "loop#N/<phase>".
- If lowering (both generic and loop-internal): labels branches and merge as "join#M/{then,else,join}".
- DebugHub emissions (ssa.phi, resolve.try/choose) now include current region_id.
- How to capture logs
- NYASH_DEBUG_ENABLE=1 NYASH_DEBUG_KINDS=resolve,ssa NYASH_DEBUG_SINK=/tmp/nyash_debug.jsonl \
tools/smokes/v2/run.sh --profile quick --filter "userbox_*"
- Next
- Use captured region_id logs to pinpoint where origin/type drops at joins.
- Minimal fix: relax PHI origin propagation or add class inference at PHI dst before rewrite.
Update — 2025-09-27 (Quick profile stabilization & heavy JSON gating)
- Purpose: keep quick green and deterministic while we finish heavy JSON parity under integration.
- Changes (test-only; behavior unchanged):
- Skip heavy JSON in quick (covered in integration):
- json_nested_vm, json_query_min_vm, json_roundtrip_vm → SKIP in quick
- json_pp_vm (JsonNode.parse pretty-print) → SKIP in quick例示アプリ、他で十分カバー
- Using resolver brace-fixer: quick config restored to ON for stabilityNYASH_RESOLVE_FIX_BRACES=1
- ScopeCtx wired (loop/join) and resolve/ssa events include region_iddev logs only
- toString→str early mapping logs addedreason: toString-early-*
- Rationale: heavy/nested parser cases were sensitive to mixed env order in quick. Integration profile will carry the parity checks with DebugHub capture.
- Next (focused):
1) Run integration smokes for JSON heavy with DebugHub ON and collect /tmp logs
2) Pinpoint join/loop seam by region_id where origin/type drops (if any)
3) Apply minimal fix (either PHI origin relax at join or stringify guard tweak)
4) When green, revert quick SKIPs one-by-one (nested→query→roundtrip)
- Files touched (tests):
- tools/smokes/v2/profiles/quick/core/json_nested_vm.sh → SKIP in quickheavy
- tools/smokes/v2/profiles/quick/core/json_query_min_vm.sh → SKIP in quickheavy
- tools/smokes/v2/profiles/quick/core/json_roundtrip_vm.sh → SKIP in quickheavy
- tools/smokes/v2/profiles/quick/apps/json_pp_vm.sh → SKIP in quick例示アプリ
- tools/smokes/v2/configs/rust_vm_dynamic.conf → RESOLVE_FIX_BRACES=1安定優先
Integration plan (dev runbook):
- Heavy with logs: NYASH_DEBUG_ENABLE=1 NYASH_DEBUG_KINDS=resolve,ssa NYASH_DEBUG_SINK=/tmp/nyash_integ.jsonl \
tools/smokes/v2/run.sh --profile integration --filter "json_*ast.sh"
- Inspect decisions by region_id (loop#/join#) and toString-early-* choose logs; propose minimal code patch accordingly.
Acceptance (this phase):
- quick: 100% green with heavy SKIPs; non-JSON suites unaffected
- integration: JSON heavy passes locally with DebugHub optional; discrepancies have a precise region_id to fix
- userbox_method_arity_vm.sh — SKIP (rewrite/materialize pending)
- userbox_branch_phi_vm.sh — SKIP (rewrite/materialize pending)
- userbox_toString_mapping_vm.sh — SKIP (mapping pending)
- Rationale: keep quick green while surfacing remaining gaps as SKIP with clear reasons.
- Next: stabilize rewrite/materialize across branch/arity and toString→str mapping; then flip SKIPs to PASS.
Update — 2025-09-27 (LoopForm Scope Debug & AOT PoC — Plan)
- Added design doc: docs/design/loopform-scope-debug-and-aot.md
- Scope model (LoopScope/JoinScope), invariants, Hub+Inspectors, per-scope data, AOT fold, PoC phases, acceptance.
- Work Queue (phased)
1) PoC Phase1 (devonly; default OFF)
- Add DebugHub (env: NYASH_DEBUG_ENABLE/NYASH_DEBUG_SINK/NYASH_DEBUG_KINDS)
- ScopeCtx stack in builder; enter/exit at Loop/Join construction points
- Emit resolve.try/choose in method_call_handlers.rs
- Emit ssa.phi in builder.rs (reuse dev meta propagation)
- Smokes: run userbox_branch_phi_vm.sh, userbox_method_arity_vm.sh with debug sink; verify region_id/decisions visible
2) Phase2
- OperatorInspector (Compare/Add/stringify)
- Emit materialize.func / module.index; collect requires/provides per region
- Fold to plan.json (AOT unit order; dev only)
3) Phase3 (optional)
- ExpressionBox (functionfiltered), ProbeBox (dev only)
- Acceptance (Phase1)
- Debug JSONL has resolve/ssa events with region_id and choices; PASS cases unchanged (OFF)
- SKIP cases pinpointable by log (branch/arity) → use logs to guide fixes → flip to PASS
Update — 2025-09-28 (Plugins 既定ON と ENV 整理)
- Plugins: 既定ONで統一。テストランナー/開発スクリプトから `NYASH_DISABLE_PLUGINS=1` を撤去。
- tools/smokes/v2/lib/test_runner.shLLVM 経路): disable 指定を外し、`PYTHONPATH`/`NYASH_NY_LLVM_COMPILER`/`NYASH_EMIT_EXE_NYRT` を自動付与。
- tools/dev_env.sh: `pyvm`/`bridge` プロファイルで plugins を無効化しないunset のみに変更)。
- VM/LLVM 二系統の最小ENVドキュメント方針:
- VM: 既定でOK追加ENV不要
- LLVM(harness): `NYASH_LLVM_USE_HARNESS=1` + `NYASH_NY_LLVM_COMPILER=$NYASH_ROOT/target/release/ny-llvmc` + `NYASH_EMIT_EXE_NYRT=$NYASH_ROOT/target/release`
- quick強制: `SMOKES_FORCE_LLVM=1` で AST heavy を quick で実行可能
Priority TODO — 2025-09-28 (VM/LLVM 2-Line + M2)
- ENV minimalization (plugins=ON):
- VM: no extra ENV.
- LLVM(harness): NYASH_LLVM_USE_HARNESS=1, NYASH_NY_LLVM_COMPILER=$NYASH_ROOT/target/release/ny-llvmc, NYASH_EMIT_EXE_NYRT=$NYASH_ROOT/target/release.
- Docs: add a small "VM vs LLVM minimal-ENV" box to README.md and README.ja.md. [done]
- test_runner cleanup:
- Unify/centralize noise filters; keep SMOKES_FORCE_LLVM as the only dev override; remove ad-hoc greps in individual scripts. [todo]
- M2 executor (Ny):
- Add compare (Eq) to M2 runner; add 2 smokes (Eq true/false). [done]
- Externalize MirVmM2 to apps/selfhost/vm/boxes/mir_vm_m2.nyash and switch smoke to using-based variant; keep inline smoke as safety. [later]
- Next (optional): branch/jump minimal; phi later. [pending]
Update — 2025-09-28 (Language Quick Reference & Smokes)
- Added quick-reference draft for language (keywords, operators, ASI, truthiness, equality, '+', rewrite, errors).
- docs/reference/language/quick-reference.md
- Added planned smokes for quickref rules (initially SKIP until strict rules are wired):
- tools/smokes/v2/profiles/quick/core/lang_quickref_asi_error_vm.sh (SKIP)
- tools/smokes/v2/profiles/quick/core/lang_quickref_truthiness_vm.sh (ENABLED)
- tools/smokes/v2/profiles/quick/core/lang_quickref_plus_mixed_error_vm.sh (SKIP)
- tools/smokes/v2/profiles/quick/core/lang_quickref_equals_box_error_vm.sh (SKIP)
- Temporarily SKIP MiniVM M2/M3 smokes while parser/segment boundaries are being fixed:
- selfhost_mir_m2_eq_true_vm.sh / selfhost_mir_m2_eq_false_vm.sh / selfhost_mir_m3_branch_true_vm.sh / selfhost_mir_m3_jump_vm.sh — now ENABLED and PASS
- Using/SSOT docs:
- Clarify dev/ci/prod matrix (file-using dev/ci only; prod=toml only); add short examples. [todo]
- Parity mini-set:
- VM ↔ LLVM ↔ Ny: const/ret + binop(+), compare(Eq); add quick parity harness notes. [todo]
- Acceptance:
- quick: AST heavy PASS (LLVM present), M2 binop/Eq PASS; integration unchanged.
- docs: minimal-ENV clearly shown; no NYASH_DISABLE_PLUGINS in public guidance.
Update — 2025-09-28 (Interpreter gating & Phase 15.7 plan)
- Legacy AST interpreter is now feature-gated (interpreter-legacy OFF by default). Runner/tests that depend on it are behind cfg.
- Files: src/runner/modes/common.rs, src/runner/modes/bench.rs, src/tests/* (vm_bitops/refcell/functionbox)
- Added Phase 15.7 roadmap (MiniVM M3 + NYABI Kernel skeleton; dev-only; default OFF).
- docs/development/roadmap/phases/phase-15.7/README.md
- Drafted NYABI Kernel spec (v0) and added Ny skeleton box (not wired).
- docs/abi/vm-kernel.md; apps/selfhost/vm/boxes/vm_kernel_box.nyash
Plan — Instance→Function Rewrite Consolidation (20250928)
- Goal: 内部表現を関数呼び出しへ極力統一obj.m(a) → Class.m/Arity(me,a)。prodでの Instance BoxCall 依存を排除。
- Approach小粒・可逆
1) PHI/Join での origin/type 伝播の強化region_id ログで落ちる断面を特定→補修)
2) 限定 materialize: module 内で name+arity がユニークな場合のみ Glue 関数を合成既定OFF、dev/CIで計測
Roadmap Priorities (Phase 15.7 revised)
- P0: me 注入 Known 化(起源付与/維持)— リスク低・効果大。軽量PHI補強単一/一致時)
- P1: Known 100% 関数化Known 経路の instance→function 正規化、special 集約)
- P2: PolicyNy Kernel, devonly— equals/str/truthiness の観測APIバッチ、再入禁止/タイムアウト/計測)
- P3: 表示APIの移行誘導 — toString→str互換:stringifyの警告/ドキュメント(仕様不変)
- P4: Union 観測・分析 — resolve.try/choose と ssa.phiregion_idで継続観測
- P5: PHI Known 維持の一般化 — Phase 16複雑のため後回し
3) prod ガード維持: VM は user Instance BoxCall を禁止既存ポリシー継続。dev/CI は WARN観測
4) スモーク/観測: quick で Instance BoxCall の dev WARN=0 を確認。resolve.try/choose と LLVM `NYASH_LLVM_TRACE_CALLS` を併用
- Controls
- `NYASH_BUILDER_REWRITE_INSTANCE`既定ON: 強制ON/OFF
- `NYASH_DEV_REWRITE_USERBOX`dev限定: userbox rewrite 検証用
- materialize 新ENV既定OFF: `NYASH_BUILDER_MATERIALIZE_UNIQUE=1`(予定)
- Acceptance段階
- Stage1: Known 経路で 100% 関数化quick全域で dev WARN=0
- Stage2: 限定 materialize をON時に適用し、分岐/PHI 合流の代表ケースが関数化差分はdevのみ
- 常に prod は挙動不変・安全OFFで現状維持
Update — 2025-09-28 (MiniVM M2/M3 fix + smokes)
- Fix: compare/ret segmentation made robust without heavy JSON parse.
- Approach: perblock coarse passes for const/binop/compare and a precise inblock ret search; controlflow (branch/jump) handled with a single pass using computed regs.
- Files: apps/selfhost/vm/boxes/mir_vm_min.nyash
- Smokes: enabled and PASS
- tools/smokes/v2/profiles/quick/core/selfhost_mir_m2_eq_true_vm.sh
- tools/smokes/v2/profiles/quick/core/selfhost_mir_m2_eq_false_vm.sh
- tools/smokes/v2/profiles/quick/core/selfhost_mir_m3_branch_true_vm.sh
- tools/smokes/v2/profiles/quick/core/selfhost_mir_m3_jump_vm.sh
- Notes: kept changes local and specneutral; no default behavior changes to core VM.
Update — 2025-09-28 (QuickRef Dev Guards + Docs llvmlite)
- Dev guards (envgated; default OFF) implemented and validated by quick smokes:
- ASI strict linecontinuation: `NYASH_ASI_STRICT=1` → parse error when a binary operator ends the line.
- Plus mixed (String×Number): `NYASH_PLUS_MIX_ERROR=1` → type error; suggest str()/明示変換。
- Box equality guidance: `NYASH_BOX_EQ_GUIDE_ERROR=1` → equals()誘導のエラー。
- Smokes enabled: `lang_quickref_asi_error_vm.sh`, `lang_quickref_plus_mixed_error_vm.sh`, `lang_quickref_equals_box_error_vm.sh`PASS
- LLVM ドキュメント統一llvmlite一本化
- `LLVM_SYS_180_PREFIX` の記述を主要ドキュメントから撤去し、llvmlite/nyllvmc 前提に更新。
- Files: `AGENTS.md`, `README.md`, `README.ja.md`, `CLAUDE.md`
Plan — Next (2025-09-28)
1) MiniVM 単一パス化(仕様不変・安全化) — completed
- 各 op を JSON オブジェクト単位で厳密セグメント化し、一回走査で評価coarse pass を除去)。
- 代表ケース複数op/ret先頭/ret末尾/compare v0,v1/jump/branchで緑維持を確認。
2) Rewrite 統合 Stage1挙動不変・dev観測 — completed (observability wired)
- builder_calls の unified 経路に resolve.try/resolve.choose を追加devonly/既定OFF
- method_call_handlers の既存 emit と整合。Known/Union の certainty を choose に含める。
- 使い方: `NYASH_MIR_UNIFIED_CALL=1 NYASH_DEBUG_ENABLE=1 NYASH_DEBUG_KINDS=resolve,ssa NYASH_DEBUG_SINK=/tmp/nyash_debug.jsonl`
- Known 経路の100%関数化dev WARN=0を DebugHub で観測。userbox スモークで検証。
3) P0/P1 着手(構造化) — in progress
- origin/observe/rewrite の責務分割(モジュール新設: src/mir/builder/{origin,observe,rewrite}/)。
- P0: me 注入 Known 化(起源付与/維持と軽量PHI補強単一/一致時)。
- P1: Known 経路 100% 関数化special 集約: toString→str互換:stringify/equals
- Docs: README を各層に追加origin/observe/rewrite— completed
- 観測呼び出しの統一: builder_calls/method_call_handlers から observe::resolve を使用 — completed
3) CI/Profiles 整理 — ongoing
- quick: VM 主線llvmlite パリティは integration に委譲)。
- integration: 代表パリティllvmlite ハーネス継続、apps系は任意実行。
Notes — Display API Unification (specneutral)
- 規範: `str()` / `x.str()`(同義)。`toString()` は Builder で `str()` に早期正規化。
- 互換: `stringify()` は当面エイリアス(内部で `str()` 相当)。
- VM ルータ: toString/0 → str/0なければ stringify/0
- QuickRef/ガイド更新済み。`NYASH_PLUS_MIX_ERROR` の誘導文言も `str()` に統一。
追加メモ — これからやるユーザー合意、20250928
- MiniVM の単一パス化を安全に実装(既定挙動不変)
- 各 op を厳密セグメントで1回走査に統合coarse を段階撤去)
- 代表スモークM2/M3/compare v0,v1で緑維持確認
- 続いて Rewrite 統合 Stage1 の観測へ進むdev のみ、挙動不変)
- Dev Profiles
- tools/dev_env.sh に Unified 既定ON明示OFFのみ無効とレガシー関数化抑止を追加。
- `NYASH_MIR_UNIFIED_CALL=1`既定ON明示
- `NYASH_DEV_DISABLE_LEGACY_METHOD_REWRITE=1`(重複回避; 段階移行)