Phase 21.7 normalization: optimization pre-work + bench harness expansion

- Add opt-in optimizations (defaults OFF)
  - Ret purity verifier: NYASH_VERIFY_RET_PURITY=1
  - strlen FAST enhancement for const handles
  - FAST_INT gate for same-BB SSA optimization
  - length cache for string literals in llvmlite
- Expand bench harness (tools/perf/microbench.sh)
  - Add branch/call/stringchain/arraymap/chip8/kilo cases
  - Auto-calculate ratio vs C reference
  - Document in benchmarks/README.md
- Compiler health improvements
  - Unify PHI insertion to insert_phi_at_head()
  - Add NYASH_LLVM_SKIP_BUILD=1 for build reuse
- Runtime & safety enhancements
  - Clarify Rust/Hako ownership boundaries
  - Strengthen receiver localization (LocalSSA/pin/after-PHIs)
  - Stop excessive PluginInvoke→BoxCall rewrites
- Update CURRENT_TASK.md, docs, and canaries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
nyash-codex
2025-11-13 16:40:58 +09:00
parent 9e2fa1e36e
commit dda65b94b7
160 changed files with 6773 additions and 1692 deletions

View File

@ -1,231 +1,126 @@
# Environment Variables — Quick Reference (Phase 22.1)
ENV Variables Index (Core, Builder, Smokes)
This document lists the environment flags introduced or used by the Phase 22.1 work. Defaults are OFF and behavior remains unchanged unless noted.
Purpose: quick reference for toggles used frequently during development and in smokes. Defaults aim to be safe and failfast.
- NYASH_JSON_ONLY=0|1
- Quiet JSON pipelines: suppresses `RC:` and routine logs on stdout (diagnostics still go to stderr).
- Used by StageB → Program(JSON) emit to keep the output clean for downstream processing.
Runtime/VM
- NYASH_FAIL_FAST=0
- Global relaxation switch; when set to 0/false, paired legacy guards may allow otherwise failfast paths.
- Use only for diagnosis; keep OFF in CI.
- HAKO_USING_SSOT=0|1
- Enables the SSOT resolver gate in the runner pipeline.
- When ON, resolution first consults the SSOT bridge (modules-only MVP). If not resolved, it falls back to the existing resolver.
- Trace: set `NYASH_RESOLVE_TRACE=1` to see `[using/ssot]` tags.
- NYASH_LEGACY_FIELDS_ENABLE=1
- Materialize legacy InstanceBox shared fields map for compatibility with older code paths.
- Default: OFF. Dev/compat only; safe to keep disabled in CI and normal runs.
- HAKO_USING_SSOT_HAKO=0|1
- Optional: within the SSOT bridge, call the Hako box `UsingResolveSSOTBox.resolve(name, ctx)` via the VM.
- MVP passes `{ modules, using_paths, cwd }` in `ctx` (modules is consulted). IO is not performed in the box.
- Requires `nyash` binary present; guard remains OFF by default.
- NYASH_JSON_ONLY=1
- When tools/wrappers set this, StageB and related emitters print only JSON to stdout. Auxiliary logs should be routed to stderr.
- Smokes expect singleline JSON on stdout for Program(JSON) and MIR(JSON) producers.
Relative inference (SSOT)
- Default OFF: `HAKO_USING_SSOT_RELATIVE=1` enables a minimal relative candidate synthesis (cwd → using_paths). When multiple candidates exist and `NYASH_USING_STRICT=1`, resolution delegates to legacy resolver (behavior unchanged).
- Ambiguous list size: `HAKO_USING_SSOT_RELATIVE_AMBIG_FIRST_N=<N>` customizes how many candidates are shown in trace (default 3, bounded 110).
- NYASH_ENABLE_USING=1, HAKO_ENABLE_USING=1
- Enable using/alias resolution in dev/VM runs. Smokes set both for symmetry.
Notes on SSOT ctx (expansion plan)
- The bridge constructs a context with:
- `modules` (Map<String,String>) — exact name → path mapping
- `using_paths` (Array<String>) — resolution bases (MVP: hint only)
- `cwd` (String) — callers directory (MVP: hint only)
- Hako box will progressively leverage `using_paths`/`cwd` for relative inference (planned; defaults remain unchanged until enabled).
Extern/Providers
- Arity suffix normalization
- The VM accepts extern names with arity suffixes and normalizes them before dispatch:
- Examples: `env.get/1``env.get`, `hostbridge.extern_invoke/3``hostbridge.extern_invoke`.
- Implemented in `src/backend/mir_interpreter/handlers/calls/externs.rs` and `handlers/externals.rs`.
- HAKO_TLV_SHIM=0|1
- Enables an identity TLV roundtrip at the end of argument encoding for plugin calls.
- Requires building with `--features tlv-shim` to link the optional crate `nyash-tlv`.
- Default OFF; when OFF, the buffer is returned unchanged.
Plugins / Autoload
- NYASH_USING_DYLIB_AUTOLOAD=1
- Enable autoload of `[using.<name>]` entries with `kind="dylib"` from `nyash.toml`.
- The runner loads the specified shared libraries at startup and registers providers for the boxes declared in each plugins `nyash_box.toml`.
- Default: OFF. Guarded to avoid surprises; respects `NYASH_DISABLE_PLUGINS=1`.
- tlv-shim (Cargo feature)
- `cargo build --features tlv-shim` links the optional `nyash-tlv` crate.
- Without this feature, `HAKO_TLV_SHIM=1` has no effect and the original path is used.
Parser/StageB
- HAKO_PARSER_STAGE3=1, NYASH_PARSER_STAGE3=1
- Accept Stage3 syntax (Break/Continue/Try/Catch, etc.). Enabled in smokes for StageB runs.
TLV shim diagnostics
- HAKO_TLV_SHIM_TRACE=0|1
- When 1 (with `tlv-shim` feature), emit a concise trace tag `[tlv/shim:<Box>.<method>]` for shimmed calls.
- Default: minimal noise`MapBox.set` のみ)。詳細な対象はフィルタで拡張。
- HAKO_TLV_SHIM_FILTER=<CommaSeparatedList>
- Filter which calls are traced例: `MapBox.set,ArrayBox.push`)。`<Box>.<method>` または `method` のみで一致。
- 未設定時は最小(`MapBox.set` のみ)。
- HAKO_TLV_SHIM_TRACE_DETAIL=0|1
- When 1, emits `[tlv/shim:detail argc=N]`.
- HAKO_STAGEB_FUNC_SCAN=1
- Devonly: inject a `defs` array into Program(JSON) with scanned method definitions for `box Main`.
Examples (TLV trace)
- `HAKO_TLV_SHIM=1 HAKO_TLV_SHIM_TRACE=1 HAKO_TLV_SHIM_FILTER=ArrayBox.push`
- `HAKO_TLV_SHIM=1 HAKO_TLV_SHIM_TRACE=1 HAKO_TLV_SHIM_FILTER=MapBox.get`
- `HAKO_TLV_SHIM=1 HAKO_TLV_SHIM_TRACE=1 HAKO_TLV_SHIM_FILTER=ArrayBox.push,MapBox.get` (複数指定)
- `HAKO_SHOW_CALL_LOGS=1 HAKO_CALL_TRACE=1 HAKO_CALL_TRACE_FILTER=env.console.log`テストランナーのフィルタ無効extern だけ観測)
Selfhost builders and wrappers
- HAKO_SELFHOST_BUILDER_FIRST=1
- Prefer the Hako MirBuilder path first; wrappers fall back to Rust CLI builder on failure to keep runs green.
Core Thinning I (Phase 22.2) — Plugin C wrapper (design hook)
- HAKO_PLUGIN_LOADER_C_WRAP=0|1
- When 1, emits a design-stage tag `[cwrap:invoke:<Box>.<method>]` at the plugin invocation site and then proceeds with the normal path.
- Default OFF; no behavior change.
- HAKO_PLUGIN_LOADER_C_WRAP_FILTER=<CommaSeparatedList>
- Filter for cwrap tags例: `MapBox.set,ArrayBox.push`)。`<Box>.<method>` または `method` のみで一致。
- NYASH_LLVM_USE_HARNESS=1
- Enable LLVM harness mode (nyllvmc crate backend). Used by builder scripts for EXE/OBJ emission.
Core Thinning I (Phase 22.2) — C-core probe (design hook)
- HAKO_C_CORE_ENABLE=0|1
- When 1, emits a tag `[c-core:invoke:<Box>.<method>]` and (when built with feature `c-core`) calls a thin C probe (`nyash-c-core`) before proceeding with the normal path.
- Default OFF; behavior unchanged.
- HAKO_C_CORE_TARGETS=<CommaSeparatedList>
- Targets to probe例: `MapBox.set,ArrayBox.push`)。未設定時は `MapBox.set` のみ。
- Build note: enable C-core with `cargo build --release -p nyash-rust --features c-core`.
- Examples:
- `HAKO_C_CORE_ENABLE=1 HAKO_C_CORE_TARGETS=ArrayBox.push`
- `HAKO_C_CORE_ENABLE=1 HAKO_C_CORE_TARGETS=ArrayBox.len,ArrayBox.length`
Related toggles used by smokes/tools (for parity with runner/test wrappers):
Call/route unified trace (optional)
- HAKO_CALL_TRACE=0|1
- When ON, emits `[call:<target>.<method>]` for both plugin calls and extern calls.
- Default OFF; logs go to stderr.
- HAKO_CALL_TRACE_FILTER=<CommaSeparatedList>
- Restrict `[call:]` logs to specific targets.
- Matches `<target>.<method>` or bare `method`.
- Examples:
- `HAKO_CALL_TRACE_FILTER=MapBox.set` (method-only)
- `HAKO_CALL_TRACE_FILTER=env.console.log,MapBox.set` (mix target+method)
- HAKO_SHOW_CALL_LOGS=0|1
- When 1, test runner disables its default log filter so `[call:]` traces appear in output.
- NYASH_PARSER_STAGE3=1, HAKO_PARSER_STAGE3=1, NYASH_PARSER_ALLOW_SEMICOLON=1
- NYASH_ENTRY_ALLOW_TOPLEVEL_MAIN=1
- NYASH_DISABLE_NY_COMPILER=1, HAKO_DISABLE_NY_COMPILER=1
LLVM backend selector (builder wrapper)
- NYASH_LLVM_BACKEND=auto|crate|llvmlite|native
- Selects the backend used by `tools/ny_mir_builder.sh` for `--emit obj|exe`.
- Default: `auto`(優先順位: `crate`ny-llvmc が存在すれば既定)→ `native`llc がある場合)/`llvmlite` は明示指定時のみ)。
- `crate`: `./target/release/ny-llvmc` を使用(`cargo build -p nyash-llvm-compiler --release`。推奨の本線EXE/AOT
- `llvmlite`: Python ハーネス `tools/llvmlite_harness.py`(内部用途。外部から使う場合は明示指定)
- `native`: 将来の Hakonative builder 用に予約。
- `--emit exe` のリンク追加ライブラリは `HAKO_AOT_LDFLAGS` で渡す(例: `-static`。nyllvmc は `--libs` 経由で受理。
- 備考: crate 経路では `ny_main` の戻り値i64がプロセスの終了コードに反映rc mapping
- NYASH_LLVM_NATIVE_TRACE=0|1
- When 1, dumps the native IR to stderr for debugging.
- Used with `native` backend to inspect generated LLVM IR before optimization.
- Default OFF; only active when NYASH_LLVM_BACKEND=native is set.
- HAKO_LLVM_CANARY_NORMALIZE=0|1
- 開発/カナリア専用の正規化スイッチ。`1` のとき、最小の JSON 形状差(`schema_version=1``"1.0"``blocks.inst``instructions``const``ty/value` 包装)を自動補正してからビルドします。
- 既定は `0`(無効)。既存ツールの挙動は変わりません。`NYASH_CLI_VERBOSE=1` のとき形状ヒントを `[ny-llvmc/hint]` で出力します。
Name mapping note (EXE link convenience)
- nyash.console.* は C リンク時にシンボル名 `nyash_console_*` に正規化される(ドット→アンダースコア)。
- 例: `externcall nyash.console.log(i8*)` → C シンボル `nyash_console_log`
- 最小 C ランタイムPhase 22.3)の `nyash-kernel-min-c``nyash_console_log(char*)` を提供(設計段階)。
Kernel Minimal C Runtime (Phase 22.3 — design)
- NYASH_KERNEL_C_MIN=0|1
- Reserved toggle for enabling the minimal C runtime shimsdesignstage; defaults OFF
- Build: `cargo build --release -p nyash-kernel-min-c`not linked by default
ENV consolidation (aliases)
- NY compiler path
- Primary: `NYASH_USE_NY_COMPILER=0|1`
- Accepted aliases (deprecated; prints a onetime warning):
- `NYASH_DISABLE_NY_COMPILER=1` → equivalent to `NYASH_USE_NY_COMPILER=0`
- `HAKO_DISABLE_NY_COMPILER=1` → equivalent to `NYASH_USE_NY_COMPILER=0`
- LLVM opt level
- Primary: `NYASH_LLVM_OPT_LEVEL`
- Accepted alias (deprecated; onetime warning): `HAKO_LLVM_OPT_LEVEL`
- GateC (Core direct route)
- Primary: `NYASH_GATE_C_CORE`
- Accepted alias (deprecated; onetime warning): `HAKO_GATE_C_CORE`
Smokes
- SMOKES_DEFAULT_TIMEOUT
- Pertest timeout (seconds) used by `tools/smokes/v2/run.sh --timeout` or auto profile defaults. Quick profile defaults to ~15s.
- Some tests wrap heavy steps (e.g., running a built EXE) with a shorter internal `timeout` to convert hangs into SKIP.
- HAKO_BUILD_TIMEOUT, HAKO_EXE_TIMEOUT
- Internal timeouts (seconds) used by several phase2100 cratebackend tests to bound nyllvmc build/link and EXE execution steps under quick.
- Defaults: `HAKO_BUILD_TIMEOUT=10`, `HAKO_EXE_TIMEOUT=5`.
Notes
- Primary keys are preferred and will be kept. Aliases remain accepted for a grace period and emit a concise deprecation line once per process.
NYASH_FAIL_FAST
- Type: 0|1 (default: 1)
- Purpose: Global FailFast policy for runtime fallbacks in Rust layer. When 1, prohibits silent or alternateroute fallbacks and panics with a stable tag (e.g., [failfast/provider/filebox:*], [failfast/ssot/*]). Set to 0 temporarily during bringup or canaries that rely on legacy routes.
- Keep default behavior unchanged for users. Use these toggles in development and CI wrappers only.
- Avoid enabling legacy paths except for targeted diagnosis. The unified call system is the default in both builder and VM.
Hakorune StageB (include policy)
- VM backend currently does not support `include` statements in Hako execution. StageB focuses on producing Program(JSON v0) (oneline) and avoids includes.
- Program(JSON) → MIR(JSON) uses the Rust GateC path by default. Hako MirBuilder is optin and introduced gradually (registry/internal toggles).
Using/Resolver
- HAKO_USING_RESOLVER_FIRST=1
- Try the SSOT using resolver (`using::resolver::resolve_using_target_common`) first in the runner pipeline.
- On failure, the pipeline falls back to the existing runner logic (aliases/builtins/plugins/relative search).
- Default: OFF. Use to validate resolverfirst migration without changing defaults.
MirBuilder toggles (optin)
- `HAKO_MIR_BUILDER_INTERNAL=0|1` (default: 1)
- Enable internal lowers (Return/If/Compare/BinOp/Method minimal). When 0, only delegate path is used (if enabled).
- `HAKO_MIR_BUILDER_REGISTRY=0|1` (default: 1)
- Enable registrydriven lowering (pattern names like `if.compare.intint`, `return.binop.intint`, `return.method.arraymap`).
- `HAKO_MIR_BUILDER_DELEGATE=0|1` (default: 0)
- Delegate to Runner (`--program-json-to-mir`) instead of internal lowers. Useful for bringup; keep OFF for selfhost canaries.
- `HAKO_MIR_BUILDER_SKIP_LOOPS=0|1` (default: 0)
- Skip heavy loop lowers (`lower_loop_sum_bc`, `lower_loop_count_param`, `lower_loop_simple`) when 1. Diagnostic/bringup aid; no behavior change when unset.
- `HAKO_MIR_BUILDER_REGISTRY_ONLY=<name>` (optional)
- Restrict registry dispatch to a single pattern name (e.g., `return.method.arraymap`) for diagnostics.
Builder/Emit (Selfhost)
- HAKO_SELFHOST_BUILDER_FIRST=1
- Prefer Hako MirBuilder path first; delegates to provider/legacy on failure. Used by `tools/hakorune_emit_mir.sh` and bench scripts.
- HAKO_MIR_BUILDER_BOX=hako.mir.builder|min
- Choose selfhost builder box (full or minimal runner).
- HAKO_SELFHOST_TRACE=1
- Print additional traces during MIR emit bench/wrappers.
MirBuilder (min) alias — bringup helper
- Module alias: `hako.mir.builder.min``lang/src/mir/builder/MirBuilderMinBox.hako`
- Minimal entry that only loads lightweight lowers (e.g., `return.method.arraymap`, `return.int`).
- Tags: emits `[mirbuilder/min:<pattern>]` on success. Default behavior unchanged; use explicitly in tests/tools.
- HAKO_MIR_BUILDER_LOOP_FORCE_JSONFRAG=1
- Force the selfhost builder (and wrappers) to emit a minimal, pure controlflow MIR(JSON) for loop cases (const/phi/compare/branch/binop/jump/ret)。
- Dev専用。purify/normalize と併用すると ret ブロックに副作用命令を混入させない形で AOT/EXE 検証がしやすくなる。
FileBox provider policydev overrides
- Baseline: FileBox は ring1 常備coreroとして登録されます。プラグイン未配置でも panic 経路にはならず、FailFast が ON の場合は明示タグで失敗します(例: `[failfast/provider/filebox:*]`)。
- `NYASH_FILEBOX_MODE=auto|core-ro|plugin-only` — provider selection modedefault auto; with plugins disabled, resolves to corero
- `NYASH_FILEBOX_ALLOW_FALLBACK=0|1` — When 1, allows corero fallback even if FailFast is ON (use sparingly; defaults OFF).
- JSON pipelines: `NYASH_JSON_ONLY=1` also permits corero fallback under FailFast (quiet structured I/O).
- HAKO_MIR_BUILDER_JSONFRAG_NORMALIZE=1, HAKO_MIR_BUILDER_JSONFRAG_PURIFY=1
- JsonFrag の正規化と純化を有効化する。purify=1 のとき newbox/boxcall/externcall/mir_call を除去し、ret 以降の命令を打ち切る(構造純化)。
Examples (FailFast tags and safe overrides)
- Default (FailFast=1): corero フォールバックを禁止(プロバイダ未設定時に失敗)
- 例外ログ: `[failfast/provider/filebox:auto-fallback-blocked]`
- 実行例:
```sh
# will fail fast without plugins
./target/release/hakorune --backend vm apps/tests/filebox_sample.hako
```
- Bringup局所的に許可: `NYASH_FILEBOX_ALLOW_FALLBACK=1` で corero を許可
- 実行例:
```sh
NYASH_FILEBOX_ALLOW_FALLBACK=1 ./target/release/hakorune --backend vm apps/tests/filebox_sample.hako
```
- JSON パイプ(静音): `NYASH_JSON_ONLY=1` を併用して整形済み JSON のみを stdout に出す
- 実行例:
```sh
NYASH_JSON_ONLY=1 NYASH_FILEBOX_ALLOW_FALLBACK=1 ./target/release/hakorune --backend vm apps/tests/emit_program_json.hako
```
Provider path (delegate)
- HAKO_MIR_NORMALIZE_PROVIDER=1
- ProviderRust出力の MIR(JSON) に対して、Hako の JsonFrag 正規化パスを適用するtools/hakorune_emit_mir.sh 内部)。
- 互換維持のため既定はOFF。Box 系で ret ブロックに副作用命令が残るようなケースの暫定純化に利用できる。
Provider policy共通
- `HAKO_PROVIDER_POLICY=strict-plugin-first|safe-core-first|static-preferred`
- Auto モード時の選択ポリシーを制御(既定: `strict-plugin-first`)。
- `safe-core-first`/`static-preferred` は ring1静的/coreroを優先、利用不可時のみプラグインにフォールバック。
- Box個別のモード指定例: FileBox
- `<BOX>_MODE=auto|ring1|plugin-only`(例: `NYASH_FILEBOX_MODE`
- `<BOX>_ALLOW_FALLBACK=0|1`FailFast 例外。局所許可)
- 既存の FileBox 変数はそのまま有効。共通パターンとして今後は他Boxにも拡張予定。
- NYASH_LLVM_FAST_INT=1
- Opt-in toggle for integer hot paths in LLVM MIR/IR. When `1`, `binop``compare` は同一ブロックの `vmap` 定義を最優先し、resolver/PHI を経由しない i64 経路を選びます。ループや分岐で i64 が綺麗に残るときだけ ON にして、CI 等では unset のまま。
- NYASH_MIR_DEV_IDEMP=1
- Dev-only: Enable idempotence markers inside MIR normalize passes. Each pass stamps `pass:function` in the module metadata after rewriting a function once, and subsequent optimizer runs skip reprocessing that function for the same pass. 仕様不変既定OFF。診断時に複数回最適化を呼んでも差分が出ないことを保証するための保険だよ。
- NYASH_LLVM_FAST=1
- Enables NyRT-based FAST helpers (string length via `nyrt_string_length`, pointer re-use, etc.) and now caches literal-backed `length/len` results so loops reuse the same constant value instead of re-calling the helper. Default OFF.
Selfhostfirst wrapper toggles (StageB → MirBuilder)
- `HAKO_SELFHOST_BUILDER_FIRST=0|1` (default: 0)
- When 1, tools like `tools/hakorune_emit_mir.sh` try StageB → MirBuilder(Hako) first, and only fall back to the Rust delegate when necessary.
- `HAKO_SELFHOST_NO_DELEGATE=0|1` (default: 0)
- When 1, forbids delegate fallback in the wrapper. If the Hako MirBuilder fails, the wrapper fails fast (useful to validate the selfhost path).
VM/AOT fast-path toggles (bench/dev only)
- NYASH_VM_FAST=0|1
- Enables small VM micro-optimizations used in micro-benchmarks (no behavior changes in normal runs):
- new StringBox("const") fast path (registry bypass when safe)
- StringBox.length/size returning i64 directly (avoid boxing)
- Default OFF.
- NYASH_LLVM_FAST=0|1
- Enables AOT lowering tweaks in ny-llvmc (opt-in, benches only):
- StringBox.length/size lowered to extern helper returning i64 (no boxing)
- Default OFF. Guarded in ny-llvmc; legacy lowering remains default.
- NYASH_MIR_LOOP_HOIST=1
- AOT 前準備AotPrepBoxでの軽ホイスティングを有効化。固定文字列の `length/len` を即値に置換JSON 書換え)する。制御フローは変更しない。既定 OFF。
MirBuilder toggles (trace and JsonFrag)
- `HAKO_MIR_BUILDER_TRACE=0|1` (default: 0)
- When 1, emits concise builder tags like `[mirbuilder/internal/*]` from Hako lowers. Errors remain visible regardless; this flag controls informational tags.
- `HAKO_MIR_BUILDER_LOOP_JSONFRAG=0|1` (default: 0)
- Enable JsonFrag minimal MIR assembly in loop adapters/lowers (structure observation only).
- `HAKO_MIR_BUILDER_JSONFRAG_NORMALIZE=0|1` (default: 0)
- Normalize JsonFrag output for stable canaries (const grouping, phi placement, numeric canonicalization).
- `HAKO_MIR_BUILDER_LOOP_RETURN=string|map` (default: string)
- Controls the return type of the loop adapter JsonFrag path. `string` returns MIR(JSON) text; `map` returns a `MapBox` like `{ mir: <string> }`.
- NYASH_AOT_COLLECTIONS_HOT=1
- AOT 前準備AotPrepBoxで Array/Map の `boxcall``externcall``nyash.array.*` / `nyash.map.*`のホットパスに張り替える。AOT 専用の最短経路で、診断を省いてオーバーヘッドを抑える。既定 OFF。
Provider diagnostics
- `HAKO_PROVIDER_TRACE=0|1` (default: 0)
- When 1, forces provider selection tags like `[provider/select:FileBox ring=1 src=static]` to appear even under `NYASH_JSON_ONLY=1`.
EXE argv bridge (crate backend)
- NYASH_EXE_ARGV=0|1
- When 1, the ny-llvmc entry wrapper calls `nyash.env.argv_get()` to obtain an `ArrayBox` of process arguments and passes it to `Main.main(args)`.
- Default OFF互換維持。OFF の場合は空の `ArrayBox` が渡されます。
- VM 実行は従来どおり環境変数経由(`NYASH_SCRIPT_ARGS_HEX_JSON` / `NYASH_SCRIPT_ARGS_JSON` / `NYASH_ARGV`)で受け取り、`Main.main(args)` に配列を渡します。
- MIR optimizer dev gate
- NYASH_MIR_DISABLE_OPT=0|1 (alias: HAKO_MIR_DISABLE_OPT)
- When 1, disables all MIR optimizer passes (diagnostics only). Default OFF.
- 用途: 最適化の誤変換切り分けや、退行時の一時回避(既定不変)
- HAKO_MIR_NORMALIZE_PRINT=1
- AotPrep の正規化パス(.hakoを有効化して、`print` 命令を `externcall env.console.log(value)` に書き換えるCFG 不変)。既定 OFF。
- HAKO_MIR_NORMALIZE_REF=1
- AotPrep の正規化パス(.hakoを有効化して、`ref_get/ref_set``boxcall getField/setField` に書き換えるCFG 不変、best-effort。既定 OFF。
- HAKO_MIR_NORMALIZE_ARRAY=1
- AotPrep の正規化パス(.hakoを有効化して、`array_get/array_set`(および一部の `map_get/map_set`)を `boxcall get/set` に書き換えるCFG 不変、best-effort。CollectionsHot の前処理として有効。既定 OFF
- NYASH_AOT_INDEX_WINDOW=1
- AotPrep(CollectionsHot 内部) の index 共有をブロック境界をまたいだ短窓で行う実験的トグル(デフォルト OFF)。
- 窓はバイト数で固定2048。誤共有のリスクがあるため初期はベンチ限定で使用
- NYASH_VERIFY_RET_PURITY=1
- Ret ブロック純化の Fail-Fast トグル。Return の直前に `Const``Copy``Phi``Nop` 以外があると MIR/VM 実行が失敗するようにする開発ガード。既定 OFF。
- tools/perf/dump_mir.sh
- MIR(JSON) を provider-first → jsonfrag フォールバックで吐く小さな wrapper。`--mode provider` で通常 builder 主導、`--mode jsonfrag` で最小化ループを強制。内部でブロックごとの op カウンターを出力するので、branch/map/kilo などのホットパスをすばやく可視化できるよ。
AOT/LLVM (ny-llvmc)
- HAKO_LLVM_OPT_LEVEL=0|1
- ny-llvmc optimization level (default 0/O0). Bench scripts keep O0 unless overridden.
- NYASH_LLVM_DUMP_MIR_IN=/path/to/out.json
- ny-llvmc が受け取る MIR(JSON) をそのまま保存する開発用トグル。AotPrep 適用後の実入力を観測するのに使う。既定 OFF。
Bench helpers
- PERF_USE_PROVIDER=1
- `tools/perf/microbench.sh` で provider/selfhost-first の MIR emit を優先jsonfrag 強制を解除)。環境により provider 失敗時は自動で最小ループにフォールバック。

View File

@ -0,0 +1,45 @@
Legacy ByName Call Removal Plan
Context
- Unified call system is default. VM routes by `Callee` kind (Global/Method/Extern/…)
- Legacy byname resolution (callee=None → string name lookup) is gated by env and OFF by default.
- Goal: remove legacy code path once parity is proven at integration/full levels.
Guards / Current Behavior
- 旧 byname 経路は削除済み。`NYASH_VM_ALLOW_LEGACY` は廃止。
- 既定: `callee=None` 生成時は FailFastBuilder で Callee を必ず付与)。
Scope of Work
1) Identify callers that could still rely on legacy resolution
- Builder emissions that may omit `callee` (should be none after Phase 1 refactor)
- Any VM entry points or helpers constructing byname calls dynamically
- Older tests or tools expecting byname lookups
2) Ensure unified path covers them
- For Global user functions: attach `Callee::Global("Module.func")`
- For dynamic/indirect calls: mark `Callee::Value` and keep VM restrictions explicit
- For externlike names: delegate via handlers/calls/externs.rs (SSOT)
- Normalize arity (`foo/1``foo`) via `utils::normalize_arity_suffix`
3) Strengthen tests
- Smokes/quick: unified pathbyname 不可)を既に強制
- Integration/full: ユーザー関数呼び出し/extern 名正規化のカナリアを追加
- 旧ENV依存がないことを確認`NYASH_VM_ALLOW_LEGACY` は削除済)
4) Remove legacy code path
- Delete `src/backend/mir_interpreter/handlers/calls/legacy.rs` and its resolver module
- Drop env guard checks referring to legacy path (keep error message clarity)
- Update READMEs to remove temporary notes
Readiness Criteria (before removal)
- quick/integration/full all green with legacy OFF
- No `callee=None` emission in builder (verified by grepping MIR JSON / code review)
- Extern SSOT accepts aritysuffixed names; Global externlike names delegate properly
Rollback Plan
- Keep a small revertable commit boundary for legacy deletion
- If issues appear, reintroduce legacy.rs with the same path under a devguard until fixed
CrossReferences
- handlers/calls/README.md (SSOT and boundaries)
- docs/ENV_VARS.md (env guards and policies)

View File

@ -0,0 +1,43 @@
Normalization Ownership — Rust vs Hakorune
Goal
- Prevent double-normalization and keep a clear Single Source of Truth (SSOT) for where each rewrite lives.
Ownership
- Hakorune layer (Hako scripts)
- Methodization: Global("Box.m/N") → mir_call(Method) への変換。
- Name/arity canonicalizationBox.method/N
- Function defs scan/injectHAKO_STAGEB_FUNC_SCAN, HAKO_MIR_BUILDER_FUNCS
- Emit JSON v1 + unified mir_callNYASH_JSON_SCHEMA_V1=1, NYASH_MIR_UNIFIED_CALL=1
- 可視化タグ/診断の出力dev のみ)
- Rust layer
- Structural/correctness: SSA/PHI、受信側ローカライズLocalSSA/Copy/pin
- Legacy JSON v0 → minimal bridgingjson_v0_bridge 内での Callee 補完など)。
- 互換/安全弁: 未定義受信の構造的回復同一BB直近 NewBoxなど、dev ガード付きの最小範囲。
- Optimizer は構造・副作用ベースの最適化に限定(意味論の再書換えはしない)。
Guards and Toggles
- Hakodev 推奨セット)
- HAKO_STAGEB_FUNC_SCAN=1
- HAKO_MIR_BUILDER_FUNCS=1
- HAKO_MIR_BUILDER_CALL_RESOLVE=1
- HAKO_MIR_BUILDER_METHODIZE=1methodize が v1+unified 出力へ寄与)
- NYASH_JSON_SCHEMA_V1=1, NYASH_MIR_UNIFIED_CALL=1
- Rustbridge/診断)
- HAKO_BRIDGE_METHODIZE=1 は bring-up 用の補助。Hako 既定化後は OFF撤去予定
- mir_plugin_invoke/plugin_only は A/B 評価・診断用途。既定 OFF。
Rules of Engagement
- v1 + unified を Hako で生成した場合、Rust 側での methodize/再書換えは行わない(構造のみ)。
- json_v0_bridge は v0 入力に対する互換のために限定運用。v1 既定化が進めば縮退する。
- dev の安全弁(未定義受信の構造回復など)は、テストが十分になり次第 OFF/撤去する。
Testing
- Canary
- tools/dev/phase217_methodize_canary.shrc=5
- tools/dev/phase217_methodize_json_canary.shschema_version + mir_call present、Method優先
- tools/dev/phase216_chain_canary_call.shrc=5
- 失敗時は Hako 側methodize→ Rust 側(構造) の順で原因を特定する。

View File

@ -1,31 +1,33 @@
# Phase 21.5 — Optimization (ny-llvm crate line)
# Phase 21.5 — Optimization (AotPrepFirst)
Scope
- Optimize hot paths for the crate (ny-llvmc) line using Hakorune scripts only.
- Preserve default behavior; all risky changes behind dev toggles.
- Measure EXE runtime (build once, run many) to avoid toolchain overhead noise.
目的
- .hako 側AotPrepで前処理最適化構造のみを行い、LLVM/AOT に渡すIRを軽量にする。
- 既定は挙動不変optin。Return 純化ガードで安全性を担保。
Targets (initial)
- loop: integer accumulations (no I/O)
- strlen: FAST=1 path (pointer → nyrt_string_length)
- box: construct/destroy minimal boxes (String/Integer)
チェックリスト
- [x] パス分割StrlenFold / LoopHoist / ConstDedup / CollectionsHot / BinopCSE
- [x] CollectionsHotArray/Map導入既定OFF
- [x] Map key モード `NYASH_AOT_MAP_KEY_MODE={h|i64|hh|auto}`
- [x] LoopHoist v1 / BinopCSE v1最小
- [x] ベンチ `linidx`/`maplin` 追加
- [ ] LoopHoist v2+/* 右項 const の連鎖前出し/fixpoint
- [ ] BinopCSE v2線形 `i*n` 共通化の強化)
- [ ] CollectionsHot v2array index の共通SSA利用
- [ ] Map auto 精緻化_is_const_or_linear の再帰判定)
- [ ] Idempotence置換済みタグで再実行時も不変
- [ ] `arraymap`/`matmul` ≤ 125%C基準
Methodology
- Build once via ny-llvmc; time execution only (`--exe` mode).
- Runs: 35; report median and average (target ≥ 100ms per run).
- Observe NYASH_VM_STATS=1 (inst/compare/branch) where relevant to correlate structure and runtime.
Commands (examples)
- tools/perf/phase215/bench_loop.sh --runs 5
- tools/perf/phase215/bench_strlen.sh --runs 5 --fast 1
- tools/perf/phase215/run_all.sh --runs 5 --timeout 120
Dev Toggles (keep OFF by default)
- NYASH_LLVM_FAST=1 (strlen FAST)
- HAKO_MIR_BUILDER_JSONFRAG_NORMALIZE=1 (normalize)
- HAKO_MIR_BUILDER_NORMALIZE_TAG=1 (tag, test-only)
Exit Criteria
- Representative microbenches stable (≤ 5% variance) and ≥ 80% of C baselines.
- No regression in EXE canaries (loop/print/strlen FAST) and VM parity canaries.
トグル
- `NYASH_MIR_LOOP_HOIST=1` … StrlenFold/LoopHoist/ConstDedup/BinopCSE を有効化
- `NYASH_AOT_COLLECTIONS_HOT=1` … CollectionsHotArray/Map
- `NYASH_AOT_MAP_KEY_MODE``h|i64|hh|auto`(推奨: `auto`
- `NYASH_VERIFY_RET_PURITY=1` … Return 純化ガード開発時ON
ベンチ(例)
```bash
export NYASH_SKIP_TOML_ENV=1 NYASH_DISABLE_PLUGINS=1 \
NYASH_LLVM_SKIP_BUILD=1 NYASH_LLVM_FAST=1 NYASH_LLVM_FAST_INT=1 \
NYASH_MIR_LOOP_HOIST=1 NYASH_AOT_COLLECTIONS_HOT=1 NYASH_VERIFY_RET_PURITY=1
for c in arraymap matmul sieve linidx maplin; do \
tools/perf/microbench.sh --case $c --exe --runs 3; echo; done
```

View File

@ -36,7 +36,13 @@ Canaries
- tools/dev/stageb_loop_json_canary.sh — Program(JSON) shape for loop(i<n){i=i+1}
- tools/dev/phase216_chain_canary.sh endtoend EXE rc=10 for minimal loop
Provider Path Notes (Dev)
- Optional normalization for provider output is available via `HAKO_MIR_NORMALIZE_PROVIDER=1`.
- This applies the same JsonFrag normalizer/purifier to MIR(JSON) emitted by the Rust Provider path.
- Keep defaults unchanged; use only during bringup to eliminate retafter effects.
- `HAKO_MIR_BUILDER_LOOP_FORCE_JSONFRAG=1` now shortcircuits both selfhostfirst and providerfirst wrappers to emit a minimal, pure controlflow MIR suitable for EXE build sanity.
- Default OFF; intended for small canaries and performance harness bringup.
Removal Plan for temporary parser fallback
- Once VM/gpos interaction is fixed and parser emits correct loop JSON without guards,
remove the conservative fallback in ParserControlBox.parse_loop.

View File

@ -0,0 +1,51 @@
# Phase 21.6 — DualEmit Parity & Cline Readiness
Goal: Produce identical MIR(JSON) from both provider (Rust) and selfhost (Hako) builder paths, measure generation cost, and keep AOT (nyllvmc) fast/green. All work is semanticspreserving; defaults remain unchanged.
## Checklists
- [ ] Dualemit parity on representative apps (MIR(JSON) normalized SHA1 equal)
- [ ] Resolverfirst ON passes quick/integration
- [ ] Selfhostfirst fallback ok (provider/legacy on failure)
- [ ] AOT obj/exe via nyllvmc (crate backend) green
- [ ] Docs updated (bench guides, env vars, quick recipes)
## Scripts
- Dual emit + compare + bench: `tools/perf/dual_emit_compare.sh <input.hako> [rounds]`
- MIR emit bench: `tools/perf/bench_hakorune_emit_mir.sh <input.hako> [rounds]`
- AOT bench: `tools/perf/bench_ny_mir_builder.sh <mir.json> [rounds]`
- MIR diff: `tools/perf/compare_mir_json.sh <a.json> <b.json>`
## Env Knobs
- `HAKO_USING_RESOLVER_FIRST=1` (resolverfirst)
- `HAKO_SELFHOST_BUILDER_FIRST=1` (selfhost→provider→legacy)
- `HAKO_MIR_BUILDER_BOX=hako.mir.builder|min`
- `NYASH_LLVM_BACKEND=crate`nyllvmc
- `HAKO_LLVM_OPT_LEVEL=0|1`AOT O0 既定)
## Benchmarks — Tracking
Record normalized parity and generation times here (edit in place).
Legend: SHA1 = normalized JSON digest; Parity=Yes when SHA1 equal; Times are medians unless noted.
| Benchmark (Hako) | Resolver | Parity | Provider p50 (ms) | Selfhost p50 (ms) | Notes |
|------------------------------------------|----------|--------|-------------------|-------------------|-------|
| apps/examples/json_query/main.hako | off/on | | | | |
| apps/examples/json_pp/main.hako | off/on | | | | |
| apps/examples/json_lint/main.hako | off/on | | | | |
| apps/examples/json_query_min/main.hako | off/on | | | | |
How to fill:
1) Run `tools/perf/dual_emit_compare.sh <file> 5`
2) Copy p50s from the summary lines and mark Parity based on `compare_mir_json.sh` output.
3) Note any diffs (callee kinds/order/phi/meta) in Notes.
## Next Steps
- [ ] If parity holds on the above set, extend to apps/tests subset
- [ ] If diffs remain, categorize and align either provider or selfhost output
- [ ] Keep AOT line green under `HAKO_LLVM_OPT_LEVEL=0` and optional `=1` spot checks

View File

@ -9,12 +9,15 @@ Targets (must be green)
Canaries
- tools/dev/phase216_chain_canary_call.sh — remains PASS when OFF, PASS when ON
- Add: tools/dev/phase217_methodize_canary.sh (dev) — asserts method callee usage in MIR or IR tags
- tools/dev/phase217_methodize_canary.sh (dev) — compile-run rc=5セマンティクス
- tools/dev/phase217_methodize_json_canary.sh (dev) — v1 rootschema_version+ mir_call presentMethodが望ましい、Globalは経過容認
Toggles
- HAKO_MIR_BUILDER_METHODIZE=1 (new)
- HAKO_STAGEB_FUNC_SCAN=1 / HAKO_MIR_BUILDER_FUNCS=1 / HAKO_MIR_BUILDER_CALL_RESOLVE=1 (existing)
- 一軍devプロファイルで既定ON: 上記3つ + NYASH_JSON_SCHEMA_V1=1 + NYASH_MIR_UNIFIED_CALL=1
- 診断既定OFF: HAKO_BRIDGE_METHODIZE=1core_bridge側の補助。Hako既定化後に撤去
Rollback
- Disable HAKO_MIR_BUILDER_METHODIZE; remove methodization rewrite; keep Global path active.
- core_bridgeの methodize ブリッジは Hako側が既定化され次第、撤去タグ: [bridge/methodize:*] を一時観測可能にして差分検知)

View File

@ -0,0 +1,67 @@
# Benchmarking MIR Generation & AOT (Quickstart)
This guide shows how to measure Hakorune's MIR emit (StageB → MIR(JSON)) and AOT build (MIR(JSON) → obj/exe) without llvmlite. All commands are semanticspreserving and keep defaults conservative (failfast, O0).
## Prerequisites
- Build binaries once (release):
- `cargo build --release`
- `cargo build --release -p nyash-llvm-compiler` (ny-llvmc)
- `cargo build --release -p nyash_kernel` (NyRT static runtime)
## Scripts
### 1) MIR emit bench (StageB → MIR(JSON))
- Script: `tools/perf/bench_hakorune_emit_mir.sh`
- Usage: `tools/perf/bench_hakorune_emit_mir.sh <input.hako> [rounds]`
- Output CSV: `round,ms,size,sha1` (sha1 is normalized JSON digest; identical = structure equal)
- Useful env toggles:
- `HAKO_USING_RESOLVER_FIRST=1` (resolverfirst)
- `HAKO_SELFHOST_BUILDER_FIRST=1` (selfhost builder → provider fallback)
- `HAKO_MIR_BUILDER_BOX=hako.mir.builder|min` (builder selector)
- `HAKO_SELFHOST_TRACE=1` (stderr trace)
Example:
```
tools/perf/bench_hakorune_emit_mir.sh apps/examples/json_query/main.hako 5
```
### 2) MIR(JSON) → obj/exe benchny-llvmc / crate backend
- Script: `tools/perf/bench_ny_mir_builder.sh`
- Usage: `tools/perf/bench_ny_mir_builder.sh <mir.json> [rounds]`
- Output CSV: `kind,round,ms`kind = obj|exe
- Useful env toggles:
- `NYASH_LLVM_BACKEND=crate`既定。ny-llvmc を使う)
- `HAKO_LLVM_OPT_LEVEL=0|1`(既定は 0O0
Example:
```
tools/perf/bench_ny_mir_builder.sh /path/to/out.json 3
```
### 3) MIR(JSON) 構造比較
- Script: `tools/perf/compare_mir_json.sh`
- Usage: `tools/perf/compare_mir_json.sh <a.json> <b.json>`
- 出力: サイズと正規化 SHA1、差分jq -S 利用時は整形差分)。
## Typical Workflow
1) Emit MIR(JSON)
- `tools/hakorune_emit_mir.sh apps/APP/main.hako out.json`
2) Measure MIR emit time
- `HAKO_USING_RESOLVER_FIRST=1 tools/perf/bench_hakorune_emit_mir.sh apps/APP/main.hako 5`
3) Measure AOTobj/exe
- `NYASH_LLVM_BACKEND=crate tools/perf/bench_ny_mir_builder.sh out.json 3`
4) Compare MIR outputs across toggles/branches
- `tools/perf/compare_mir_json.sh out_before.json out_after.json`
## Notes
- All benches are besteffort micromeasurements; run multiple rounds and compare medians.
- Keep defaults strict: resolver/selfhost togglesは明示時のみON。AOTは O0 既定(`HAKO_LLVM_OPT_LEVEL` で上げられます)。