docs/ci: selfhost bootstrap/exe-first workflows; add ny-llvmc scaffolding + JSON v0 schema validation; plan: unify to Nyash ABI v2 (no backwards compat)

This commit is contained in:
Selfhosting Dev
2025-09-17 20:33:19 +09:00
parent a5054a271b
commit 4ea3ca2685
56 changed files with 2275 additions and 1623 deletions

View File

@ -0,0 +1,50 @@
name: Selfhost Bootstrap Smoke
on:
push:
paths:
- 'src/**'
- 'apps/**'
- 'tools/**'
- 'docs/**'
- 'Cargo.toml'
- 'Cargo.lock'
- '.github/workflows/selfhost-bootstrap.yml'
pull_request:
paths:
- 'src/**'
- 'apps/**'
- 'tools/**'
- 'docs/**'
jobs:
selfhost-bootstrap:
runs-on: ubuntu-latest
timeout-minutes: 10
env:
CARGO_TERM_COLOR: always
NYASH_DISABLE_PLUGINS: '1'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Rust (stable)
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Build (release, cranelift-jit)
run: cargo build --release --features cranelift-jit -j 2
- name: Run bootstrap selfhost smoke
run: timeout -s KILL 40s bash tools/bootstrap_selfhost_smoke.sh

View File

@ -0,0 +1,57 @@
name: Selfhost EXE-first (Optional)
on:
workflow_dispatch:
schedule:
- cron: '0 7 * * *'
jobs:
selfhost-exe-first:
runs-on: ubuntu-latest
timeout-minutes: 25
env:
CARGO_TERM_COLOR: always
NYASH_DISABLE_PLUGINS: '1'
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Rust (stable)
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and build
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Install LLVM 18 (llvm-config-18)
run: |
sudo apt-get update
sudo apt-get install -y curl ca-certificates lsb-release wget gnupg python3-pip
curl -fsSL https://apt.llvm.org/llvm.sh -o llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 18
llvm-config-18 --version
- name: Install llvmlite (Python)
run: |
python3 -m pip install --upgrade pip
python3 -m pip install llvmlite jsonschema
- name: Build nyash (release)
run: cargo build --release -j 2
- name: ny-llvmc dummy smoke
run: |
cargo build --release -p nyash-llvm-compiler -j 2
./target/release/ny-llvmc --dummy --out /tmp/dummy.o
file /tmp/dummy.o || true
- name: Run EXE-first smoke (parser EXE + bridge)
run: timeout -s KILL 10m bash tools/exe_first_smoke.sh

View File

@ -3,6 +3,7 @@
Summary Summary
- Default execution is MIR13 (PHIoff). Bridge/Builder do not emit PHIs; llvmlite synthesizes PHIs when needed. MIR14 (PHIon) remains experimental for targeted tests. - Default execution is MIR13 (PHIoff). Bridge/Builder do not emit PHIs; llvmlite synthesizes PHIs when needed. MIR14 (PHIon) remains experimental for targeted tests.
- PyVM is the semantic reference engine; llvmlite is used for AOT and parity checks. - PyVM is the semantic reference engine; llvmlite is used for AOT and parity checks.
- GC: user modes defined; controller実装rc+cycle skeleton + metrics/diagnosticsに移行。LLVM safepoint輸出/NyRT配線と自動挿入envゲートONを完了。
What Changed (recent) What Changed (recent)
- MIR13 default enabled - MIR13 default enabled
@ -16,30 +17,56 @@ What Changed (recent)
- Selfhost/PyVM スモークを通して E2E 確認peek/ternary - Selfhost/PyVM スモークを通して E2E 確認peek/ternary
- llvmlite stability for MIR13bringup進行中 - llvmlite stability for MIR13bringup進行中
- Controlflow 分離: `instructions/controlflow/{branch,jump,while_.py}` を導入し、`llvm_builder.py` の責務を縮小。 - Controlflow 分離: `instructions/controlflow/{branch,jump,while_.py}` を導入し、`llvm_builder.py` の責務を縮小。
- プリパス導入(環境変数で有効化): `NYASH_LLVM_PREPASS_LOOP=1` - プリパス(環境変数で有効化): `NYASH_LLVM_PREPASS_LOOP=1`, `NYASH_LLVM_PREPASS_IFMERGE=1`
- ループ検出(単純 while 形)→ 構造化 lowerLoopForm失敗時は regular while - ループ検出(単純 while 形)→ 構造化 lowerLoopForm失敗時は regular while
- CFG ユーティリティ: `cfg/utils.py`preds/succs - ifmergeretmerge前処理: ret 値 PHI の前宣言と finalize 配線の一意化
- 値解決ポリシー共通化: `utils/values.py`prefer sameblock SSA → resolver - CFG ユーティリティ: `src/llvm_py/cfg/utils.py`preds/succs
- vmap の perblock 化: `lower_block` 内で `vmap_cur` を用意し、ブロック末に `block_end_values` へスナップショット。crossblock 汚染を抑制。 - PHI 配線の分離: `src/llvm_py/phi_wiring.py` に placeholder/finalize を移管builder 薄化)
- Resolver 強化: endofblock解決で他ブロックのPHIを安易に採用しない自己参照/非支配回避)。 - 値解決ポリシー共通化: `src/llvm_py/utils/values.py`prefer sameblock SSA → resolver
- vmap の perblock 化: `vmap_cur` を用意し、ブロック末に `block_end_values` へスナップショット。crossblock 汚染を抑制。
- Resolver 強化: endofblock 解決で他ブロック PHI を安易に採用しない(自己参照/非支配を回避)。
- BuildCtx 導入: `src/llvm_py/build_ctx.py` で lowering 引数を集約compare/ret/call/boxcall/externcall/typeop/newbox/safepoint が ctx 対応)
- トレース統一: `src/llvm_py/trace.py` を追加し、`NYASH_CLI_VERBOSE`/`NYASH_LLVM_TRACE_PHI`/`NYASH_LLVM_TRACE_VALUES` を一元管理
- curated スモーク拡張: `tools/smokes/curated_llvm.sh --with-if-merge` を追加ifmerge ケース含む)
- Parity runner pragmatics - Parity runner pragmatics
- `tools/pyvm_vs_llvmlite.sh` compares exit code by default; use `CMP_STRICT=1` for stdout+exit. - `tools/pyvm_vs_llvmlite.sh` compares exit code by default; use `CMP_STRICT=1` for stdout+exit.
- Stage2 smokes更新: `tools/selfhost_stage2_smoke.sh` に "Peek basic" を追加。 - Stage2 smokes更新: `tools/selfhost_stage2_smoke.sh` に "Peek basic" を追加。
- GC controller/metricsPhase15
- `GcController`統合フック導入CLI `--gc``NYASH_GC_MODE`。CountingGcは互換ラッパに縮退。
- NyRT exports: `ny_safepoint` / `ny_check_safepoint` / `nyash.gc.barrier_write` → runtime hooks 連携。
- LLVM自動 safepoint 挿入loop header / call / externcall / boxcall`NYASH_LLVM_AUTO_SAFEPOINT` で制御(既定=1
- メトリクスtext/JSON`NYASH_GC_METRICS{,_JSON}=1`。JSONに alloc_count/alloc_bytes/trial_nodes/edges/collections/last_ms/reason_bits/thresholds を含む。
- 診断:`NYASH_GC_LEAK_DIAG=1` でハンドルTopK残存出力。Array/Map 到達集合の試走gc_trace
- CI/DevOpsSelfHosting パイロット強化)
- 追加: `.github/workflows/selfhost-bootstrap.yml`(常時) — `tools/bootstrap_selfhost_smoke.sh` を40s timeoutで実行。
- 追加: `.github/workflows/selfhost-exe-first.yml`(任意/cron — LLVM18 + llvmlite をセットアップし `tools/exe_first_smoke.sh` を実行。
- スモーク堅牢化: `tools/bootstrap_selfhost_smoke.sh`/`tools/exe_first_smoke.sh` に timeout を付与。
- JSON v0 スキーマ追加: `docs/reference/mir/json_v0.schema.json` と検証ツール `tools/validate_mir_json.py`。EXEfirst スモークに組み込み。
- LLVM crate 分離の足場Phase15.6 向け)
- 新規クレート(スキャフォールド): `crates/nyash-llvm-compiler`CLI名: `ny-llvmc`)。
- `--dummy --out <o>` でダミー `.o` を生成。
- `--in <mir.json|-> --out <o>` で MIR(JSON)→`.o` を llvmlite ハーネス経由で生成(`tools/llvmlite_harness.py`)。
- ドキュメント追記: `docs/LLVM_HARNESS.md``ny-llvmc` とスキーマ検証の項を追加。
- Nyash ABI v2TypeBox検出の足場
- ローダに `nyash_typebox_<Box>` シンボル検出を追加(`abi_tag='TYBX'`/`version`/`invoke_id`し、Boxスペックへ保持まだ実行には未使用
Current Status Current Status
- Selfhosting Bridge → PyVM smokes: PASSStage2 代表: array/string/logic/if/loop/ternary/peek/dot-chain - Selfhosting Bridge → PyVM smokes: PASSStage2 代表: array/string/logic/if/loop/ternary/peek/dot-chain
- PyVM core fixes applied: compare(None,x) の安全化、Copy 命令サポート、最大ステップ上限NYASH_PYVM_MAX_STEPS - PyVM core fixes applied: compare(None,x) の安全化、Copy 命令サポート、最大ステップ上限NYASH_PYVM_MAX_STEPS
- MIR13PHIoff: if/ternary/loop の合流で Copy が正しく JSON に出るよう修正emit_mir_json + builder nophi 合流) - MIR13PHIoff: if/ternary/loop の合流で Copy が正しく JSON に出るよう修正emit_mir_json + builder nophi 合流)
- Curated LLVMPHIoff 既定): 継続(個別ケースの IR 生成不備は未着手) - Curated LLVMPHIoff 既定): 継続(個別ケースの IR 生成不備は未着手)
- LLVM ハーネスllvmlite: LLVM ハーネスllvmlite/AOT:
- `loop_if_phi`: プリパスON構造化whileで EXE 退出コード 0 - `loop_if_phi`: プリパスON構造化whileで EXE 退出コード 0
- `ternary_nested`: vmap perblock で安定度向上。残タスク: merge(ret) の PHI 配線をプリパス/resolve 側で確定・重複排除 - `ternary_nested`: ifmerge プリパスphi_wiring で retmerge を構造化し、退出コード一致(緑)
Next (short plan) Next (short plan)
0) Refactor/Structure継続 0) Refactor/Structure継続
- controlflow の切出し完了branch/jump/while。binop/compare/copy の前処理を `utils/values.resolve_i64_strict` に集約(完了)。 - BuildCtx 展開を完了barrier/atomic/loopform も ctx 主経路に)
- vmap perblock 化完了。builder の責務縮小と prepass/cfg/util への移譲(進行中)。 - trace 化の残り掃除(環境直読み print を削減)
- ifmerge プリパス実装: retmerge の構造化/PHI確定予定 - phi_wiring を関数分割(解析/配線/タグ付け)→ ユニットテスト追加
1) Legacy Interpreter/VM offboarding (phaseA): 1) Legacy Interpreter/VM offboarding (phaseA):
- ✅ Introduced `vm-legacy` feature (default OFF) to gate old VM execution層。 - ✅ Introduced `vm-legacy` feature (default OFF) to gate old VM execution層。
- ✅ 抽出: JIT が参照する最小型(例: `VMValue`)を薄い共通モジュールへ切替(`vm_types`)。 - ✅ 抽出: JIT が参照する最小型(例: `VMValue`)を薄い共通モジュールへ切替(`vm_types`)。
@ -50,23 +77,48 @@ Next (short plan)
2) Legacy Interpreter/VM offboarding (phaseB): 2) Legacy Interpreter/VM offboarding (phaseB):
- 物理移動: `src/archive/{interpreter_legacy,vm_legacy}/` へ移設(ドキュメント更新)。 - 物理移動: `src/archive/{interpreter_legacy,vm_legacy}/` へ移設(ドキュメント更新)。
3) LLVM/llvmlite 整備(優先中): 3) LLVM/llvmlite 整備(優先中):
- MIR13 の Copy 合流を LLVM IR に等価反映predlocalize or PHI 合成): perblock vmap 完了、resolver 強化済。 - MIR13 の Copy 合流を LLVM IR に等価反映predlocalize or PHI 合成): perblock vmap 完了、resolver/phi_wiring 強化済。
- 代表ケース: - 代表ケース:
- `apps/tests/loop_if_phi.nyash`: プリパスONで緑退出コード一致 - `apps/tests/loop_if_phi.nyash`: プリパスONで緑退出コード一致
- `apps/tests/ternary_nested.nyash`: ifmerge プリパスでの構造化/PHI 確定を実装 → IR 検証通過・退出コード一致まで - `apps/tests/ternary_nested.nyash`: ifmerge + phi_wiring で退出コード一致を継続
- `tools/pyvm_vs_llvmlite.sh` で PyVM と EXE の退出コード一致(必要に応じて CMP_STRICT=1 - `tools/pyvm_vs_llvmlite.sh` で PyVM と EXE の退出コード一致(必要に応じて CMP_STRICT=1
4) PHIon lane任意: `loop_if_phi` 支配関係を finalize/resolve の順序強化で観察(低優先)。 4) PHIon lane任意: `loop_if_phi` 支配関係を finalize/resolve の順序強化で観察(低優先)。
5) Runner refactor小PR: 5) Runner refactor小PR:
- `selfhost/{child.rs,json.rs}` 分離; `modes/common/{io,resolve,exec}.rs` 分割; `runner/mod.rs`の表面削減。 - `selfhost/{child.rs,json.rs}` 分離; `modes/common/{io,resolve,exec}.rs` 分割; `runner/mod.rs`の表面削減。
6) Optimizer/Verifier thinhub cleanup非機能: orchestrator最小化とパス境界の明確化。 6) Optimizer/Verifier thinhub cleanup非機能: orchestrator最小化とパス境界の明確化。
7) GCcontroller観測の磨き込み
- JSON: running averages / roots要約任意 / 理由タグ拡張
- 収集頻度のサンプリング支援
- plugin/FFI は非移動のまま、ハンドル間接を継続
8) LLVM crate splitEXEfirst
- LLVM harness/builder を `nyash-llvm-compiler` crate と CLI`ny-llvmc`)に分離(入力: MIR JSON v0 / 出力: .o/.exe
- `tools/build_llvm.sh` 内部を新crate APIに寄せ、Runnerからも呼べるよう段階移行
- CI: selfhost smokes と LLVM EXE smokes を分離しアーティファクト配布線を評価
9) Nyash ABI v2 統一(後方互換なし)
- 方針: 既存 TypeC ABIlibrarylevel `nyash_plugin_invoke`を撤退し、Box単位の TypeBox へ一本化。
- ローダ: `nyash_typebox_<Box>``invoke_id(instance_id, method_id, ...)` を実行ポインタとして保持し、birth/fini も含めて統一。
- プラグイン: 公式プラグインString/File/Array/Map/Console/Integerを順次 v2 へ移行。`resolve(name)->method_id` 実装。
- 仕様: エラー規約OK/E_SHORT/E_ARGS/E_TYPE/E_METHOD/E_HANDLE/E_PLUGIN・TLVタグ一覧を docs に凍結、Cヘッダ雛形`nyash_abi.h`)を配布。
- CI: v2専用スモークを常時化Linux。Windows/macOS は任意ジョブで追随。
How to Run How to Run
- PyVM reference smokes: `tools/pyvm_stage2_smoke.sh` - PyVM reference smokes: `tools/pyvm_stage2_smoke.sh`
- Bridge → PyVM smokes: `tools/selfhost_stage2_bridge_smoke.sh` - Bridge → PyVM smokes: `tools/selfhost_stage2_bridge_smoke.sh`
- LLVM curated (PHIoff default): `tools/smokes/curated_llvm.sh` - LLVM curated (PHIoff default): `tools/smokes/curated_llvm.sh`
- LLVM PHIon (experimental): `tools/smokes/curated_llvm.sh --phi-on` - LLVM PHIon (experimental): `tools/smokes/curated_llvm.sh --phi-on`
- LLVM curated with ifmerge prepass: `tools/smokes/curated_llvm.sh --with-if-merge`
- Parity (AOT vs PyVM): `tools/pyvm_vs_llvmlite.sh <file.nyash>` (`CMP_STRICT=1` to enable stdout check) - Parity (AOT vs PyVM): `tools/pyvm_vs_llvmlite.sh <file.nyash>` (`CMP_STRICT=1` to enable stdout check)
- 開発時の補助: `NYASH_LLVM_PREPASS_LOOP=1` を併用loop/ifmerge のプリパス有効化)。 - 開発時の補助: `NYASH_LLVM_PREPASS_LOOP=1` を併用loop/ifmerge のプリパス有効化)。
- GC modes/metrics: see `docs/reference/runtime/gc.md``--gc` / 自動 safepoint / 収集トリガ / JSONメトリクス
SelfHosting CI
- Bootstrap常時: `.github/workflows/selfhost-bootstrap.yml`
- EXEfirst任意: `.github/workflows/selfhost-exe-first.yml`
LLVM Crate試用
- ダミー: `cargo build -p nyash-llvm-compiler --release && ./target/release/ny-llvmc --dummy --out /tmp/dummy.o`
- JSON→.o: `./target/release/ny-llvmc --in mir.json --out out.o`
Operational Notes Operational Notes
- 環境変数 - 環境変数
@ -93,9 +145,13 @@ Key Flags
- `NYASH_VERIFY_ALLOW_NO_PHI` (default 1): relax verifier for PHIless MIR. - `NYASH_VERIFY_ALLOW_NO_PHI` (default 1): relax verifier for PHIless MIR.
- `NYASH_LLVM_USE_HARNESS=1`: route AOT through llvmlite harness. - `NYASH_LLVM_USE_HARNESS=1`: route AOT through llvmlite harness.
- `NYASH_LLVM_TRACE_PHI=1`: trace PHI resolution/wiring. - `NYASH_LLVM_TRACE_PHI=1`: trace PHI resolution/wiring.
- `NYASH_LLVM_PREPASS_LOOP=1`: enable loop prepass (while detection/structure)
- `NYASH_LLVM_PREPASS_IFMERGE=1`: enable ifmerge (retmerge) prepass
- `NYASH_LLVM_TRACE_VALUES=1`: trace value resolution path
Notes / Policies Notes / Policies
- Focus is selfhosting stability. JIT/Cranelift is out of scope (safety fixes only). - Focus is selfhosting stability. JIT/Cranelift is out of scope (safety fixes only).
- PHI generation remains centralized in llvmlite; Bridge/Builder keep PHIoff by default. - PHI generation remains centralized in llvmlite; Bridge/Builder keep PHIoff by default.
- No full tracing GC yet; handles/Arc lifetimes govern object retention. Safepoint/barrier/roots are staging utilities. - No full tracing/moving GC yet; handles/Arc lifetimes govern object retention. Safepoint/barrier/roots are staging utilities.
- GC mode UX: keep userfacing modes minimal (rc+cycle, minorgen); advanced modes are optin for language dev.
- Legacy Interpreter/VM は段階的にアーカイブへ。日常の意味論確認は PyVM を基準として継続。 - Legacy Interpreter/VM は段階的にアーカイブへ。日常の意味論確認は PyVM を基準として継続。

BIN
app_async Normal file

Binary file not shown.

BIN
app_gc_smoke Normal file

Binary file not shown.

View File

@ -0,0 +1,11 @@
[package]
name = "nyash-llvm-compiler"
version = "0.1.0"
edition = "2021"
description = "Nyash LLVM compiler CLI (harness wrapper). Compiles MIR(JSON) -> object (.o) or dummy."
[dependencies]
anyhow = "1.0"
clap = { version = "4.5", features = ["derive"] }
serde_json = "1.0"

View File

@ -0,0 +1,122 @@
use std::fs::File;
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use std::process::Command;
use anyhow::{bail, Context, Result};
use clap::{ArgAction, Parser};
#[derive(Parser, Debug)]
#[command(name = "ny-llvmc", about = "Nyash LLVM compiler (llvmlite harness wrapper)")]
struct Args {
/// MIR JSON input file path (use '-' to read from stdin). When omitted with --dummy, a dummy ny_main is emitted.
#[arg(long = "in", value_name = "FILE", default_value = "-")]
infile: String,
/// Output object file (.o)
#[arg(long, value_name = "FILE")]
out: PathBuf,
/// Generate a dummy object (ny_main -> i32 0). Ignores --in when set.
#[arg(long, action = ArgAction::SetTrue)]
dummy: bool,
/// Path to Python harness script (defaults to tools/llvmlite_harness.py in CWD)
#[arg(long, value_name = "FILE")]
harness: Option<PathBuf>,
}
fn main() -> Result<()> {
let args = Args::parse();
// Ensure parent dir exists
if let Some(parent) = args.out.parent() {
std::fs::create_dir_all(parent).ok();
}
// Resolve harness path
let harness_path = if let Some(p) = args.harness.clone() {
p
} else {
PathBuf::from("tools/llvmlite_harness.py")
};
if args.dummy {
run_harness_dummy(&harness_path, &args.out)
.with_context(|| "failed to run harness in dummy mode")?;
println!("[ny-llvmc] dummy object written: {}", args.out.display());
return Ok(());
}
// Prepare input JSON path: either from file or stdin -> temp file
let mut temp_path: Option<PathBuf> = None;
let input_path = if args.infile == "-" {
let mut buf = String::new();
std::io::stdin()
.read_to_string(&mut buf)
.context("reading MIR JSON from stdin")?;
// Basic sanity check that it's JSON
let _: serde_json::Value = serde_json::from_str(&buf)
.context("stdin does not contain valid JSON")?;
let tmp = std::env::temp_dir().join("ny_llvmc_stdin.json");
let mut f = File::create(&tmp).context("create temp json file")?;
f.write_all(buf.as_bytes()).context("write temp json")?;
temp_path = Some(tmp.clone());
tmp
} else {
PathBuf::from(&args.infile)
};
if !input_path.exists() {
bail!("input JSON not found: {}", input_path.display());
}
run_harness_in(&harness_path, &input_path, &args.out)
.with_context(|| format!("failed to compile MIR JSON via harness: {}", input_path.display()))?;
println!("[ny-llvmc] object written: {}", args.out.display());
// Cleanup temp file if used
if let Some(p) = temp_path {
let _ = std::fs::remove_file(p);
}
Ok(())
}
fn run_harness_dummy(harness: &Path, out: &Path) -> Result<()> {
ensure_python()?;
let status = Command::new("python3")
.arg(harness)
.arg("--out")
.arg(out)
.status()
.context("failed to execute python harness (dummy)")?;
if !status.success() {
bail!("harness exited with status: {:?}", status.code());
}
Ok(())
}
fn run_harness_in(harness: &Path, input: &Path, out: &Path) -> Result<()> {
ensure_python()?;
let status = Command::new("python3")
.arg(harness)
.arg("--in")
.arg(input)
.arg("--out")
.arg(out)
.status()
.context("failed to execute python harness")?;
if !status.success() {
bail!("harness exited with status: {:?}", status.code());
}
Ok(())
}
fn ensure_python() -> Result<()> {
match Command::new("python3").arg("--version").output() {
Ok(out) if out.status.success() => Ok(()),
_ => bail!("python3 not found in PATH (required for llvmlite harness)"),
}
}

View File

@ -78,6 +78,7 @@ pub extern "C" fn nyash_string_concat_hh_export(a_h: i64, b_h: i64) -> i64 {
String::new() String::new()
}; };
let s = format!("{}{}", to_s(a_h), to_s(b_h)); let s = format!("{}{}", to_s(a_h), to_s(b_h));
nyash_rust::runtime::global_hooks::gc_alloc(s.len() as u64);
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s)); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s));
let h = handles::to_handle(arc) as i64; let h = handles::to_handle(arc) as i64;
eprintln!("[TRACE] concat_hh -> {}", h); eprintln!("[TRACE] concat_hh -> {}", h);
@ -134,6 +135,7 @@ pub extern "C" fn nyash_string_substring_hii_export(h: i64, start: i64, end: i64
let (st_u, en_u) = (st as usize, en as usize); let (st_u, en_u) = (st as usize, en as usize);
let sub = s.get(st_u.min(s.len())..en_u.min(s.len())).unwrap_or(""); let sub = s.get(st_u.min(s.len())..en_u.min(s.len())).unwrap_or("");
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(sub.to_string())); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(sub.to_string()));
nyash_rust::runtime::global_hooks::gc_alloc(sub.len() as u64);
let nh = handles::to_handle(arc) as i64; let nh = handles::to_handle(arc) as i64;
eprintln!("[TRACE] substring_hii -> {}", nh); eprintln!("[TRACE] substring_hii -> {}", nh);
nh nh
@ -196,7 +198,8 @@ pub extern "C" fn nyash_box_from_i8_string(ptr: *const i8) -> i64 {
Ok(v) => v.to_string(), Ok(v) => v.to_string(),
Err(_) => return 0, Err(_) => return 0,
}; };
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s)); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s.clone()));
nyash_rust::runtime::global_hooks::gc_alloc(s.len() as u64);
let h = handles::to_handle(arc) as i64; let h = handles::to_handle(arc) as i64;
eprintln!("[TRACE] from_i8_string -> {}", h); eprintln!("[TRACE] from_i8_string -> {}", h);
h h
@ -208,6 +211,7 @@ pub extern "C" fn nyash_box_from_i8_string(ptr: *const i8) -> i64 {
pub extern "C" fn nyash_box_from_f64(val: f64) -> i64 { pub extern "C" fn nyash_box_from_f64(val: f64) -> i64 {
use nyash_rust::{box_trait::NyashBox, boxes::FloatBox, jit::rt::handles}; use nyash_rust::{box_trait::NyashBox, boxes::FloatBox, jit::rt::handles};
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(FloatBox::new(val)); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(FloatBox::new(val));
nyash_rust::runtime::global_hooks::gc_alloc(8);
handles::to_handle(arc) as i64 handles::to_handle(arc) as i64
} }
@ -220,6 +224,7 @@ pub extern "C" fn nyash_box_from_i64(val: i64) -> i64 {
jit::rt::handles, jit::rt::handles,
}; };
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(IntegerBox::new(val)); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(IntegerBox::new(val));
nyash_rust::runtime::global_hooks::gc_alloc(8);
handles::to_handle(arc) as i64 handles::to_handle(arc) as i64
} }
@ -487,7 +492,8 @@ pub extern "C" fn nyash_string_from_u64x2_export(lo: i64, hi: i64, len: i64) ->
bytes.push(((hi_u >> (8 * i)) & 0xff) as u8); bytes.push(((hi_u >> (8 * i)) & 0xff) as u8);
} }
let s = String::from_utf8_lossy(&bytes).to_string(); let s = String::from_utf8_lossy(&bytes).to_string();
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s)); let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s.clone()));
nyash_rust::runtime::global_hooks::gc_alloc(s.len() as u64);
handles::to_handle(arc) as i64 handles::to_handle(arc) as i64
} }
@ -542,9 +548,25 @@ pub extern "C" fn nyash_gc_barrier_write_export(handle_or_ptr: i64) -> i64 {
if std::env::var("NYASH_GC_BARRIER_TRACE").ok().as_deref() == Some("1") { if std::env::var("NYASH_GC_BARRIER_TRACE").ok().as_deref() == Some("1") {
eprintln!("[nyrt] nyash.gc.barrier_write h=0x{:x}", handle_or_ptr); eprintln!("[nyrt] nyash.gc.barrier_write h=0x{:x}", handle_or_ptr);
} }
// Forward to runtime GC hooks when available (Write barrier)
nyash_rust::runtime::global_hooks::gc_barrier(nyash_rust::runtime::BarrierKind::Write);
0 0
} }
// LLVM safepoint exports (llvmlite harness)
// export: ny_safepoint(live_count: i64, live_values: i64*) -> void
#[no_mangle]
pub extern "C" fn ny_safepoint(_live_count: i64, _live_values: *const i64) {
// For now we ignore live-values; runtime uses cooperative safepoint + poll
nyash_rust::runtime::global_hooks::safepoint_and_poll();
}
// export: ny_check_safepoint() -> void
#[no_mangle]
pub extern "C" fn ny_check_safepoint() {
nyash_rust::runtime::global_hooks::safepoint_and_poll();
}
#[export_name = "nyash.string.birth_h"] #[export_name = "nyash.string.birth_h"]
pub extern "C" fn nyash_string_birth_h_export() -> i64 { pub extern "C" fn nyash_string_birth_h_export() -> i64 {
// Create a new StringBox via unified plugin host; return runtime handle as i64 // Create a new StringBox via unified plugin host; return runtime handle as i64
@ -552,6 +574,7 @@ pub extern "C" fn nyash_string_birth_h_export() -> i64 {
if let Ok(b) = host_g.create_box("StringBox", &[]) { if let Ok(b) = host_g.create_box("StringBox", &[]) {
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b); let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b);
let h = nyash_rust::jit::rt::handles::to_handle(arc); let h = nyash_rust::jit::rt::handles::to_handle(arc);
nyash_rust::runtime::global_hooks::gc_alloc(0);
return h as i64; return h as i64;
} }
} }
@ -564,6 +587,7 @@ pub extern "C" fn nyash_integer_birth_h_export() -> i64 {
if let Ok(b) = host_g.create_box("IntegerBox", &[]) { if let Ok(b) = host_g.create_box("IntegerBox", &[]) {
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b); let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b);
let h = nyash_rust::jit::rt::handles::to_handle(arc); let h = nyash_rust::jit::rt::handles::to_handle(arc);
nyash_rust::runtime::global_hooks::gc_alloc(0);
return h as i64; return h as i64;
} }
} }
@ -576,6 +600,7 @@ pub extern "C" fn nyash_console_birth_h_export() -> i64 {
if let Ok(b) = host_g.create_box("ConsoleBox", &[]) { if let Ok(b) = host_g.create_box("ConsoleBox", &[]) {
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b); let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = std::sync::Arc::from(b);
let h = nyash_rust::jit::rt::handles::to_handle(arc); let h = nyash_rust::jit::rt::handles::to_handle(arc);
nyash_rust::runtime::global_hooks::gc_alloc(0);
return h as i64; return h as i64;
} }
} }
@ -587,6 +612,7 @@ pub extern "C" fn nyash_console_birth_h_export() -> i64 {
pub extern "C" fn nyash_array_birth_h_export() -> i64 { pub extern "C" fn nyash_array_birth_h_export() -> i64 {
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(nyash_rust::boxes::array::ArrayBox::new()); std::sync::Arc::new(nyash_rust::boxes::array::ArrayBox::new());
nyash_rust::runtime::global_hooks::gc_alloc(0);
nyash_rust::jit::rt::handles::to_handle(arc) as i64 nyash_rust::jit::rt::handles::to_handle(arc) as i64
} }
@ -595,6 +621,7 @@ pub extern "C" fn nyash_array_birth_h_export() -> i64 {
pub extern "C" fn nyash_map_birth_h_export() -> i64 { pub extern "C" fn nyash_map_birth_h_export() -> i64 {
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> = let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(nyash_rust::boxes::map_box::MapBox::new()); std::sync::Arc::new(nyash_rust::boxes::map_box::MapBox::new());
nyash_rust::runtime::global_hooks::gc_alloc(0);
nyash_rust::jit::rt::handles::to_handle(arc) as i64 nyash_rust::jit::rt::handles::to_handle(arc) as i64
} }
// ---- Process entry (driver) ---- // ---- Process entry (driver) ----
@ -646,6 +673,15 @@ pub extern "C" fn main() -> i32 {
} }
} }
} }
// Initialize a minimal runtime to back global hooks (GC/scheduler) for safepoints
// Choose GC hooks based on env (default dev: Counting for observability unless explicitly off)
let mut rt_builder = nyash_rust::runtime::NyashRuntimeBuilder::new();
let gc_mode = nyash_rust::runtime::gc_mode::GcMode::from_env();
let controller = std::sync::Arc::new(nyash_rust::runtime::gc_controller::GcController::new(gc_mode));
rt_builder = rt_builder.with_gc_hooks(controller);
let rt_hooks = rt_builder.build();
nyash_rust::runtime::global_hooks::set_from_runtime(&rt_hooks);
let mut inited = false; let mut inited = false;
if let Some(dir) = &exe_dir { if let Some(dir) = &exe_dir {
let candidate = dir.join("nyash.toml"); let candidate = dir.join("nyash.toml");
@ -680,6 +716,70 @@ pub extern "C" fn main() -> i32 {
let v = ny_main(); let v = ny_main();
// Print standardized result line for golden comparisons // Print standardized result line for golden comparisons
println!("Result: {}", v); println!("Result: {}", v);
// Optional GC metrics after program completes
let want_json = std::env::var("NYASH_GC_METRICS_JSON").ok().as_deref() == Some("1");
let want_text = std::env::var("NYASH_GC_METRICS").ok().as_deref() == Some("1");
if want_json || want_text {
let (sp, br, bw) = rt_hooks
.gc
.snapshot_counters()
.unwrap_or((0, 0, 0));
let handles = nyash_rust::jit::rt::handles::len();
let gc_mode_s = gc_mode.as_str();
// Include allocation totals if controller is used
let any_gc: &dyn std::any::Any = &*rt_hooks.gc;
let (alloc_count, alloc_bytes, trial_nodes, trial_edges, collect_total, collect_sp, collect_alloc, last_ms, last_reason) = if let Some(ctrl) = any_gc
.downcast_ref::<nyash_rust::runtime::gc_controller::GcController>()
{
let (ac, ab) = ctrl.alloc_totals();
let (tn, te) = ctrl.trial_reachability_last();
let (ct, csp, calloc) = ctrl.collection_totals();
let lms = ctrl.trial_duration_last_ms();
let lrf = ctrl.trial_reason_last_bits();
(ac, ab, tn, te, ct, csp, calloc, lms, lrf)
} else {
(0, 0, 0, 0, 0, 0, 0, 0, 0)
};
// Settings snapshot (env)
let sp_interval = std::env::var("NYASH_GC_COLLECT_SP").ok().and_then(|s| s.parse::<u64>().ok()).unwrap_or(0);
let alloc_thresh = std::env::var("NYASH_GC_COLLECT_ALLOC").ok().and_then(|s| s.parse::<u64>().ok()).unwrap_or(0);
let auto_sp = std::env::var("NYASH_LLVM_AUTO_SAFEPOINT").ok().map(|v| v == "1").unwrap_or(true);
if want_json {
// Minimal JSON assembly to avoid extra deps in nyrt
println!(
"{{\"kind\":\"gc_metrics\",\"safepoints\":{},\"barrier_reads\":{},\"barrier_writes\":{},\"jit_handles\":{},\"alloc_count\":{},\"alloc_bytes\":{},\"trial_nodes\":{},\"trial_edges\":{},\"collections\":{},\"collect_by_sp\":{},\"collect_by_alloc\":{},\"last_collect_ms\":{},\"last_reason_bits\":{},\"sp_interval\":{},\"alloc_threshold\":{},\"auto_safepoint\":{},\"gc_mode\":\"{}\"}}",
sp, br, bw, handles, alloc_count, alloc_bytes, trial_nodes, trial_edges, collect_total, collect_sp, collect_alloc, last_ms, last_reason, sp_interval, alloc_thresh, if auto_sp {1} else {0}, gc_mode_s
);
} else if want_text {
eprintln!(
"[GC] metrics: safepoints={} read_barriers={} write_barriers={} jit_handles={} allocs={} bytes={} collections={} (sp={} alloc={}) last_ms={} mode={}",
sp, br, bw, handles, alloc_count, alloc_bytes, collect_total, collect_sp, collect_alloc, last_ms, gc_mode_s
);
}
// Threshold warning
if let Ok(s) = std::env::var("NYASH_GC_ALLOC_THRESHOLD") {
if let Ok(th) = s.parse::<u64>() {
if alloc_bytes > th {
eprintln!(
"[GC][warn] allocation bytes {} exceeded threshold {}",
alloc_bytes, th
);
}
}
}
}
// Leak diagnostics: report remaining JIT handles by type (Top-10)
if std::env::var("NYASH_GC_LEAK_DIAG").ok().as_deref() == Some("1") {
let tally = nyash_rust::jit::rt::handles::type_tally();
let total = tally.iter().map(|(_, n)| *n as u64).sum::<u64>();
if total > 0 {
eprintln!("[leak] Remaining handles by type (top 10):");
for (i, (ty, n)) in tally.into_iter().take(10).enumerate() {
eprintln!(" {}. {} x{}", i + 1, ty, n);
}
}
}
v as i32 v as i32
} }
} }

View File

@ -131,7 +131,7 @@ pub extern "C" fn nyash_console_readline_export() -> *mut i8 {
// Use read_to_end if stdin is not a TTY? Simpler: read_line through BufRead // Use read_to_end if stdin is not a TTY? Simpler: read_line through BufRead
// For simplicity, read from stdin into buffer until newline or EOF // For simplicity, read from stdin into buffer until newline or EOF
let mut buf = String::new(); let mut buf = String::new();
let mut handle = io::stdin(); // Note: use std::io::stdin() directly without an unused handle binding
// On failure or EOF, return empty string // On failure or EOF, return empty string
match io::stdin().read_line(&mut buf) { match io::stdin().read_line(&mut buf) {
Ok(_n) => { Ok(_n) => {

View File

@ -114,7 +114,7 @@ pub extern "C" fn nyash_future_spawn_method_h(
let handle = let handle =
nyash_rust::jit::rt::handles::to_handle(fut_box.clone() as std::sync::Arc<dyn NyashBox>); nyash_rust::jit::rt::handles::to_handle(fut_box.clone() as std::sync::Arc<dyn NyashBox>);
// Copy data for async task // Copy data for async task
let mut cap: usize = 512; let cap: usize = 512;
let tlv = buf.clone(); let tlv = buf.clone();
let inv = invoke.unwrap(); let inv = invoke.unwrap();
nyash_rust::runtime::global_hooks::spawn_task( nyash_rust::runtime::global_hooks::spawn_task(

View File

@ -1,4 +1,5 @@
use crate::encode::{nyrt_encode_arg_or_legacy, nyrt_encode_from_legacy_at}; use crate::encode::{nyrt_encode_arg_or_legacy, nyrt_encode_from_legacy_at};
use crate::plugin::invoke_core;
#[no_mangle] #[no_mangle]
pub extern "C" fn nyash_plugin_invoke3_i64( pub extern "C" fn nyash_plugin_invoke3_i64(
type_id: i64, type_id: i64,
@ -8,39 +9,14 @@ pub extern "C" fn nyash_plugin_invoke3_i64(
a1: i64, a1: i64,
a2: i64, a2: i64,
) -> i64 { ) -> i64 {
use nyash_rust::runtime::plugin_loader_v2::PluginBoxV2; // Resolve receiver via shared core helper
// Resolve receiver instance from handle first; fallback to legacy VM args (param index) let recv = match invoke_core::resolve_receiver_for_a0(a0) {
let mut instance_id: u32 = 0; Some(r) => r,
let mut real_type_id: u32 = 0; None => return 0,
let mut invoke: Option< };
unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32, let instance_id: u32 = recv.instance_id;
> = None; let _real_type_id: u32 = recv.real_type_id;
if a0 > 0 { let invoke = recv.invoke;
if let Some(obj) = nyash_rust::jit::rt::handles::get(a0 as u64) {
if let Some(p) = obj.as_any().downcast_ref::<PluginBoxV2>() {
instance_id = p.instance_id();
real_type_id = p.inner.type_id;
invoke = Some(p.inner.invoke_fn);
}
}
}
if invoke.is_none()
&& a0 >= 0
&& std::env::var("NYASH_JIT_ARGS_HANDLE_ONLY").ok().as_deref() != Some("1")
{
nyash_rust::jit::rt::with_legacy_vm_args(|args| {
let idx = a0 as usize;
if let Some(nyash_rust::backend::vm::VMValue::BoxRef(b)) = args.get(idx) {
if let Some(p) = b.as_any().downcast_ref::<PluginBoxV2>() {
instance_id = p.instance_id();
invoke = Some(p.inner.invoke_fn);
}
}
});
}
if invoke.is_none() {
return 0;
}
// Build TLV args from a1/a2 if present. Prefer handles/StringBox/IntegerBox via runtime host. // Build TLV args from a1/a2 if present. Prefer handles/StringBox/IntegerBox via runtime host.
use nyash_rust::{backend::vm::VMValue, jit::rt::handles}; use nyash_rust::{backend::vm::VMValue, jit::rt::handles};
// argc from LLVM lowering is explicit arg count (excludes receiver) // argc from LLVM lowering is explicit arg count (excludes receiver)
@ -48,164 +24,11 @@ pub extern "C" fn nyash_plugin_invoke3_i64(
let mut buf = nyash_rust::runtime::plugin_ffi_common::encode_tlv_header(nargs as u16); let mut buf = nyash_rust::runtime::plugin_ffi_common::encode_tlv_header(nargs as u16);
// Encode legacy VM arg at position into provided buffer (avoid capturing &mut buf) // Encode legacy VM arg at position into provided buffer (avoid capturing &mut buf)
let mut encode_from_legacy_into = |dst: &mut Vec<u8>, arg_pos: usize| { let mut encode_from_legacy_into = |dst: &mut Vec<u8>, arg_pos: usize| {
nyash_rust::jit::rt::with_legacy_vm_args(|args| { nyrt_encode_from_legacy_at(dst, arg_pos)
if let Some(v) = args.get(arg_pos) {
match v {
VMValue::String(s) => {
nyash_rust::runtime::plugin_ffi_common::encode::string(dst, s)
}
VMValue::Integer(i) => {
nyash_rust::runtime::plugin_ffi_common::encode::i64(dst, *i)
}
VMValue::Float(f) => {
nyash_rust::runtime::plugin_ffi_common::encode::f64(dst, *f)
}
VMValue::Bool(b) => {
nyash_rust::runtime::plugin_ffi_common::encode::bool(dst, *b)
}
VMValue::BoxRef(b) => {
// BufferBox → TLV bytes
if let Some(bufbox) = b
.as_any()
.downcast_ref::<nyash_rust::boxes::buffer::BufferBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::bytes(
dst,
&bufbox.to_vec(),
);
return;
}
if let Some(p) = b.as_any().downcast_ref::<PluginBoxV2>() {
// Prefer StringBox/IntegerBox primitives when possible
let host = nyash_rust::runtime::get_global_plugin_host();
if let Ok(hg) = host.read() {
if p.box_type == "StringBox" {
if let Ok(Some(sb)) = hg.invoke_instance_method(
"StringBox",
"toUtf8",
p.instance_id(),
&[],
) {
if let Some(s) = sb
.as_any()
.downcast_ref::<nyash_rust::box_trait::StringBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::string(
dst, &s.value,
);
return;
}
}
} else if p.box_type == "IntegerBox" {
if let Ok(Some(ibx)) = hg.invoke_instance_method(
"IntegerBox",
"get",
p.instance_id(),
&[],
) {
if let Some(i) =
ibx.as_any()
.downcast_ref::<nyash_rust::box_trait::IntegerBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::i64(
dst, i.value,
);
return;
}
}
}
}
// Fallback: pass handle as plugin-handle TLV
nyash_rust::runtime::plugin_ffi_common::encode::plugin_handle(
dst,
p.inner.type_id,
p.instance_id(),
);
} else {
// Stringify unknown boxes
let s = b.to_string_box().value;
nyash_rust::runtime::plugin_ffi_common::encode::string(dst, &s)
}
}
_ => {}
}
}
});
}; };
// Encode argument value or fallback to legacy slot (avoid capturing &mut buf) // Encode argument value or fallback to legacy slot (avoid capturing &mut buf)
let mut encode_arg_into = |dst: &mut Vec<u8>, val: i64, pos: usize| { let mut encode_arg_into = |dst: &mut Vec<u8>, val: i64, pos: usize| {
let mut appended = false; nyrt_encode_arg_or_legacy(dst, val, pos)
// Try handle first
if val > 0 {
if let Some(obj) = handles::get(val as u64) {
// BufferBox handle → TLV bytes
if let Some(bufbox) = obj
.as_any()
.downcast_ref::<nyash_rust::boxes::buffer::BufferBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::bytes(dst, &bufbox.to_vec());
appended = true;
return;
}
if let Some(p) = obj.as_any().downcast_ref::<PluginBoxV2>() {
let host = nyash_rust::runtime::get_global_plugin_host();
if let Ok(hg) = host.read() {
if p.box_type == "StringBox" {
if let Ok(Some(sb)) = hg.invoke_instance_method(
"StringBox",
"toUtf8",
p.instance_id(),
&[],
) {
if let Some(s) = sb
.as_any()
.downcast_ref::<nyash_rust::box_trait::StringBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::string(
dst, &s.value,
);
appended = true;
return;
}
}
} else if p.box_type == "IntegerBox" {
if let Ok(Some(ibx)) =
hg.invoke_instance_method("IntegerBox", "get", p.instance_id(), &[])
{
if let Some(i) = ibx
.as_any()
.downcast_ref::<nyash_rust::box_trait::IntegerBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::i64(
dst, i.value,
);
appended = true;
return;
}
}
}
}
// Otherwise, pass as handle TLV
nyash_rust::runtime::plugin_ffi_common::encode::plugin_handle(
dst,
p.inner.type_id,
p.instance_id(),
);
appended = true;
return;
}
}
}
// Legacy VM args by positional index (1-based for a1)
let before = dst.len();
encode_from_legacy_into(dst, pos);
if dst.len() != before {
appended = true;
}
// If still nothing appended (no-op), fallback to raw i64
if !appended {
nyash_rust::runtime::plugin_ffi_common::encode::i64(dst, val);
}
}; };
if nargs >= 1 { if nargs >= 1 {
encode_arg_into(&mut buf, a1, 1); encode_arg_into(&mut buf, a1, 1);
@ -219,123 +42,20 @@ pub extern "C" fn nyash_plugin_invoke3_i64(
encode_from_legacy_into(&mut buf, pos); encode_from_legacy_into(&mut buf, pos);
} }
} }
// Prepare output buffer (dynamic growth on short buffer) // Call invoke with dynamic buffer logic centralized
let mut cap: usize = 256; let (tag_ret, sz_ret, payload_ret): (u8, usize, Vec<u8>) = match invoke_core::plugin_invoke_call(
let (mut tag_ret, mut sz_ret, mut payload_ret): (u8, usize, Vec<u8>) = (0, 0, Vec::new()); invoke,
loop {
let mut out = vec![0u8; cap];
let mut out_len: usize = out.len();
let rc = unsafe {
invoke.unwrap()(
type_id as u32, type_id as u32,
method_id as u32, method_id as u32,
instance_id, instance_id,
buf.as_ptr(), &buf,
buf.len(), ) {
out.as_mut_ptr(), Some((t, s, p)) => (t, s, p),
&mut out_len, None => return 0,
)
}; };
if rc != 0 {
// Retry on short buffer hint (-1) or when plugin wrote beyond capacity (len > cap)
if rc == -1 || out_len > cap {
cap = cap.saturating_mul(2).max(out_len + 16);
if cap > 1 << 20 {
break;
}
continue;
}
return 0;
}
let slice = &out[..out_len];
if let Some((t, s, p)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(slice) {
tag_ret = t;
sz_ret = s;
payload_ret = p.to_vec();
}
break;
}
if payload_ret.is_empty() {
return 0;
}
if let Some((tag, sz, payload)) = Some((tag_ret, sz_ret, payload_ret.as_slice())) { if let Some((tag, sz, payload)) = Some((tag_ret, sz_ret, payload_ret.as_slice())) {
match tag { if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke) {
2 => { return v;
// I32
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
}
3 => {
// I64
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return i64::from_le_bytes(b);
}
}
6 | 7 => {
// String/Bytes -> register StringBox handle
use nyash_rust::box_trait::{NyashBox, StringBox};
let s = nyash_rust::runtime::plugin_ffi_common::decode::string(payload);
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s));
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
8 => {
// Handle(tag=8) -> register and return handle id (i64)
if sz == 8 {
let mut t = [0u8; 4];
t.copy_from_slice(&payload[0..4]);
let mut i = [0u8; 4];
i.copy_from_slice(&payload[4..8]);
let r_type = u32::from_le_bytes(t);
let r_inst = u32::from_le_bytes(i);
// Build PluginBoxV2 and register into handle-registry
let meta_opt =
nyash_rust::runtime::plugin_loader_v2::metadata_for_type_id(r_type);
let (box_type_name, invoke_ptr) = if let Some(meta) = meta_opt {
(meta.box_type.clone(), meta.invoke_fn)
} else {
("PluginBox".to_string(), invoke.unwrap())
};
let pb = nyash_rust::runtime::plugin_loader_v2::make_plugin_box_v2(
box_type_name.clone(),
r_type,
r_inst,
invoke_ptr,
);
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(pb);
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
}
1 => {
// Bool
return if nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.unwrap_or(false)
{
1
} else {
0
};
}
5 => {
// F64 → optional conversion to i64
if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") {
if sz == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
let f = f64::from_le_bytes(b);
return f as i64;
}
}
}
_ => {}
} }
} }
0 0
@ -484,77 +204,7 @@ pub extern "C" fn nyash_plugin_invoke3_f64(
}); });
}; };
let mut encode_arg = |val: i64, pos: usize| { let mut encode_arg = |val: i64, pos: usize| {
let mut appended = false; crate::encode::nyrt_encode_arg_or_legacy(&mut buf, val, pos)
if val > 0 {
if let Some(obj) = handles::get(val as u64) {
if let Some(bufbox) = obj
.as_any()
.downcast_ref::<nyash_rust::boxes::buffer::BufferBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::bytes(
&mut buf,
&bufbox.to_vec(),
);
appended = true;
return;
}
if let Some(p) = obj.as_any().downcast_ref::<PluginBoxV2>() {
let host = nyash_rust::runtime::get_global_plugin_host();
if let Ok(hg) = host.read() {
if p.box_type == "StringBox" {
if let Ok(Some(sb)) = hg.invoke_instance_method(
"StringBox",
"toUtf8",
p.instance_id(),
&[],
) {
if let Some(s) = sb
.as_any()
.downcast_ref::<nyash_rust::box_trait::StringBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::string(
&mut buf, &s.value,
);
appended = true;
return;
}
}
} else if p.box_type == "IntegerBox" {
if let Ok(Some(ibx)) =
hg.invoke_instance_method("IntegerBox", "get", p.instance_id(), &[])
{
if let Some(i) = ibx
.as_any()
.downcast_ref::<nyash_rust::box_trait::IntegerBox>()
{
nyash_rust::runtime::plugin_ffi_common::encode::i64(
&mut buf, i.value,
);
appended = true;
return;
}
}
}
}
nyash_rust::runtime::plugin_ffi_common::encode::plugin_handle(
&mut buf,
p.inner.type_id,
p.instance_id(),
);
appended = true;
return;
}
}
}
let before = buf.len();
// Use global helper to avoid nested mutable borrows on buf
nyrt_encode_from_legacy_at(&mut buf, pos);
if buf.len() != before {
appended = true;
}
if !appended {
nyash_rust::runtime::plugin_ffi_common::encode::i64(&mut buf, val);
}
}; };
if nargs >= 1 { if nargs >= 1 {
encode_arg(a1, 1); encode_arg(a1, 1);
@ -567,77 +217,20 @@ pub extern "C" fn nyash_plugin_invoke3_f64(
nyrt_encode_from_legacy_at(&mut buf, pos); nyrt_encode_from_legacy_at(&mut buf, pos);
} }
} }
// Prepare output buffer (dynamic growth on short buffer) // Invoke via shared helper
let mut cap: usize = 256; let (mut tag_ret, mut sz_ret, mut payload_ret): (u8, usize, Vec<u8>) = match invoke_core::plugin_invoke_call(
let (mut tag_ret, mut sz_ret, mut payload_ret): (u8, usize, Vec<u8>) = (0, 0, Vec::new()); invoke.unwrap(),
loop {
let mut out = vec![0u8; cap];
let mut out_len: usize = out.len();
let rc = unsafe {
invoke.unwrap()(
type_id as u32, type_id as u32,
method_id as u32, method_id as u32,
instance_id, instance_id,
buf.as_ptr(), &buf,
buf.len(), ) {
out.as_mut_ptr(), Some((t, s, p)) => (t, s, p),
&mut out_len, None => return 0.0,
)
}; };
if rc != 0 {
// Retry on short buffer (-1) or when plugin wrote beyond capacity
if rc == -1 || out_len > cap {
cap = cap.saturating_mul(2).max(out_len + 16);
if cap > 1 << 20 {
break;
}
continue;
}
return 0.0;
}
let slice = &out[..out_len];
if let Some((t, s, p)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(slice) {
tag_ret = t;
sz_ret = s;
payload_ret = p.to_vec();
}
break;
}
if payload_ret.is_empty() {
return 0.0;
}
if let Some((tag, sz, payload)) = Some((tag_ret, sz_ret, payload_ret.as_slice())) { if let Some((tag, sz, payload)) = Some((tag_ret, sz_ret, payload_ret.as_slice())) {
match tag { if let Some(f) = invoke_core::decode_entry_to_f64(tag, sz, payload) {
5 => { return f;
// F64
if sz == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return f64::from_le_bytes(b);
}
}
3 => {
// I64 -> f64
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as f64;
}
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return (i64::from_le_bytes(b)) as f64;
}
}
1 => {
// Bool -> f64
return if nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.unwrap_or(false)
{
1.0
} else {
0.0
};
}
_ => {}
} }
} }
0.0 0.0
@ -832,42 +425,10 @@ fn nyash_plugin_invoke_name_common_i64(method: &str, argc: i64, a0: i64, a1: i64
&mut out_len, &mut out_len,
) )
}; };
if rc != 0 { if rc != 0 { return 0; }
return 0;
}
let out_slice = &out[..out_len]; let out_slice = &out[..out_len];
if let Some((tag, _sz, payload)) = if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(out_slice) {
nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(out_slice) if let Some(v) = super::invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; }
{
match tag {
3 => {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return i64::from_le_bytes(b);
}
}
1 => {
return if nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.unwrap_or(false)
{
1
} else {
0
};
}
5 => {
if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
let f = f64::from_le_bytes(b);
return f as i64;
}
}
}
_ => {}
}
} }
0 0
} }
@ -1106,73 +667,8 @@ pub extern "C" fn nyash_plugin_invoke3_tagged_i64(
if rc != 0 { if rc != 0 {
return 0; return 0;
} }
if let Some((tag, _sz, payload)) = if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) {
nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; }
{
match tag {
2 => {
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
}
3 => {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return i64::from_le_bytes(b);
}
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
}
6 | 7 => {
use nyash_rust::box_trait::{NyashBox, StringBox};
let s = nyash_rust::runtime::plugin_ffi_common::decode::string(payload);
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s));
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
1 => {
return if nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.unwrap_or(false)
{
1
} else {
0
};
}
8 => {
if payload.len() == 8 {
let mut t = [0u8; 4];
t.copy_from_slice(&payload[0..4]);
let mut i = [0u8; 4];
i.copy_from_slice(&payload[4..8]);
let r_type = u32::from_le_bytes(t);
let r_inst = u32::from_le_bytes(i);
let pb = nyash_rust::runtime::plugin_loader_v2::make_plugin_box_v2(
"PluginBox".into(),
r_type,
r_inst,
invoke.unwrap(),
);
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(pb);
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
}
5 => {
if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
let f = f64::from_le_bytes(b);
return f as i64;
}
}
}
_ => {}
}
} }
0 0
} }
@ -1263,73 +759,8 @@ pub extern "C" fn nyash_plugin_invoke_tagged_v_i64(
if rc != 0 { if rc != 0 {
return 0; return 0;
} }
if let Some((tag, _sz, payload)) = if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) {
nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; }
{
match tag {
2 => {
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
}
3 => {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return i64::from_le_bytes(b);
}
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return v as i64;
}
}
6 | 7 => {
use nyash_rust::box_trait::{NyashBox, StringBox};
let s = nyash_rust::runtime::plugin_ffi_common::decode::string(payload);
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s));
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
1 => {
return if nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.unwrap_or(false)
{
1
} else {
0
};
}
8 => {
if payload.len() == 8 {
let mut t = [0u8; 4];
t.copy_from_slice(&payload[0..4]);
let mut i = [0u8; 4];
i.copy_from_slice(&payload[4..8]);
let r_type = u32::from_le_bytes(t);
let r_inst = u32::from_le_bytes(i);
let pb = nyash_rust::runtime::plugin_loader_v2::make_plugin_box_v2(
"PluginBox".into(),
r_type,
r_inst,
invoke.unwrap(),
);
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(pb);
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return h as i64;
}
}
5 => {
if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") {
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
let f = f64::from_le_bytes(b);
return f as i64;
}
}
}
_ => {}
}
} }
0 0
} }

View File

@ -0,0 +1,199 @@
use nyash_rust::runtime::plugin_loader_v2::PluginBoxV2;
/// Thin shared helpers for plugin invoke shims (i64/f64)
///
/// Goal: centralize receiver resolution and the dynamic buffer call loop,
/// keeping extern functions in invoke.rs small and consistent.
pub struct Receiver {
pub instance_id: u32,
pub real_type_id: u32,
pub invoke:
unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32,
}
/// Resolve receiver from a0: prefer handle registry; fallback to legacy VM args when allowed.
pub fn resolve_receiver_for_a0(a0: i64) -> Option<Receiver> {
// 1) Handle registry (preferred)
if a0 > 0 {
if let Some(obj) = nyash_rust::jit::rt::handles::get(a0 as u64) {
if let Some(p) = obj.as_any().downcast_ref::<PluginBoxV2>() {
return Some(Receiver {
instance_id: p.instance_id(),
real_type_id: p.inner.type_id,
invoke: p.inner.invoke_fn,
});
}
}
}
// 2) Legacy VM args (index by a0) unless handle-only is enforced
if a0 >= 0
&& std::env::var("NYASH_JIT_ARGS_HANDLE_ONLY").ok().as_deref() != Some("1")
{
nyash_rust::jit::rt::with_legacy_vm_args(|args| {
let idx = a0 as usize;
if let Some(nyash_rust::backend::vm::VMValue::BoxRef(b)) = args.get(idx) {
if let Some(p) = b.as_any().downcast_ref::<PluginBoxV2>() {
return Some(Receiver {
instance_id: p.instance_id(),
real_type_id: p.inner.type_id,
invoke: p.inner.invoke_fn,
});
}
}
None
})
} else {
None
}
}
/// Call plugin invoke with dynamic buffer growth, returning first TLV entry on success.
pub fn plugin_invoke_call(
invoke: unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32,
type_id: u32,
method_id: u32,
instance_id: u32,
tlv_args: &[u8],
) -> Option<(u8, usize, Vec<u8>)> {
let mut cap: usize = 256;
let mut tag_ret: u8 = 0;
let mut sz_ret: usize = 0;
let mut payload_ret: Vec<u8> = Vec::new();
loop {
let mut out = vec![0u8; cap];
let mut out_len: usize = out.len();
let rc = unsafe {
invoke(
type_id,
method_id,
instance_id,
tlv_args.as_ptr(),
tlv_args.len(),
out.as_mut_ptr(),
&mut out_len,
)
};
if rc != 0 {
// Retry on short buffer hint (-1) or when plugin wrote beyond capacity (len > cap)
if rc == -1 || out_len > cap {
cap = cap.saturating_mul(2).max(out_len + 16);
if cap > 1 << 20 {
break;
}
continue;
}
return None;
}
let slice = &out[..out_len];
if let Some((t, s, p)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(slice) {
tag_ret = t;
sz_ret = s;
payload_ret = p.to_vec();
}
break;
}
if payload_ret.is_empty() {
return None;
}
Some((tag_ret, sz_ret, payload_ret))
}
/// Decode a single TLV entry to i64 with side-effects (handle registration) when applicable.
pub fn decode_entry_to_i64(
tag: u8,
sz: usize,
payload: &[u8],
fallback_invoke: unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32,
) -> Option<i64> {
match tag {
2 => nyash_rust::runtime::plugin_ffi_common::decode::i32(payload).map(|v| v as i64),
3 => {
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return Some(v as i64);
}
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return Some(i64::from_le_bytes(b));
}
None
}
6 | 7 => {
use nyash_rust::box_trait::{NyashBox, StringBox};
let s = nyash_rust::runtime::plugin_ffi_common::decode::string(payload);
let arc: std::sync::Arc<dyn NyashBox> = std::sync::Arc::new(StringBox::new(s));
let h = nyash_rust::jit::rt::handles::to_handle(arc);
Some(h as i64)
}
8 => {
if sz == 8 {
let mut t = [0u8; 4];
t.copy_from_slice(&payload[0..4]);
let mut i = [0u8; 4];
i.copy_from_slice(&payload[4..8]);
let r_type = u32::from_le_bytes(t);
let r_inst = u32::from_le_bytes(i);
// Use metadata if available to set box_type/invoke_fn
let meta_opt = nyash_rust::runtime::plugin_loader_v2::metadata_for_type_id(r_type);
let (box_type_name, invoke_ptr) = if let Some(meta) = meta_opt {
(meta.box_type.clone(), meta.invoke_fn)
} else {
("PluginBox".to_string(), fallback_invoke)
};
let pb = nyash_rust::runtime::plugin_loader_v2::make_plugin_box_v2(
box_type_name,
r_type,
r_inst,
invoke_ptr,
);
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
std::sync::Arc::new(pb);
let h = nyash_rust::jit::rt::handles::to_handle(arc);
return Some(h as i64);
}
None
}
1 => nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.map(|b| if b { 1 } else { 0 }),
5 => {
if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") && sz == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
let f = f64::from_le_bytes(b);
return Some(f as i64);
}
None
}
_ => None,
}
}
/// Decode a single TLV entry to f64 when possible.
pub fn decode_entry_to_f64(tag: u8, sz: usize, payload: &[u8]) -> Option<f64> {
match tag {
5 => {
if sz == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
Some(f64::from_le_bytes(b))
} else {
None
}
}
3 => {
if let Some(v) = nyash_rust::runtime::plugin_ffi_common::decode::i32(payload) {
return Some(v as f64);
}
if payload.len() == 8 {
let mut b = [0u8; 8];
b.copy_from_slice(payload);
return Some((i64::from_le_bytes(b)) as f64);
}
None
}
1 => nyash_rust::runtime::plugin_ffi_common::decode::bool(payload)
.map(|b| if b { 1.0 } else { 0.0 }),
_ => None,
}
}

View File

@ -4,6 +4,7 @@ pub mod console;
pub mod future; pub mod future;
pub mod instance; pub mod instance;
pub mod invoke; pub mod invoke;
pub mod invoke_core;
pub mod map; pub mod map;
pub mod semantics; pub mod semantics;
pub mod string; pub mod string;
@ -14,6 +15,7 @@ pub use console::*;
pub use future::*; pub use future::*;
pub use instance::*; pub use instance::*;
pub use invoke::*; pub use invoke::*;
pub use invoke_core::*;
pub use map::*; pub use map::*;
pub use semantics::*; pub use semantics::*;
pub use string::*; pub use string::*;

View File

@ -16,6 +16,12 @@ Protocol
- Output: `.o` オブジェクト(既定: `NYASH_AOT_OBJECT_OUT` または `NYASH_LLVM_OBJ_OUT`)。 - Output: `.o` オブジェクト(既定: `NYASH_AOT_OBJECT_OUT` または `NYASH_LLVM_OBJ_OUT`)。
- 入口: `ny_main() -> i64`(戻り値は exit code 相当。必要時 handle 正規化を行う)。 - 入口: `ny_main() -> i64`(戻り値は exit code 相当。必要時 handle 正規化を行う)。
CLIcrate
- `crates/nyash-llvm-compiler` 提供の `ny-llvmc` は llvmlite ハーネスの薄ラッパーだよ。
- ダミー: `./target/release/ny-llvmc --dummy --out /tmp/dummy.o`
- JSON から: `./target/release/ny-llvmc --in mir.json --out out.o`
- 既定のハーネスパスは `tools/llvmlite_harness.py`。変更は `--harness <path>` で上書き可。
Quick Start Quick Start
- 依存: `python3 -m pip install llvmlite` - 依存: `python3 -m pip install llvmlite`
- ダミー生成(配線検証): - ダミー生成(配線検証):
@ -41,5 +47,9 @@ Notes
- 初版は固定 `ny_main` から開始してもよい配線確認。以降、MIR 命令を順次対応。 - 初版は固定 `ny_main` から開始してもよい配線確認。以降、MIR 命令を順次対応。
- ハーネスは自律(外部状態に依存しない)。エラーは即 stderr に詳細を出す。 - ハーネスは自律(外部状態に依存しない)。エラーは即 stderr に詳細を出す。
Schema Validation任意
- JSON v0 のスキーマは `docs/reference/mir/json_v0.schema.json` にあるよ。
- 検証: `python3 tools/validate_mir_json.py <mir.json>`(要: `python3 -m pip install jsonschema`)。
Appendix: 静的リンクについて Appendix: 静的リンクについて
- 生成 EXE は NyRTlibnyrt.aを静的リンク。完全静的-staticは musl 推奨dlopen 不可になるため動的プラグインは使用不可)。 - 生成 EXE は NyRTlibnyrt.aを静的リンク。完全静的-staticは musl 推奨dlopen 不可になるため動的プラグインは使用不可)。

View File

@ -55,6 +55,7 @@
- [言語リファレンス](reference/language/LANGUAGE_REFERENCE_2025.md) - [言語リファレンス](reference/language/LANGUAGE_REFERENCE_2025.md)
- [アーキテクチャ概要](reference/architecture/TECHNICAL_ARCHITECTURE_2025.md) - [アーキテクチャ概要](reference/architecture/TECHNICAL_ARCHITECTURE_2025.md)
- [実行バックエンド](reference/architecture/execution-backends.md) - [実行バックエンド](reference/architecture/execution-backends.md)
- [GC モードと運用](reference/runtime/gc.md)
- [プラグインシステム](reference/plugin-system/) - [プラグインシステム](reference/plugin-system/)
- [CLIオプション早見表](tools/cli-options.md) - [CLIオプション早見表](tools/cli-options.md)

View File

@ -0,0 +1,36 @@
SelfHosting Pilot — Quick Guide (Phase15)
Overview
- Goal: Run Ny→JSON v0 via the selfhost compiler path and execute with PyVM/LLVM.
- Default remains envgated for safety; CI runs smokes to build confidence.
Recommended Flows
- Runner (pilot): `NYASH_USE_NY_COMPILER=1 ./target/release/nyash --backend vm apps/examples/string_p0.nyash`
- Emitonly: `NYASH_USE_NY_COMPILER=1 NYASH_NY_COMPILER_EMIT_ONLY=1 ...`
- EXEfirst (parser EXE): `tools/build_compiler_exe.sh && NYASH_USE_NY_COMPILER=1 NYASH_USE_NY_COMPILER_EXE=1 ./target/release/nyash --backend vm apps/examples/string_p0.nyash`
- LLVM AOT: `NYASH_LLVM_USE_HARNESS=1 tools/build_llvm.sh apps/... -o app && ./app`
CI Workflows
- Selfhost Bootstrap (always): `.github/workflows/selfhost-bootstrap.yml`
- Builds nyash (`cranelift-jit`) and runs `tools/bootstrap_selfhost_smoke.sh`.
- Selfhost EXEfirst (optional): `.github/workflows/selfhost-exe-first.yml`
- Installs LLVM 18 + llvmlite, then runs `tools/exe_first_smoke.sh`.
Useful Env Flags
- `NYASH_USE_NY_COMPILER=1`: Enable selfhost compiler pipeline.
- `NYASH_NY_COMPILER_EMIT_ONLY=1`: Print JSON v0 only (no execution).
- `NYASH_NY_COMPILER_TIMEOUT_MS=4000`: Child timeout (ms). Default 2000.
- `NYASH_USE_NY_COMPILER_EXE=1`: Prefer external parser EXE.
- `NYASH_NY_COMPILER_EXE_PATH=<path>`: Override EXE path.
- `NYASH_SELFHOST_READ_TMP=1`: Child reads `tmp/ny_parser_input.ny` when supported.
Troubleshooting (short)
- No Python found: install `python3` (PyVM / harness).
- No `llvm-config-18`: install LLVM 18 dev (see EXEfirst workflow).
- llvmlite import error: `python3 -m pip install llvmlite`.
- Parser child timeout: raise `NYASH_NY_COMPILER_TIMEOUT_MS`.
- EXEfirst bridge mismatch: rerun with `NYASH_CLI_VERBOSE=1` and keep `dist/nyash_compiler/sample.json` for inspection.
Notes
- JSON v0 schema is stable but not yet versioned; validation is planned.
- Default backend `vm` maps to PyVM unless legacy VM features are enabled.

View File

@ -0,0 +1,96 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://nyash.dev/schema/mir/json_v0.schema.json",
"title": "Nyash MIR JSON v0",
"type": "object",
"additionalProperties": true,
"properties": {
"schema_version": { "type": ["integer", "string" ] },
"functions": {
"oneOf": [
{ "$ref": "#/definitions/functionList" },
{ "$ref": "#/definitions/functionMap" }
]
}
},
"required": ["functions"],
"definitions": {
"functionList": {
"type": "array",
"items": { "$ref": "#/definitions/function" }
},
"functionMap": {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/functionBody" }
},
"function": {
"type": "object",
"additionalProperties": true,
"properties": {
"name": { "type": "string" },
"params": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true,
"properties": {
"name": { "type": "string" },
"type": { "type": "string" }
},
"required": ["name", "type"]
}
},
"return_type": { "type": "string" },
"entry_block": { "type": ["integer", "string"] },
"blocks": { "$ref": "#/definitions/blocks" }
},
"required": ["name", "blocks"]
},
"functionBody": {
"type": "object",
"additionalProperties": true,
"properties": {
"params": { "$ref": "#/definitions/function/properties/params" },
"return_type": { "type": "string" },
"entry_block": { "type": ["integer", "string"] },
"blocks": { "$ref": "#/definitions/blocks" }
},
"required": ["blocks"]
},
"blocks": {
"oneOf": [
{
"type": "array",
"items": { "$ref": "#/definitions/block" }
},
{
"type": "object",
"additionalProperties": { "$ref": "#/definitions/block" }
}
]
},
"block": {
"type": "object",
"additionalProperties": true,
"properties": {
"id": { "type": ["integer", "string"] },
"instructions": { "$ref": "#/definitions/instructions" },
"terminator": { "$ref": "#/definitions/instruction" }
},
"required": ["id", "instructions"]
},
"instructions": {
"type": "array",
"items": { "$ref": "#/definitions/instruction" }
},
"instruction": {
"type": "object",
"additionalProperties": true,
"properties": {
"kind": { "type": "string" }
},
"required": ["kind"]
}
}
}

View File

@ -0,0 +1,62 @@
Nyash GC Modes — Design and Usage
Overview
- Nyash adopts a pragmatic GC strategy that balances safety, performance, and simplicity.
- Default is reference counting with a periodic cycle collector; advanced modes exist for tuning and debugging.
UserFacing Modes (recommended)
- rc+cycle (default, safe)
- Reference counting with periodic cycle detection/collection.
- Recommended for most applications; memory leaks from cycles are handled.
- minorgen (highperformance)
- Lightweight generational GC: moving nursery (Gen0), nonmoving upper generations.
- Write barrier (old→new) is minimal; plugin/FFI objects remain nonmoving via handle indirection.
Advanced Modes (for language dev/debug)
- stw (debug/verification)
- Nonmoving stoptheworld markandsweep. Useful for strict correctness checks and leak cause isolation.
- rc (baseline for comparisons)
- rc+cycle with cycle detection disabled. For performance comparisons or targeted debugging.
- off (expert, selfresponsibility)
- Cycle detection and tracing off. Use only when cycles are guaranteed not to occur. Not recommended for longrunning services.
Selection & Precedence
- CLI: `--gc {auto,rc+cycle,minorgen,stw,rc,off}` (auto = rc+cycle)
- ENV: `NYASH_GC_MODE` (overridden by CLI)
- nyash.toml [env] applies last
Instrumentation & Diagnostics
- `NYASH_GC_METRICS=1`: print brief metrics (allocs/bytes/cycles/pauses)
- `NYASH_GC_METRICS_JSON=1`: emit JSON metrics for CI/aggregation
- `NYASH_GC_LEAK_DIAG=1`: on exit, dump suspected unreleased objects (TopK by type/site)
- `NYASH_GC_ALLOC_THRESHOLD=<N>`: warn or fail when allocations/bytes exceed threshold
Operational Guidance
- Default: rc+cycle for stable operations.
- Try minorgen when throughput/latency matter; it will fall back to rc+cycle on unsupported platforms or when plugin objects are declared nonmoving.
- off/rc are for special cases only; prefer enabling leak diagnostics when using them in development.
Implementation Roadmap (Stepwise)
1) Wiring & Observability
- Introduce `GcMode`, `GcController`, unify roots (handles, globals, frames) and safepoints.
- Add `LeakRegistry` (allocation ledger) and exittime dump.
- Ship rc+cycle (trial deletion) behind the controller (dev default can be rc+cycle).
2) minorgen (nursery)
- Moving Gen0 with simple promotion; upper generations nonmoving marksweep.
- Minimal write barrier (old→new card marking). Plugin/FFI remain nonmoving.
3) stw (dev verify)
- Nonmoving STW markandsweep for correctness checks.
Notes
- Safepoint and barrier MIR ops already exist and are reused as GC coordination hooks.
- Handle indirection keeps future moving GCs compatible with plugin/FFI boundaries.
LLVM Safepoints
- Automatic safepoint insertion can be toggled for the LLVM harness/backend:
- NYASH_LLVM_AUTO_SAFEPOINT=1 enables insertion (default 1)
- Injection points: loop headers, function calls, externcalls, and selected boxcalls.
- Safepoints call ny_check_safepoint/ny_safepoint in NyRT, which forwards to runtime hooks (GC.safepoint + scheduler poll).
Controller & Metrics
- The unified GcController implements GcHooks and aggregates metrics (safepoints/read/write/alloc).
- CountingGc is a thin wrapper around GcController for compatibility.

View File

@ -16,6 +16,19 @@
- `--vm-stats`: VM命令統計を有効化`NYASH_VM_STATS=1` - `--vm-stats`: VM命令統計を有効化`NYASH_VM_STATS=1`
- `--vm-stats-json`: VM統計をJSONで出力`NYASH_VM_STATS_JSON=1` - `--vm-stats-json`: VM統計をJSONで出力`NYASH_VM_STATS_JSON=1`
## GC
- `--gc {auto|rc+cycle|minorgen|stw|rc|off}`: GCモード既定: `auto` → rc+cycle
- `rc+cycle`: 参照カウント + 循環回収(推奨・安定)
- `minorgen`: 高速向けの軽量世代別Gen0移動、上位非移動
- `stw`: 検証用の非移動MarkSweep開発者向け
- `rc`: 循環回収なしのRC比較用
- `off`: 自己責任モード(循環はリーク)
- 関連ENV
- `NYASH_GC_MODE`CLIが優先
- `NYASH_GC_METRICS` / `NYASH_GC_METRICS_JSON`
- `NYASH_GC_LEAK_DIAG` / `NYASH_GC_ALLOC_THRESHOLD`
- 詳細: `docs/reference/runtime/gc.md`
## WASM/AOT ## WASM/AOT
- `--compile-wasm`: WATを出力 - `--compile-wasm`: WATを出力
- `--compile-native` / `--aot`: AOT実行ファイル出力要wasm-backend - `--compile-native` / `--aot`: AOT実行ファイル出力要wasm-backend

View File

@ -58,6 +58,8 @@ pub struct CliConfig {
// Phase-15: JSON IR v0 bridge // Phase-15: JSON IR v0 bridge
pub ny_parser_pipe: bool, pub ny_parser_pipe: bool,
pub json_file: Option<String>, pub json_file: Option<String>,
// GC mode (dev; forwarded to env as NYASH_GC_MODE)
pub gc_mode: Option<String>,
// Build system (MVP) // Build system (MVP)
pub build_path: Option<String>, pub build_path: Option<String>,
pub build_app: Option<String>, pub build_app: Option<String>,
@ -105,6 +107,12 @@ impl CliConfig {
.value_name("FILE") .value_name("FILE")
.index(1) .index(1)
) )
.arg(
Arg::new("gc")
.long("gc")
.value_name("{auto,rc+cycle,minorgen,stw,rc,off}")
.help("Select GC mode (default: rc+cycle)")
)
.arg( .arg(
Arg::new("parser") Arg::new("parser")
.long("parser") .long("parser")
@ -456,6 +464,7 @@ impl CliConfig {
cli_verbose: matches.get_flag("verbose"), cli_verbose: matches.get_flag("verbose"),
run_task: matches.get_one::<String>("run-task").cloned(), run_task: matches.get_one::<String>("run-task").cloned(),
load_ny_plugins: matches.get_flag("load-ny-plugins"), load_ny_plugins: matches.get_flag("load-ny-plugins"),
gc_mode: matches.get_one::<String>("gc").cloned(),
parser_ny: matches parser_ny: matches
.get_one::<String>("parser") .get_one::<String>("parser")
.map(|s| s == "ny") .map(|s| s == "ny")
@ -516,6 +525,7 @@ impl Default for CliConfig {
cli_verbose: false, cli_verbose: false,
run_task: None, run_task: None,
load_ny_plugins: false, load_ny_plugins: false,
gc_mode: None,
parser_ny: false, parser_ny: false,
ny_parser_pipe: false, ny_parser_pipe: false,
json_file: None, json_file: None,

View File

@ -211,6 +211,41 @@ pub fn gc_trace_level() -> u8 {
} }
} }
// ---- GC mode and instrumentation ----
/// Return current GC mode string (auto default = "rc+cycle").
/// Allowed: "auto", "rc+cycle", "minorgen", "stw", "rc", "off"
pub fn gc_mode() -> String {
match std::env::var("NYASH_GC_MODE").ok() {
Some(m) if !m.trim().is_empty() => m,
_ => "rc+cycle".to_string(),
}
}
/// Brief metrics emission (text)
pub fn gc_metrics() -> bool {
std::env::var("NYASH_GC_METRICS").ok().as_deref() == Some("1")
}
/// JSON metrics emission (single line)
pub fn gc_metrics_json() -> bool {
std::env::var("NYASH_GC_METRICS_JSON").ok().as_deref() == Some("1")
}
/// Leak diagnostics on exit
pub fn gc_leak_diag() -> bool {
std::env::var("NYASH_GC_LEAK_DIAG").ok().as_deref() == Some("1")
}
/// Optional allocation threshold; if Some(n) and exceeded, print warning
pub fn gc_alloc_threshold() -> Option<u64> {
std::env::var("NYASH_GC_ALLOC_THRESHOLD").ok()?.parse().ok()
}
/// Run a collection every N safepoints (if Some)
pub fn gc_collect_sp_interval() -> Option<u64> {
std::env::var("NYASH_GC_COLLECT_SP").ok()?.parse().ok()
}
/// Run a collection when allocated bytes since last >= N (if Some)
pub fn gc_collect_alloc_bytes() -> Option<u64> {
std::env::var("NYASH_GC_COLLECT_ALLOC").ok()?.parse().ok()
}
// ---- Rewriter flags (optimizer transforms) // ---- Rewriter flags (optimizer transforms)
pub fn rewrite_debug() -> bool { pub fn rewrite_debug() -> bool {
std::env::var("NYASH_REWRITE_DEBUG").ok().as_deref() == Some("1") std::env::var("NYASH_REWRITE_DEBUG").ok().as_deref() == Some("1")

View File

@ -162,6 +162,28 @@ pub mod handles {
pub fn len() -> usize { pub fn len() -> usize {
REG.with(|cell| cell.borrow().map.len()) REG.with(|cell| cell.borrow().map.len())
} }
/// Tally handles by NyashBox type name (best-effort)
pub fn type_tally() -> Vec<(String, usize)> {
use std::collections::HashMap;
REG.with(|cell| {
let reg = cell.borrow();
let mut map: HashMap<String, usize> = HashMap::new();
for (_h, obj) in reg.map.iter() {
let tn = obj.type_name().to_string();
*map.entry(tn).or_insert(0) += 1;
}
let mut v: Vec<(String, usize)> = map.into_iter().collect();
v.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
v
})
}
/// Snapshot current handle objects (Arc clones)
pub fn snapshot_arcs() -> Vec<Arc<dyn crate::box_trait::NyashBox>> {
REG.with(|cell| {
let reg = cell.borrow();
reg.map.values().cloned().collect()
})
}
// Scope management: track and clear handles created within a JIT call // Scope management: track and clear handles created within a JIT call
pub fn begin_scope() { pub fn begin_scope() {

View File

@ -51,7 +51,8 @@ def lower_atomic_op(
resolver=None, resolver=None,
preds=None, preds=None,
block_end_values=None, block_end_values=None,
bb_map=None bb_map=None,
ctx=None,
) -> None: ) -> None:
""" """
Lower atomic operations Lower atomic operations
@ -66,7 +67,12 @@ def lower_atomic_op(
ordering: Memory ordering ordering: Memory ordering
""" """
# Get pointer # Get pointer
if resolver is not None and preds is not None and block_end_values is not None and bb_map is not None: if ctx is not None:
try:
ptr = ctx.resolver.resolve_ptr(ptr_vid, builder.block, ctx.preds, ctx.block_end_values, ctx.vmap)
except Exception:
ptr = vmap.get(ptr_vid)
elif resolver is not None and preds is not None and block_end_values is not None and bb_map is not None:
ptr = resolver.resolve_ptr(ptr_vid, builder.block, preds, block_end_values, vmap) ptr = resolver.resolve_ptr(ptr_vid, builder.block, preds, block_end_values, vmap)
else: else:
ptr = vmap.get(ptr_vid) ptr = vmap.get(ptr_vid)
@ -85,7 +91,12 @@ def lower_atomic_op(
elif op == "store": elif op == "store":
# Atomic store # Atomic store
if val_vid is not None: if val_vid is not None:
if resolver is not None and preds is not None and block_end_values is not None and bb_map is not None: if ctx is not None:
try:
val = ctx.resolver.resolve_i64(val_vid, builder.block, ctx.preds, ctx.block_end_values, ctx.vmap, ctx.bb_map)
except Exception:
val = vmap.get(val_vid, ir.Constant(ir.IntType(64), 0))
elif resolver is not None and preds is not None and block_end_values is not None and bb_map is not None:
val = resolver.resolve_i64(val_vid, builder.block, preds, block_end_values, vmap, bb_map) val = resolver.resolve_i64(val_vid, builder.block, preds, block_end_values, vmap, bb_map)
else: else:
val = vmap.get(val_vid, ir.Constant(ir.IntType(64), 0)) val = vmap.get(val_vid, ir.Constant(ir.IntType(64), 0))
@ -94,7 +105,12 @@ def lower_atomic_op(
elif op == "add": elif op == "add":
# Atomic add (fetch_add) # Atomic add (fetch_add)
if val_vid is not None: if val_vid is not None:
if resolver is not None and preds is not None and block_end_values is not None and bb_map is not None: if ctx is not None:
try:
val = ctx.resolver.resolve_i64(val_vid, builder.block, ctx.preds, ctx.block_end_values, ctx.vmap, ctx.bb_map)
except Exception:
val = ir.Constant(ir.IntType(64), 1)
elif resolver is not None and preds is not None and block_end_values is not None and bb_map is not None:
val = resolver.resolve_i64(val_vid, builder.block, preds, block_end_values, vmap, bb_map) val = resolver.resolve_i64(val_vid, builder.block, preds, block_end_values, vmap, bb_map)
else: else:
val = ir.Constant(ir.IntType(64), 1) val = ir.Constant(ir.IntType(64), 1)

View File

@ -5,6 +5,7 @@ Core of Nyash's "Everything is Box" philosophy
import llvmlite.ir as ir import llvmlite.ir as ir
from typing import Dict, List, Optional, Any from typing import Dict, List, Optional, Any
from instructions.safepoint import insert_automatic_safepoint
def _declare(module: ir.Module, name: str, ret, args): def _declare(module: ir.Module, name: str, ret, args):
for f in module.functions: for f in module.functions:
@ -68,6 +69,13 @@ def lower_boxcall(
i64 = ir.IntType(64) i64 = ir.IntType(64)
i8 = ir.IntType(8) i8 = ir.IntType(8)
i8p = i8.as_pointer() i8p = i8.as_pointer()
# Insert a safepoint around potential heavy boxcall sites (pre-call)
try:
import os
if os.environ.get('NYASH_LLVM_AUTO_SAFEPOINT', '1') == '1':
insert_automatic_safepoint(builder, module, "boxcall")
except Exception:
pass
# Short-hands with ctx (backward-compatible fallback) # Short-hands with ctx (backward-compatible fallback)
r = resolver r = resolver

View File

@ -6,6 +6,7 @@ Handles regular function calls (not BoxCall or ExternCall)
import llvmlite.ir as ir import llvmlite.ir as ir
from typing import Dict, List, Optional, Any from typing import Dict, List, Optional, Any
from trace import debug as trace_debug from trace import debug as trace_debug
from instructions.safepoint import insert_automatic_safepoint
def lower_call( def lower_call(
builder: ir.IRBuilder, builder: ir.IRBuilder,
@ -45,6 +46,13 @@ def lower_call(
bb_map = ctx.bb_map bb_map = ctx.bb_map
except Exception: except Exception:
pass pass
# Insert an automatic safepoint after the function call
try:
import os
if os.environ.get('NYASH_LLVM_AUTO_SAFEPOINT', '1') == '1':
insert_automatic_safepoint(builder, module, "function_call")
except Exception:
pass
# Short-hands with ctx (backward-compatible fallback) # Short-hands with ctx (backward-compatible fallback)
r = resolver r = resolver
p = preds p = preds

View File

@ -4,6 +4,7 @@ Lowering helpers for while-control flow (regular structured)
from typing import List, Dict, Any from typing import List, Dict, Any
import llvmlite.ir as ir import llvmlite.ir as ir
from instructions.safepoint import insert_automatic_safepoint
def lower_while_regular( def lower_while_regular(
builder: ir.IRBuilder, builder: ir.IRBuilder,
@ -57,6 +58,13 @@ def lower_while_regular(
else: else:
cond_val = ir.Constant(i1, 0) cond_val = ir.Constant(i1, 0)
# Insert a safepoint at loop header to allow cooperative GC
try:
import os
if os.environ.get('NYASH_LLVM_AUTO_SAFEPOINT', '1') == '1':
insert_automatic_safepoint(cbuild, builder.block.parent.module, "loop_header")
except Exception:
pass
cbuild.cbranch(cond_val, body_bb, exit_bb) cbuild.cbranch(cond_val, body_bb, exit_bb)
# Body block # Body block
@ -77,4 +85,3 @@ def lower_while_regular(
# Continue at exit # Continue at exit
builder.position_at_end(exit_bb) builder.position_at_end(exit_bb)

View File

@ -5,6 +5,7 @@ Minimal mapping for NyRT-exported symbols (console/log family等)
import llvmlite.ir as ir import llvmlite.ir as ir
from typing import Dict, List, Optional, Any from typing import Dict, List, Optional, Any
from instructions.safepoint import insert_automatic_safepoint
def lower_externcall( def lower_externcall(
builder: ir.IRBuilder, builder: ir.IRBuilder,
@ -197,3 +198,10 @@ def lower_externcall(
vmap[dst_vid] = ir.Constant(i64, 0) vmap[dst_vid] = ir.Constant(i64, 0)
else: else:
vmap[dst_vid] = result vmap[dst_vid] = result
# Insert an automatic safepoint after externcall
try:
import os
if os.environ.get('NYASH_LLVM_AUTO_SAFEPOINT', '1') == '1':
insert_automatic_safepoint(builder, module, "extern_call")
except Exception:
pass

View File

@ -7,6 +7,7 @@ import os
import llvmlite.ir as ir import llvmlite.ir as ir
from dataclasses import dataclass from dataclasses import dataclass
from typing import Dict, Tuple, List, Optional, Any from typing import Dict, Tuple, List, Optional, Any
from instructions.safepoint import insert_automatic_safepoint
@dataclass @dataclass
class LoopFormContext: class LoopFormContext:
@ -53,7 +54,8 @@ def lower_while_loopform(
bb_map: Dict[int, ir.Block], bb_map: Dict[int, ir.Block],
resolver=None, resolver=None,
preds=None, preds=None,
block_end_values=None block_end_values=None,
ctx=None,
) -> bool: ) -> bool:
""" """
Lower a while loop using LoopForm structure Lower a while loop using LoopForm structure
@ -72,9 +74,22 @@ def lower_while_loopform(
builder.position_at_end(lf.preheader) builder.position_at_end(lf.preheader)
builder.branch(lf.header) builder.branch(lf.header)
# Header: Evaluate condition # Header: Evaluate condition (insert a safepoint at loop header)
builder.position_at_end(lf.header) builder.position_at_end(lf.header)
if resolver is not None and preds is not None and block_end_values is not None: try:
import os
if os.environ.get('NYASH_LLVM_AUTO_SAFEPOINT', '1') == '1':
insert_automatic_safepoint(builder, func.module, "loop_header")
except Exception:
pass
if ctx is not None:
try:
cond64 = ctx.resolver.resolve_i64(condition_vid, builder.block, ctx.preds, ctx.block_end_values, ctx.vmap, ctx.bb_map)
zero64 = ir.IntType(64)(0)
cond = builder.icmp_unsigned('!=', cond64, zero64)
except Exception:
cond = vmap.get(condition_vid, ir.Constant(ir.IntType(1), 0))
elif resolver is not None and preds is not None and block_end_values is not None:
cond64 = resolver.resolve_i64(condition_vid, builder.block, preds, block_end_values, vmap, bb_map) cond64 = resolver.resolve_i64(condition_vid, builder.block, preds, block_end_values, vmap, bb_map)
zero64 = ir.IntType(64)(0) zero64 = ir.IntType(64)(0)
cond = builder.icmp_unsigned('!=', cond64, zero64) cond = builder.icmp_unsigned('!=', cond64, zero64)

View File

@ -950,7 +950,8 @@ class NyashLLVMBuilder:
self.loop_count += 1 self.loop_count += 1
if not lower_while_loopform(builder, func, cond, body, if not lower_while_loopform(builder, func, cond, body,
self.loop_count, self.vmap, self.bb_map, self.loop_count, self.vmap, self.bb_map,
self.resolver, self.preds, self.block_end_values): self.resolver, self.preds, self.block_end_values,
getattr(self, 'ctx', None)):
# Fallback to regular while (structured) # Fallback to regular while (structured)
try: try:
self.resolver._owner_lower_instruction = self.lower_instruction self.resolver._owner_lower_instruction = self.lower_instruction
@ -975,58 +976,10 @@ class NyashLLVMBuilder:
except Exception: except Exception:
pass pass
def _lower_while_regular(self, builder: ir.IRBuilder, inst: Dict[str, Any], func: ir.Function): # NOTE: regular while lowering is implemented in
"""Fallback regular while lowering""" # instructions/controlflow/while_.py::lower_while_regular and invoked
# Create basic blocks: cond -> body -> cond, and exit # from NyashLLVMBuilder.lower_instruction(). This legacy helper is removed
cond_vid = inst.get("cond") # to avoid divergence between two implementations.
body_insts = inst.get("body", [])
cur_bb = builder.block
cond_bb = func.append_basic_block(name=f"while{self.loop_count}_cond")
body_bb = func.append_basic_block(name=f"while{self.loop_count}_body")
exit_bb = func.append_basic_block(name=f"while{self.loop_count}_exit")
# Jump from current to cond
builder.branch(cond_bb)
# Cond block
cbuild = ir.IRBuilder(cond_bb)
try:
# Resolve against the condition block to localize dominance
cond_val = self.resolver.resolve_i64(cond_vid, cbuild.block, self.preds, self.block_end_values, self.vmap, self.bb_map)
except Exception:
cond_val = self.vmap.get(cond_vid)
if cond_val is None:
cond_val = ir.Constant(self.i1, 0)
# Normalize to i1
if hasattr(cond_val, 'type'):
if isinstance(cond_val.type, ir.IntType) and cond_val.type.width == 64:
zero64 = ir.Constant(self.i64, 0)
cond_val = cbuild.icmp_unsigned('!=', cond_val, zero64, name="while_cond_i1")
elif isinstance(cond_val.type, ir.PointerType):
nullp = ir.Constant(cond_val.type, None)
cond_val = cbuild.icmp_unsigned('!=', cond_val, nullp, name="while_cond_p1")
elif isinstance(cond_val.type, ir.IntType) and cond_val.type.width == 1:
# already i1
pass
else:
# Fallback: treat as false
cond_val = ir.Constant(self.i1, 0)
else:
cond_val = ir.Constant(self.i1, 0)
cbuild.cbranch(cond_val, body_bb, exit_bb)
# Body block
bbuild = ir.IRBuilder(body_bb)
# Allow nested lowering of body instructions within this block
self._lower_instruction_list(bbuild, body_insts, func)
# Ensure terminator: if not terminated, branch back to cond
if bbuild.block.terminator is None:
bbuild.branch(cond_bb)
# Continue at exit
builder.position_at_end(exit_bb)
def _lower_instruction_list(self, builder: ir.IRBuilder, insts: List[Dict[str, Any]], func: ir.Function): def _lower_instruction_list(self, builder: ir.IRBuilder, insts: List[Dict[str, Any]], func: ir.Function):
"""Lower a flat list of instructions using current builder and function.""" """Lower a flat list of instructions using current builder and function."""

View File

@ -5,20 +5,17 @@ PHI wiring helpers
- finalize_phis: Wire PHI incomings using end-of-block snapshots and resolver - finalize_phis: Wire PHI incomings using end-of-block snapshots and resolver
These operate on the NyashLLVMBuilder instance to keep changes minimal. These operate on the NyashLLVMBuilder instance to keep changes minimal.
Refactor note: core responsibilities are split into smaller helpers so they
can be unit-tested in isolation.
""" """
from typing import Dict, List, Any from typing import Dict, List, Any, Optional, Tuple
import llvmlite.ir as ir import llvmlite.ir as ir
def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]): # ---- Small helpers (analyzable/testable) ----
"""Predeclare PHIs and collect incoming metadata for finalize_phis.
This pass is function-local and must be invoked after basic blocks are def _collect_produced_stringish(blocks: List[Dict[str, Any]]) -> Dict[int, bool]:
created and before lowering individual blocks. It also tags string-ish
values eagerly to help downstream resolvers choose correct intrinsics.
"""
try:
# Pass A: collect producer stringish hints per value-id
produced_str: Dict[int, bool] = {} produced_str: Dict[int, bool] = {}
for block_data in blocks: for block_data in blocks:
for inst in block_data.get("instructions", []) or []: for inst in block_data.get("instructions", []) or []:
@ -31,9 +28,9 @@ def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]):
if opx == "const": if opx == "const":
v = inst.get("value", {}) or {} v = inst.get("value", {}) or {}
t = v.get("type") t = v.get("type")
if t == "string" or (isinstance(t, dict) and t.get("kind") in ("handle","ptr") and t.get("box_type") == "StringBox"): if t == "string" or (isinstance(t, dict) and t.get("kind") in ("handle", "ptr") and t.get("box_type") == "StringBox"):
is_str = True is_str = True
elif opx in ("binop","boxcall","externcall"): elif opx in ("binop", "boxcall", "externcall"):
t = inst.get("dst_type") t = inst.get("dst_type")
if isinstance(t, dict) and t.get("kind") == "handle" and t.get("box_type") == "StringBox": if isinstance(t, dict) and t.get("kind") == "handle" and t.get("box_type") == "StringBox":
is_str = True is_str = True
@ -41,11 +38,13 @@ def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]):
produced_str[int(dstx)] = True produced_str[int(dstx)] = True
except Exception: except Exception:
pass pass
# Pass B: materialize PHI placeholders and record incoming metadata return produced_str
builder.block_phi_incomings = {}
def analyze_incomings(blocks: List[Dict[str, Any]]) -> Dict[int, Dict[int, List[Tuple[int, int]]]]:
"""Return block_phi_incomings map: block_id -> { dst_vid -> [(decl_b, v_src), ...] }"""
result: Dict[int, Dict[int, List[Tuple[int, int]]]] = {}
for block_data in blocks: for block_data in blocks:
bid0 = block_data.get("id", 0) bid0 = block_data.get("id", 0)
bb0 = builder.bb_map.get(bid0)
for inst in block_data.get("instructions", []) or []: for inst in block_data.get("instructions", []) or []:
if inst.get("op") == "phi": if inst.get("op") == "phi":
try: try:
@ -55,30 +54,138 @@ def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]):
dst0 = None; incoming0 = [] dst0 = None; incoming0 = []
if dst0 is None: if dst0 is None:
continue continue
# Record incoming metadata for finalize_phis
try: try:
builder.block_phi_incomings.setdefault(bid0, {})[dst0] = [ result.setdefault(int(bid0), {})[dst0] = [(int(b), int(v)) for (v, b) in incoming0]
(int(b), int(v)) for (v, b) in incoming0
]
except Exception: except Exception:
pass pass
# Ensure placeholder exists at block head return result
if bb0 is not None:
b0 = ir.IRBuilder(bb0) def ensure_phi(builder, block_id: int, dst_vid: int, bb: ir.Block) -> ir.Instruction:
"""Ensure a PHI placeholder exists at the block head for dst_vid and return it."""
b = ir.IRBuilder(bb)
try: try:
b0.position_at_start(bb0) b.position_at_start(bb)
except Exception: except Exception:
pass pass
existing = builder.vmap.get(dst0) # Prefer predeclared PHIs (e.g., from if-merge prepass)
is_phi = False predecl = getattr(builder, 'predeclared_ret_phis', {}) if hasattr(builder, 'predeclared_ret_phis') else {}
phi = predecl.get((int(block_id), int(dst_vid))) if predecl else None
if phi is not None:
builder.vmap[dst_vid] = phi
return phi
# Reuse current if it is a PHI in the correct block
cur = builder.vmap.get(dst_vid)
try: try:
is_phi = hasattr(existing, 'add_incoming') if cur is not None and hasattr(cur, 'add_incoming') and getattr(getattr(cur, 'basic_block', None), 'name', None) == bb.name:
return cur
except Exception: except Exception:
is_phi = False pass
if not is_phi: # Create a new placeholder
ph0 = b0.phi(builder.i64, name=f"phi_{dst0}") ph = b.phi(builder.i64, name=f"phi_{dst_vid}")
builder.vmap[dst0] = ph0 builder.vmap[dst_vid] = ph
# Tag propagation: if explicit dst_type marks string or any incoming was produced as string-ish, tag dst return ph
def _build_succs(preds: Dict[int, List[int]]) -> Dict[int, List[int]]:
succs: Dict[int, List[int]] = {}
for to_bid, from_list in (preds or {}).items():
for fr in from_list:
succs.setdefault(fr, []).append(to_bid)
return succs
def _nearest_pred_on_path(succs: Dict[int, List[int]], preds_list: List[int], decl_b: int, target_bid: int) -> Optional[int]:
from collections import deque
q = deque([decl_b])
visited = set([decl_b])
parent: Dict[int, Any] = {decl_b: None}
while q:
cur = q.popleft()
if cur == target_bid:
par = parent.get(target_bid)
return par if par in preds_list else None
for nx in succs.get(cur, []):
if nx not in visited:
visited.add(nx)
parent[nx] = cur
q.append(nx)
return None
def wire_incomings(builder, block_id: int, dst_vid: int, incoming: List[Tuple[int, int]]):
"""Wire PHI incoming edges for (block_id, dst_vid) using declared (decl_b, v_src) pairs."""
bb = builder.bb_map.get(block_id)
if bb is None:
return
phi = ensure_phi(builder, block_id, dst_vid, bb)
# Normalize predecessor list
preds_raw = [p for p in builder.preds.get(block_id, []) if p != block_id]
seen = set()
preds_list: List[int] = []
for p in preds_raw:
if p not in seen:
preds_list.append(p)
seen.add(p)
succs = _build_succs(builder.preds)
# Precompute a non-self initial source for self-carry
init_src_vid = None
for (_bd0, vs0) in incoming:
try:
vi = int(vs0)
except Exception:
continue
if vi != int(dst_vid):
init_src_vid = vi
break
chosen: Dict[int, ir.Value] = {}
for (b_decl, v_src) in incoming:
try:
bd = int(b_decl); vs = int(v_src)
except Exception:
continue
pred_match = _nearest_pred_on_path(succs, preds_list, bd, block_id)
if pred_match is None:
continue
if vs == int(dst_vid) and init_src_vid is not None:
vs = int(init_src_vid)
try:
val = builder.resolver._value_at_end_i64(vs, pred_match, builder.preds, builder.block_end_values, builder.vmap, builder.bb_map)
except Exception:
val = None
if val is None:
val = ir.Constant(builder.i64, 0)
chosen[pred_match] = val
for pred_bid, val in chosen.items():
pred_bb = builder.bb_map.get(pred_bid)
if pred_bb is None:
continue
phi.add_incoming(val, pred_bb)
# ---- Public API (used by llvm_builder) ----
def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]):
"""Predeclare PHIs and collect incoming metadata for finalize_phis.
This pass is function-local and must be invoked after basic blocks are
created and before lowering individual blocks. It also tags string-ish
values eagerly to help downstream resolvers choose correct intrinsics.
"""
try:
produced_str = _collect_produced_stringish(blocks)
builder.block_phi_incomings = analyze_incomings(blocks)
# Materialize placeholders and propagate stringish tags
for block_data in blocks:
bid0 = block_data.get("id", 0)
bb0 = builder.bb_map.get(bid0)
for inst in block_data.get("instructions", []) or []:
if inst.get("op") != "phi":
continue
try:
dst0 = int(inst.get("dst"))
incoming0 = inst.get("incoming", []) or []
except Exception:
dst0 = None; incoming0 = []
if dst0 is None or bb0 is None:
continue
_ = ensure_phi(builder, bid0, dst0, bb0)
# Tag propagation
try: try:
dst_type0 = inst.get("dst_type") dst_type0 = inst.get("dst_type")
mark_str = isinstance(dst_type0, dict) and dst_type0.get("kind") == "handle" and dst_type0.get("box_type") == "StringBox" mark_str = isinstance(dst_type0, dict) and dst_type0.get("kind") == "handle" and dst_type0.get("box_type") == "StringBox"
@ -93,108 +200,23 @@ def setup_phi_placeholders(builder, blocks: List[Dict[str, Any]]):
builder.resolver.mark_string(int(dst0)) builder.resolver.mark_string(int(dst0))
except Exception: except Exception:
pass pass
# Definition hint: PHI defines dst in this block
try:
builder.def_blocks.setdefault(int(dst0), set()).add(int(bid0))
except Exception:
pass
# Sync to resolver
try:
builder.resolver.block_phi_incomings = builder.block_phi_incomings
except Exception:
pass
except Exception: except Exception:
pass pass
def finalize_phis(builder): def finalize_phis(builder):
"""Finalize PHIs declared in JSON by wiring incoming edges at block heads. """Finalize PHIs declared in JSON by wiring incoming edges at block heads.
Uses resolver._value_at_end_i64 to materialize values at predecessor ends, Uses resolver._value_at_end_i64 to materialize values at predecessor ends.
ensuring casts/boxing are inserted in predecessor blocks (dominance-safe).""" """
# Build succ map for nearest-predecessor mapping
succs: Dict[int, List[int]] = {}
for to_bid, from_list in (builder.preds or {}).items():
for fr in from_list:
succs.setdefault(fr, []).append(to_bid)
for block_id, dst_map in (getattr(builder, 'block_phi_incomings', {}) or {}).items(): for block_id, dst_map in (getattr(builder, 'block_phi_incomings', {}) or {}).items():
bb = builder.bb_map.get(block_id)
if bb is None:
continue
b = ir.IRBuilder(bb)
try:
b.position_at_start(bb)
except Exception:
pass
for dst_vid, incoming in (dst_map or {}).items(): for dst_vid, incoming in (dst_map or {}).items():
# Ensure placeholder exists at block head wire_incomings(builder, int(block_id), int(dst_vid), incoming)
# Prefer predeclared ret-phi when available and force using it.
predecl = getattr(builder, 'predeclared_ret_phis', {}) if hasattr(builder, 'predeclared_ret_phis') else {}
phi = predecl.get((int(block_id), int(dst_vid))) if predecl else None
if phi is not None:
builder.vmap[dst_vid] = phi
else:
phi = builder.vmap.get(dst_vid)
need_local_phi = False
try:
if not (phi is not None and hasattr(phi, 'add_incoming')):
need_local_phi = True
else:
bb_of_phi = getattr(getattr(phi, 'basic_block', None), 'name', None)
if bb_of_phi != bb.name:
need_local_phi = True
except Exception:
need_local_phi = True
if need_local_phi:
phi = b.phi(builder.i64, name=f"phi_{dst_vid}")
builder.vmap[dst_vid] = phi
# Wire incoming per CFG predecessor; map src_vid when provided
preds_raw = [p for p in builder.preds.get(block_id, []) if p != block_id]
# Deduplicate while preserving order
seen = set()
preds_list: List[int] = []
for p in preds_raw:
if p not in seen:
preds_list.append(p)
seen.add(p)
# Helper: find the nearest immediate predecessor on a path decl_b -> ... -> block_id
def nearest_pred_on_path(decl_b: int):
from collections import deque
q = deque([decl_b])
visited = set([decl_b])
parent: Dict[int, Any] = {decl_b: None}
while q:
cur = q.popleft()
if cur == block_id:
par = parent.get(block_id)
return par if par in preds_list else None
for nx in succs.get(cur, []):
if nx not in visited:
visited.add(nx)
parent[nx] = cur
q.append(nx)
return None
# Precompute a non-self initial source (if present) to use for self-carry cases
init_src_vid = None
for (b_decl0, v_src0) in incoming:
try:
vs0 = int(v_src0)
except Exception:
continue
if vs0 != int(dst_vid):
init_src_vid = vs0
break
# Pre-resolve declared incomings to nearest immediate predecessors
chosen: Dict[int, ir.Value] = {}
for (b_decl, v_src) in incoming:
try:
bd = int(b_decl); vs = int(v_src)
except Exception:
continue
pred_match = nearest_pred_on_path(bd)
if pred_match is None:
continue
if vs == int(dst_vid) and init_src_vid is not None:
vs = int(init_src_vid)
try:
val = builder.resolver._value_at_end_i64(vs, pred_match, builder.preds, builder.block_end_values, builder.vmap, builder.bb_map)
except Exception:
val = None
if val is None:
# As a last resort, zero
val = ir.Constant(builder.i64, 0)
chosen[pred_match] = val
# Finally add incomings
for pred_bid, val in chosen.items():
pred_bb = builder.bb_map.get(pred_bid)
if pred_bb is None:
continue
phi.add_incoming(val, pred_bb)

View File

@ -0,0 +1,65 @@
"""
Unit tests for phi_wiring helpers
These tests construct a minimal function with two blocks and a PHI in the
second block. We verify that placeholders are created and incoming edges
are wired from the correct predecessor, using end-of-block snapshots.
"""
import sys
from pathlib import Path
# Ensure 'src' is importable when running this test directly
TEST_DIR = Path(__file__).resolve().parent
PKG_DIR = TEST_DIR.parent # src/llvm_py
ROOT = PKG_DIR.parent # src
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
if str(PKG_DIR) not in sys.path:
sys.path.insert(0, str(PKG_DIR))
import llvmlite.ir as ir # type: ignore
from phi_wiring import setup_phi_placeholders, finalize_phis # type: ignore
import llvm_builder # type: ignore
def _simple_mir_with_phi():
"""
Build a minimal MIR JSON that compiles to:
bb0: const v1=42; jump bb1
bb1: phi v2=[(bb0,v1)] ; ret v2
"""
return {
"functions": [
{
"name": "main",
"params": [],
"blocks": [
{"id": 0, "instructions": [
{"op": "const", "dst": 1, "value": {"type": "int", "value": 42}},
{"op": "jump", "target": 1}
]},
{"id": 1, "instructions": [
{"op": "phi", "dst": 2, "incoming": [[1, 0]]},
{"op": "ret", "value": 2}
]}
]
}
]
}
def test_phi_placeholders_and_finalize_basic():
mir = _simple_mir_with_phi()
b = llvm_builder.NyashLLVMBuilder()
# Build once to create function, blocks, preds; stop before finalize by calling internals like lower_function
reader_functions = mir["functions"]
assert reader_functions
b.lower_function(reader_functions[0])
# After lowering a function, finalize_phis is already called at the end of lower_function.
# Verify via IR text that a PHI exists in bb1 with an incoming from bb0.
ir_text = str(b.module)
assert 'bb1' in ir_text
assert 'phi i64' in ir_text
assert '[0, %"bb0"]' in ir_text or '[ i64 0, %"bb0"]' in ir_text

View File

@ -105,7 +105,7 @@ impl MirOptimizer {
// Pass 7 (optional): Core-13 pure normalization // Pass 7 (optional): Core-13 pure normalization
if crate::config::env::mir_core13_pure() { if crate::config::env::mir_core13_pure() {
stats.merge(self.normalize_pure_core13(module)); stats.merge(crate::mir::optimizer_passes::normalize_core13_pure::normalize_pure_core13(self, module));
} }
if self.debug { if self.debug {
@ -147,200 +147,7 @@ impl MirOptimizer {
/// Neg x => BinOp(Sub, Const 0, x) /// Neg x => BinOp(Sub, Const 0, x)
/// Not x => Compare(Eq, x, Const false) /// Not x => Compare(Eq, x, Const false)
/// BitNot x => BinOp(BitXor, x, Const(-1)) /// BitNot x => BinOp(BitXor, x, Const(-1))
fn normalize_pure_core13(&mut self, module: &mut MirModule) -> OptimizationStats { // normalize_pure_core13 moved to optimizer_passes::normalize_core13_pure
use super::instruction::ConstValue;
use super::{BinaryOp, CompareOp, MirInstruction as I};
let mut stats = OptimizationStats::new();
for (_fname, function) in &mut module.functions {
for (_bb, block) in &mut function.blocks {
let mut out: Vec<I> = Vec::with_capacity(block.instructions.len() + 8);
let old = std::mem::take(&mut block.instructions);
for inst in old.into_iter() {
match inst {
I::Load { dst, ptr } => {
out.push(I::ExternCall {
dst: Some(dst),
iface_name: "env.local".to_string(),
method_name: "get".to_string(),
args: vec![ptr],
effects: super::EffectMask::READ,
});
stats.intrinsic_optimizations += 1;
}
I::Store { value, ptr } => {
out.push(I::ExternCall {
dst: None,
iface_name: "env.local".to_string(),
method_name: "set".to_string(),
args: vec![ptr, value],
effects: super::EffectMask::WRITE,
});
stats.intrinsic_optimizations += 1;
}
I::NewBox {
dst,
box_type,
mut args,
} => {
// prepend type name as Const String
let ty_id = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const {
dst: ty_id,
value: ConstValue::String(box_type),
});
let mut call_args = Vec::with_capacity(1 + args.len());
call_args.push(ty_id);
call_args.append(&mut args);
out.push(I::ExternCall {
dst: Some(dst),
iface_name: "env.box".to_string(),
method_name: "new".to_string(),
args: call_args,
effects: super::EffectMask::PURE, // constructor is logically alloc; conservatively PURE here
});
stats.intrinsic_optimizations += 1;
}
I::UnaryOp { dst, op, operand } => {
match op {
super::UnaryOp::Neg => {
let zero = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const {
dst: zero,
value: ConstValue::Integer(0),
});
out.push(I::BinOp {
dst,
op: BinaryOp::Sub,
lhs: zero,
rhs: operand,
});
}
super::UnaryOp::Not => {
let f = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const {
dst: f,
value: ConstValue::Bool(false),
});
out.push(I::Compare {
dst,
op: CompareOp::Eq,
lhs: operand,
rhs: f,
});
}
super::UnaryOp::BitNot => {
let all1 = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const {
dst: all1,
value: ConstValue::Integer(-1),
});
out.push(I::BinOp {
dst,
op: BinaryOp::BitXor,
lhs: operand,
rhs: all1,
});
}
}
stats.intrinsic_optimizations += 1;
}
other => out.push(other),
}
}
block.instructions = out;
if let Some(term) = block.terminator.take() {
block.terminator = Some(match term {
I::Load { dst, ptr } => I::ExternCall {
dst: Some(dst),
iface_name: "env.local".to_string(),
method_name: "get".to_string(),
args: vec![ptr],
effects: super::EffectMask::READ,
},
I::Store { value, ptr } => I::ExternCall {
dst: None,
iface_name: "env.local".to_string(),
method_name: "set".to_string(),
args: vec![ptr, value],
effects: super::EffectMask::WRITE,
},
I::NewBox {
dst,
box_type,
mut args,
} => {
let ty_id = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const {
dst: ty_id,
value: ConstValue::String(box_type),
});
let mut call_args = Vec::with_capacity(1 + args.len());
call_args.push(ty_id);
call_args.append(&mut args);
I::ExternCall {
dst: Some(dst),
iface_name: "env.box".to_string(),
method_name: "new".to_string(),
args: call_args,
effects: super::EffectMask::PURE,
}
}
I::UnaryOp { dst, op, operand } => match op {
super::UnaryOp::Neg => {
let zero = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const {
dst: zero,
value: ConstValue::Integer(0),
});
I::BinOp {
dst,
op: BinaryOp::Sub,
lhs: zero,
rhs: operand,
}
}
super::UnaryOp::Not => {
let f = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const {
dst: f,
value: ConstValue::Bool(false),
});
I::Compare {
dst,
op: CompareOp::Eq,
lhs: operand,
rhs: f,
}
}
super::UnaryOp::BitNot => {
let all1 = super::ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const {
dst: all1,
value: ConstValue::Integer(-1),
});
I::BinOp {
dst,
op: BinaryOp::BitXor,
lhs: operand,
rhs: all1,
}
}
},
other => other,
});
}
}
}
stats
}
/// Eliminate dead code in a single function /// Eliminate dead code in a single function
fn eliminate_dead_code_in_function(&mut self, function: &mut MirFunction) -> usize { fn eliminate_dead_code_in_function(&mut self, function: &mut MirFunction) -> usize {

View File

@ -3,3 +3,5 @@ pub mod diagnostics;
pub mod intrinsics; pub mod intrinsics;
pub mod normalize; pub mod normalize;
pub mod reorder; pub mod reorder;
pub mod normalize_core13_pure;
pub mod normalize_legacy_all;

View File

@ -0,0 +1,145 @@
use crate::mir::optimizer::MirOptimizer;
use crate::mir::optimizer_stats::OptimizationStats;
use crate::mir::{BinaryOp, CompareOp, EffectMask, MirInstruction as I, MirModule, MirType, ValueId};
/// Core-13 "pure" normalization: rewrite a few non-13 ops to allowed forms.
/// - Load(dst, ptr) => ExternCall(Some dst, env.local.get, [ptr])
/// - Store(val, ptr) => ExternCall(None, env.local.set, [ptr, val])
/// - NewBox(dst, T, args...) => ExternCall(Some dst, env.box.new, [Const String(T), args...])
/// - UnaryOp:
/// Neg x => BinOp(Sub, Const 0, x)
/// Not x => Compare(Eq, x, Const false)
/// BitNot x => BinOp(BitXor, x, Const(-1))
pub fn normalize_pure_core13(_opt: &mut MirOptimizer, module: &mut MirModule) -> OptimizationStats {
use crate::mir::instruction::ConstValue;
let mut stats = OptimizationStats::new();
for (_fname, function) in &mut module.functions {
for (_bb, block) in &mut function.blocks {
let mut out: Vec<I> = Vec::with_capacity(block.instructions.len() + 8);
let old = std::mem::take(&mut block.instructions);
for inst in old.into_iter() {
match inst {
I::Load { dst, ptr } => {
out.push(I::ExternCall {
dst: Some(dst),
iface_name: "env.local".to_string(),
method_name: "get".to_string(),
args: vec![ptr],
effects: EffectMask::READ,
});
stats.intrinsic_optimizations += 1;
}
I::Store { value, ptr } => {
out.push(I::ExternCall {
dst: None,
iface_name: "env.local".to_string(),
method_name: "set".to_string(),
args: vec![ptr, value],
effects: EffectMask::WRITE,
});
stats.intrinsic_optimizations += 1;
}
I::NewBox { dst, box_type, mut args } => {
// prepend type name as Const String
let ty_id = ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const { dst: ty_id, value: ConstValue::String(box_type) });
let mut call_args = Vec::with_capacity(1 + args.len());
call_args.push(ty_id);
call_args.append(&mut args);
out.push(I::ExternCall {
dst: Some(dst),
iface_name: "env.box".to_string(),
method_name: "new".to_string(),
args: call_args,
effects: EffectMask::PURE, // constructor is logically alloc; conservatively PURE here
});
stats.intrinsic_optimizations += 1;
}
I::UnaryOp { dst, op, operand } => {
match op {
crate::mir::UnaryOp::Neg => {
let zero = ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const { dst: zero, value: ConstValue::Integer(0) });
out.push(I::BinOp { dst, op: BinaryOp::Sub, lhs: zero, rhs: operand });
}
crate::mir::UnaryOp::Not => {
let f = ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const { dst: f, value: ConstValue::Bool(false) });
out.push(I::Compare { dst, op: CompareOp::Eq, lhs: operand, rhs: f });
}
crate::mir::UnaryOp::BitNot => {
let all1 = ValueId::new(function.next_value_id);
function.next_value_id += 1;
out.push(I::Const { dst: all1, value: ConstValue::Integer(-1) });
out.push(I::BinOp { dst, op: BinaryOp::BitXor, lhs: operand, rhs: all1 });
}
}
stats.intrinsic_optimizations += 1;
}
other => out.push(other),
}
}
block.instructions = out;
if let Some(term) = block.terminator.take() {
block.terminator = Some(match term {
I::Load { dst, ptr } => I::ExternCall {
dst: Some(dst),
iface_name: "env.local".to_string(),
method_name: "get".to_string(),
args: vec![ptr],
effects: EffectMask::READ,
},
I::Store { value, ptr } => I::ExternCall {
dst: None,
iface_name: "env.local".to_string(),
method_name: "set".to_string(),
args: vec![ptr, value],
effects: EffectMask::WRITE,
},
I::NewBox { dst, box_type, mut args } => {
let ty_id = ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const { dst: ty_id, value: ConstValue::String(box_type) });
let mut call_args = Vec::with_capacity(1 + args.len());
call_args.push(ty_id);
call_args.append(&mut args);
I::ExternCall {
dst: Some(dst),
iface_name: "env.box".to_string(),
method_name: "new".to_string(),
args: call_args,
effects: EffectMask::PURE,
}
}
I::UnaryOp { dst, op, operand } => match op {
crate::mir::UnaryOp::Neg => {
let zero = ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const { dst: zero, value: ConstValue::Integer(0) });
I::BinOp { dst, op: BinaryOp::Sub, lhs: zero, rhs: operand }
}
crate::mir::UnaryOp::Not => {
let f = ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const { dst: f, value: ConstValue::Bool(false) });
I::Compare { dst, op: CompareOp::Eq, lhs: operand, rhs: f }
}
crate::mir::UnaryOp::BitNot => {
let all1 = ValueId::new(function.next_value_id);
function.next_value_id += 1;
block.instructions.push(I::Const { dst: all1, value: ConstValue::Integer(-1) });
I::BinOp { dst, op: BinaryOp::BitXor, lhs: operand, rhs: all1 }
}
},
other => other,
});
}
}
}
stats
}

View File

@ -0,0 +1,8 @@
use crate::mir::optimizer::MirOptimizer;
use crate::mir::optimizer_stats::OptimizationStats;
/// Delegate: legacy normalization (moved from optimizer.rs)
pub fn normalize_legacy_instructions(opt: &mut MirOptimizer, module: &mut crate::mir::MirModule) -> OptimizationStats {
crate::mir::optimizer_passes::normalize::normalize_legacy_instructions(opt, module)
}

View File

@ -143,6 +143,12 @@ impl NyashRunner {
if self.config.cli_verbose { if self.config.cli_verbose {
std::env::set_var("NYASH_CLI_VERBOSE", "1"); std::env::set_var("NYASH_CLI_VERBOSE", "1");
} }
// GC mode forwarding: map CLI --gc to NYASH_GC_MODE for downstream runtimes
if let Some(ref m) = self.config.gc_mode {
if !m.trim().is_empty() {
std::env::set_var("NYASH_GC_MODE", m);
}
}
// Script-level env directives (special comments) — parse early // Script-level env directives (special comments) — parse early
// Supported: // Supported:
// // @env KEY=VALUE // // @env KEY=VALUE

View File

@ -11,8 +11,6 @@ use std::thread::sleep;
use crate::runner::pipeline::{suggest_in_base, resolve_using_target}; use crate::runner::pipeline::{suggest_in_base, resolve_using_target};
use crate::runner::trace::cli_verbose; use crate::runner::trace::cli_verbose;
use crate::cli_v; use crate::cli_v;
use crate::runner::trace::cli_verbose;
use crate::cli_v;
// (moved) suggest_in_base is now in runner/pipeline.rs // (moved) suggest_in_base is now in runner/pipeline.rs
@ -131,97 +129,13 @@ impl NyashRunner {
/// Helper: run PyVM harness over a MIR module, returning the exit code /// Helper: run PyVM harness over a MIR module, returning the exit code
fn run_pyvm_harness(&self, module: &nyash_rust::mir::MirModule, tag: &str) -> Result<i32, String> { fn run_pyvm_harness(&self, module: &nyash_rust::mir::MirModule, tag: &str) -> Result<i32, String> {
let py3 = which::which("python3").map_err(|e| format!("python3 not found: {}", e))?; super::common_util::pyvm::run_pyvm_harness(module, tag)
let runner = std::path::Path::new("tools/pyvm_runner.py");
if !runner.exists() { return Err(format!("PyVM runner not found: {}", runner.display())); }
let tmp_dir = std::path::Path::new("tmp");
let _ = std::fs::create_dir_all(tmp_dir);
let mir_json_path = tmp_dir.join("nyash_pyvm_mir.json");
crate::runner::mir_json_emit::emit_mir_json_for_harness_bin(module, &mir_json_path)
.map_err(|e| format!("PyVM MIR JSON emit error: {}", e))?;
cli_v!("[ny-compiler] using PyVM ({} ) → {}", tag, mir_json_path.display());
// Determine entry function hint (prefer Main.main if present)
let entry = if module.functions.contains_key("Main.main") { "Main.main" }
else if module.functions.contains_key("main") { "main" } else { "Main.main" };
let status = std::process::Command::new(py3)
.args([
runner.to_string_lossy().as_ref(),
"--in",
&mir_json_path.display().to_string(),
"--entry",
entry,
])
.status()
.map_err(|e| format!("spawn pyvm: {}", e))?;
let code = status.code().unwrap_or(1);
if !status.success() { cli_v!("❌ PyVM ({}) failed (status={})", tag, code); }
Ok(code)
} }
/// Helper: try external selfhost compiler EXE to parse Ny -> JSON v0 and return MIR module /// Helper: try external selfhost compiler EXE to parse Ny -> JSON v0 and return MIR module
/// Returns Some(module) on success, None on failure (timeout/invalid output/missing exe) /// Returns Some(module) on success, None on failure (timeout/invalid output/missing exe)
fn exe_try_parse_json_v0(&self, filename: &str, timeout_ms: u64) -> Option<nyash_rust::mir::MirModule> { fn exe_try_parse_json_v0(&self, filename: &str, timeout_ms: u64) -> Option<nyash_rust::mir::MirModule> {
// Resolve parser EXE path super::common_util::selfhost_exe::exe_try_parse_json_v0(filename, timeout_ms)
let exe_path = if let Ok(p) = std::env::var("NYASH_NY_COMPILER_EXE_PATH") {
std::path::PathBuf::from(p)
} else {
let mut p = std::path::PathBuf::from("dist/nyash_compiler");
#[cfg(windows)]
{ p.push("nyash_compiler.exe"); }
#[cfg(not(windows))]
{ p.push("nyash_compiler"); }
if !p.exists() {
if let Ok(w) = which::which("nyash_compiler") { w } else { p }
} else { p }
};
if !exe_path.exists() { cli_v!("[ny-compiler] exe not found at {}", exe_path.display()); return None; }
// Build command
let mut cmd = std::process::Command::new(&exe_path);
cmd.arg(filename);
if crate::config::env::ny_compiler_min_json() { cmd.arg("--min-json"); }
if crate::config::env::selfhost_read_tmp() { cmd.arg("--read-tmp"); }
if let Some(raw) = crate::config::env::ny_compiler_child_args() { for tok in raw.split_whitespace() { cmd.arg(tok); } }
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
let mut child = match cmd.spawn() { Ok(c) => c, Err(e) => { eprintln!("[ny-compiler] exe spawn failed: {}", e); return None; } };
let mut ch_stdout = child.stdout.take();
let mut ch_stderr = child.stderr.take();
let start = Instant::now();
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_)) => break,
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) { let _ = child.kill(); let _ = child.wait(); timed_out = true; break; }
sleep(Duration::from_millis(10));
}
Err(e) => { eprintln!("[ny-compiler] exe wait error: {}", e); return None; }
}
}
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout { let _ = s.read_to_end(&mut out_buf); }
if let Some(mut s) = ch_stderr { let _ = s.read_to_end(&mut err_buf); }
if timed_out {
let head = String::from_utf8_lossy(&out_buf).chars().take(200).collect::<String>();
eprintln!("[ny-compiler] exe timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n"));
return None;
}
let stdout = match String::from_utf8(out_buf) { Ok(s) => s, Err(_) => String::new() };
let mut json_line = String::new();
for line in stdout.lines() { let t = line.trim(); if t.starts_with('{') && t.contains("\"version\"") && t.contains("\"kind\"") { json_line = t.to_string(); break; } }
if json_line.is_empty() {
if cli_verbose() {
let head: String = stdout.chars().take(200).collect();
let errh: String = String::from_utf8_lossy(&err_buf).chars().take(200).collect();
cli_v!("[ny-compiler] exe produced no JSON; stdout(head)='{}' stderr(head)='{}'", head.replace('\n', "\\n"), errh.replace('\n', "\\n"));
}
return None;
}
match json_v0_bridge::parse_json_v0_to_module(&json_line) {
Ok(module) => Some(module),
Err(e) => { eprintln!("[ny-compiler] JSON parse failed (exe): {}", e); None }
}
} }
/// Phase-15.3: Attempt Ny compiler pipeline (Ny -> JSON v0 via Ny program), then execute MIR /// Phase-15.3: Attempt Ny compiler pipeline (Ny -> JSON v0 via Ny program), then execute MIR
@ -324,38 +238,22 @@ impl NyashRunner {
if crate::config::env::selfhost_read_tmp() { cmd.arg("--read-tmp"); } if crate::config::env::selfhost_read_tmp() { cmd.arg("--read-tmp"); }
if let Some(raw) = crate::config::env::ny_compiler_child_args() { for tok in raw.split_whitespace() { cmd.arg(tok); } } if let Some(raw) = crate::config::env::ny_compiler_child_args() { for tok in raw.split_whitespace() { cmd.arg(tok); } }
let timeout_ms: u64 = crate::config::env::ny_compiler_timeout_ms(); let timeout_ms: u64 = crate::config::env::ny_compiler_timeout_ms();
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped()); let out = match super::common_util::io::spawn_with_timeout(cmd, timeout_ms) {
let mut child = match cmd.spawn() { Ok(c) => c, Err(e) => { eprintln!("[ny-compiler] exe spawn failed: {}", e); return false; } }; Ok(o) => o,
let mut ch_stdout = child.stdout.take(); Err(e) => { eprintln!("[ny-compiler] exe spawn failed: {}", e); return false; }
let mut ch_stderr = child.stderr.take(); };
let start = Instant::now(); if out.timed_out {
let mut timed_out = false; let head = String::from_utf8_lossy(&out.stdout).chars().take(200).collect::<String>();
loop {
match child.try_wait() {
Ok(Some(_status)) => { break; }
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) { let _ = child.kill(); let _ = child.wait(); timed_out = true; break; }
sleep(Duration::from_millis(10));
}
Err(e) => { eprintln!("[ny-compiler] exe wait error: {}", e); return false; }
}
}
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout { let _ = s.read_to_end(&mut out_buf); }
if let Some(mut s) = ch_stderr { let _ = s.read_to_end(&mut err_buf); }
if timed_out {
let head = String::from_utf8_lossy(&out_buf).chars().take(200).collect::<String>();
eprintln!("[ny-compiler] exe timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n")); eprintln!("[ny-compiler] exe timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n"));
return false; return false;
} }
let stdout = match String::from_utf8(out_buf) { Ok(s) => s, Err(_) => String::new() }; let stdout = match String::from_utf8(out.stdout) { Ok(s) => s, Err(_) => String::new() };
let mut json_line = String::new(); let mut json_line = String::new();
for line in stdout.lines() { let t = line.trim(); if t.starts_with('{') && t.contains("\"version\"") && t.contains("\"kind\"") { json_line = t.to_string(); break; } } for line in stdout.lines() { let t = line.trim(); if t.starts_with('{') && t.contains("\"version\"") && t.contains("\"kind\"") { json_line = t.to_string(); break; } }
if json_line.is_empty() { if json_line.is_empty() {
if crate::config::env::cli_verbose() { if crate::config::env::cli_verbose() {
let head: String = stdout.chars().take(200).collect(); let head: String = stdout.chars().take(200).collect();
let errh: String = String::from_utf8_lossy(&err_buf).chars().take(200).collect(); let errh: String = String::from_utf8_lossy(&out.stderr).chars().take(200).collect();
eprintln!("[ny-compiler] exe produced no JSON; stdout(head)='{}' stderr(head)='{}'", head.replace('\n', "\\n"), errh.replace('\n', "\\n")); eprintln!("[ny-compiler] exe produced no JSON; stdout(head)='{}' stderr(head)='{}'", head.replace('\n', "\\n"), errh.replace('\n', "\\n"));
} }
return false; return false;
@ -450,40 +348,15 @@ impl NyashRunner {
.ok() .ok()
.and_then(|s| s.parse::<u64>().ok()) .and_then(|s| s.parse::<u64>().ok())
.unwrap_or(2000); .unwrap_or(2000);
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped()); let out = match super::common_util::io::spawn_with_timeout(cmd, timeout_ms) {
let mut child = match cmd.spawn() { Ok(o) => o,
Ok(c) => c,
Err(e) => { eprintln!("[ny-compiler] spawn failed: {}", e); return false; } Err(e) => { eprintln!("[ny-compiler] spawn failed: {}", e); return false; }
}; };
let mut ch_stdout = child.stdout.take(); if out.timed_out {
let mut ch_stderr = child.stderr.take(); let head = String::from_utf8_lossy(&out.stdout).chars().take(200).collect::<String>();
let start = Instant::now();
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_status)) => { break; }
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) {
let _ = child.kill();
let _ = child.wait();
timed_out = true;
break;
}
sleep(Duration::from_millis(10));
}
Err(e) => { eprintln!("[ny-compiler] wait error: {}", e); return false; }
}
}
// Collect any available output
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout { let _ = s.read_to_end(&mut out_buf); }
if let Some(mut s) = ch_stderr { let _ = s.read_to_end(&mut err_buf); }
if timed_out {
let head = String::from_utf8_lossy(&out_buf).chars().take(200).collect::<String>();
eprintln!("[ny-compiler] child timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n")); eprintln!("[ny-compiler] child timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n"));
} }
let stdout = match String::from_utf8(out_buf) { Ok(s) => s, Err(_) => String::new() }; let stdout = match String::from_utf8(out.stdout.clone()) { Ok(s) => s, Err(_) => String::new() };
if timed_out { if timed_out {
// Fall back path will be taken below when json_line remains empty // Fall back path will be taken below when json_line remains empty
} else if let Ok(s) = String::from_utf8(err_buf.clone()) { } else if let Ok(s) = String::from_utf8(err_buf.clone()) {

View File

@ -0,0 +1,55 @@
use std::io::Read;
use std::process::{Command, Stdio};
use std::thread::sleep;
use std::time::{Duration, Instant};
pub struct ChildOutput {
pub stdout: Vec<u8>,
pub stderr: Vec<u8>,
pub status_ok: bool,
pub exit_code: Option<i32>,
pub timed_out: bool,
}
/// Spawn command with timeout (ms), capture stdout/stderr, and return ChildOutput.
pub fn spawn_with_timeout(mut cmd: Command, timeout_ms: u64) -> std::io::Result<ChildOutput> {
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
let mut child = cmd.spawn()?;
let mut ch_stdout = child.stdout.take();
let mut ch_stderr = child.stderr.take();
let start = Instant::now();
let mut timed_out = false;
let mut exit_status: Option<std::process::ExitStatus> = None;
loop {
match child.try_wait()? {
Some(status) => { exit_status = Some(status); break },
None => {
if start.elapsed() >= Duration::from_millis(timeout_ms) {
let _ = child.kill();
let _ = child.wait();
timed_out = true;
break;
}
sleep(Duration::from_millis(10));
}
}
}
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout {
let _ = s.read_to_end(&mut out_buf);
}
if let Some(mut s) = ch_stderr {
let _ = s.read_to_end(&mut err_buf);
}
let (status_ok, exit_code) = if let Some(st) = exit_status {
(st.success(), st.code())
} else { (false, None) };
Ok(ChildOutput {
stdout: out_buf,
stderr: err_buf,
status_ok,
exit_code,
timed_out,
})
}

View File

@ -0,0 +1,9 @@
/*!
* Shared helpers for runner/modes/common.rs
*
* Minimal extraction to reduce duplication and prepare for full split.
*/
pub mod pyvm;
pub mod selfhost_exe;
pub mod io;

View File

@ -0,0 +1,39 @@
use std::process::Stdio;
/// Run PyVM harness over a MIR module, returning the exit code
pub fn run_pyvm_harness(module: &crate::mir::MirModule, tag: &str) -> Result<i32, String> {
let py3 = which::which("python3").map_err(|e| format!("python3 not found: {}", e))?;
let runner = std::path::Path::new("tools/pyvm_runner.py");
if !runner.exists() {
return Err(format!("PyVM runner not found: {}", runner.display()));
}
let tmp_dir = std::path::Path::new("tmp");
let _ = std::fs::create_dir_all(tmp_dir);
let mir_json_path = tmp_dir.join("nyash_pyvm_mir.json");
crate::runner::mir_json_emit::emit_mir_json_for_harness_bin(module, &mir_json_path)
.map_err(|e| format!("PyVM MIR JSON emit error: {}", e))?;
crate::cli_v!("[ny-compiler] using PyVM ({} ) → {}", tag, mir_json_path.display());
// Determine entry function hint (prefer Main.main if present)
let entry = if module.functions.contains_key("Main.main") {
"Main.main"
} else if module.functions.contains_key("main") {
"main"
} else {
"Main.main"
};
let status = std::process::Command::new(py3)
.args([
runner.to_string_lossy().as_ref(),
"--in",
&mir_json_path.display().to_string(),
"--entry",
entry,
])
.status()
.map_err(|e| format!("spawn pyvm: {}", e))?;
let code = status.code().unwrap_or(1);
if !status.success() {
crate::cli_v!("❌ PyVM ({}) failed (status={})", tag, code);
}
Ok(code)
}

View File

@ -0,0 +1,131 @@
use std::io::Read;
use std::process::Stdio;
use std::thread::sleep;
use std::time::{Duration, Instant};
/// Try external selfhost compiler EXE to parse Ny -> JSON v0 and return MIR module.
/// Returns Some(module) on success, None on failure (timeout/invalid output/missing exe)
pub fn exe_try_parse_json_v0(filename: &str, timeout_ms: u64) -> Option<crate::mir::MirModule> {
// Resolve parser EXE path
let exe_path = if let Ok(p) = std::env::var("NYASH_NY_COMPILER_EXE_PATH") {
std::path::PathBuf::from(p)
} else {
let mut p = std::path::PathBuf::from("dist/nyash_compiler");
#[cfg(windows)]
{
p.push("nyash_compiler.exe");
}
#[cfg(not(windows))]
{
p.push("nyash_compiler");
}
if !p.exists() {
if let Ok(w) = which::which("nyash_compiler") {
w
} else {
p
}
} else {
p
}
};
if !exe_path.exists() {
crate::cli_v!("[ny-compiler] exe not found at {}", exe_path.display());
return None;
}
// Build command
let mut cmd = std::process::Command::new(&exe_path);
cmd.arg(filename);
if crate::config::env::ny_compiler_min_json() {
cmd.arg("--min-json");
}
if crate::config::env::selfhost_read_tmp() {
cmd.arg("--read-tmp");
}
if let Some(raw) = crate::config::env::ny_compiler_child_args() {
for tok in raw.split_whitespace() {
cmd.arg(tok);
}
}
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
let mut child = match cmd.spawn() {
Ok(c) => c,
Err(e) => {
eprintln!("[ny-compiler] exe spawn failed: {}", e);
return None;
}
};
let mut ch_stdout = child.stdout.take();
let mut ch_stderr = child.stderr.take();
let start = Instant::now();
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_)) => break,
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) {
let _ = child.kill();
let _ = child.wait();
timed_out = true;
break;
}
sleep(Duration::from_millis(10));
}
Err(e) => {
eprintln!("[ny-compiler] exe wait error: {}", e);
return None;
}
}
}
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout {
let _ = s.read_to_end(&mut out_buf);
}
if let Some(mut s) = ch_stderr {
let _ = s.read_to_end(&mut err_buf);
}
if timed_out {
let head = String::from_utf8_lossy(&out_buf)
.chars()
.take(200)
.collect::<String>();
eprintln!(
"[ny-compiler] exe timeout after {} ms; stdout(head)='{}'",
timeout_ms,
head.replace('\n', "\\n")
);
return None;
}
let stdout = match String::from_utf8(out_buf) {
Ok(s) => s,
Err(_) => String::new(),
};
let mut json_line = String::new();
for line in stdout.lines() {
let t = line.trim();
if t.starts_with('{') && t.contains("\"version\"") && t.contains("\"kind\"") {
json_line = t.to_string();
break;
}
}
if json_line.is_empty() {
if crate::config::env::cli_verbose() {
let head: String = stdout.chars().take(200).collect();
let errh: String = String::from_utf8_lossy(&err_buf).chars().take(200).collect();
crate::cli_v!(
"[ny-compiler] exe produced no JSON; stdout(head)='{}' stderr(head)='{}'",
head.replace('\n', "\\n"),
errh.replace('\n', "\\n")
);
}
return None;
}
match crate::runner::json_v0_bridge::parse_json_v0_to_module(&json_line) {
Ok(module) => Some(module),
Err(e) => {
eprintln!("[ny-compiler] JSON parse failed: {}", e);
None
}
}
}

View File

@ -84,21 +84,22 @@ impl NyashRunner {
); );
} }
// 2) Run harness with --in/--out失敗時は即エラー // 2) Run harness with --in/--out失敗時は即エラー
let status = std::process::Command::new(py3) let mut cmd = std::process::Command::new(py3);
.args([ cmd.args([
harness.to_string_lossy().as_ref(), harness.to_string_lossy().as_ref(),
"--in", "--in",
&mir_json_path.display().to_string(), &mir_json_path.display().to_string(),
"--out", "--out",
&_out_path, &_out_path,
]) ]);
.status() let out = crate::runner::modes::common_util::io::spawn_with_timeout(cmd, 20_000)
.map_err(|e| format!("spawn harness: {}", e)) .map_err(|e| format!("spawn harness: {}", e))
.unwrap(); .unwrap();
if !status.success() { if out.timed_out || !out.status_ok {
eprintln!( eprintln!(
"❌ llvmlite harness failed (status={})", "❌ llvmlite harness failed (timeout={} code={:?})",
status.code().unwrap_or(-1) out.timed_out,
out.exit_code
); );
process::exit(1); process::exit(1);
} }

View File

@ -8,5 +8,8 @@ pub mod mir;
pub mod vm; pub mod vm;
pub mod pyvm; pub mod pyvm;
// Shared helpers extracted from common.rs (in progress)
pub mod common_util;
#[cfg(feature = "cranelift-jit")] #[cfg(feature = "cranelift-jit")]
pub mod aot; pub mod aot;

View File

@ -76,22 +76,20 @@ pub fn execute_pyvm_only(_runner: &NyashRunner, filename: &str) {
mir_json_path.display() mir_json_path.display()
); );
} }
let status = std::process::Command::new(py3) let mut cmd = std::process::Command::new(py3);
.args([ cmd.args([
runner.to_string_lossy().as_ref(), runner.to_string_lossy().as_ref(),
"--in", "--in",
&mir_json_path.display().to_string(), &mir_json_path.display().to_string(),
"--entry", "--entry",
entry, entry,
]) ]);
.status() let out = crate::runner::modes::common_util::io::spawn_with_timeout(cmd, 10_000)
.map_err(|e| format!("spawn pyvm: {}", e)) .map_err(|e| format!("spawn pyvm: {}", e))
.unwrap(); .unwrap();
let code = status.code().unwrap_or(1); let code = if out.timed_out { 1 } else { out.exit_code.unwrap_or(1) };
if !status.success() { if out.timed_out && std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") {
if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") { eprintln!("❌ PyVM timeout");
eprintln!("❌ PyVM failed (status={})", code);
}
} }
process::exit(code); process::exit(code);
} else { } else {
@ -99,4 +97,3 @@ pub fn execute_pyvm_only(_runner: &NyashRunner, filename: &str) {
process::exit(1); process::exit(1);
} }
} }

View File

@ -207,23 +207,20 @@ impl NyashRunner {
"Main.main" "Main.main"
}; };
// Spawn runner // Spawn runner
let status = std::process::Command::new(py3) let mut cmd = std::process::Command::new(py3);
.args([ cmd.args([
runner.to_string_lossy().as_ref(), runner.to_string_lossy().as_ref(),
"--in", "--in",
&mir_json_path.display().to_string(), &mir_json_path.display().to_string(),
"--entry", "--entry",
entry, entry,
]) ]);
.status() let out = super::common_util::io::spawn_with_timeout(cmd, 10_000)
.map_err(|e| format!("spawn pyvm: {}", e)) .map_err(|e| format!("spawn pyvm: {}", e))
.unwrap(); .unwrap();
// Always propagate PyVM exit code to match llvmlite semantics let code = if out.timed_out { 1 } else { out.exit_code.unwrap_or(1) };
let code = status.code().unwrap_or(1); if out.timed_out && std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") {
if !status.success() { eprintln!("❌ PyVM timeout");
if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") {
eprintln!("❌ PyVM failed (status={})", code);
}
} }
process::exit(code); process::exit(code);
} else { } else {

View File

@ -248,14 +248,15 @@ impl NyashRunner {
if py.exists() { if py.exists() {
let mut cmd = std::process::Command::new(&py3); let mut cmd = std::process::Command::new(&py3);
cmd.arg(py).arg(&tmp_path); cmd.arg(py).arg(&tmp_path);
let out = match cmd.output() { let timeout_ms: u64 = std::env::var("NYASH_NY_COMPILER_TIMEOUT_MS")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or(2000);
let out = match super::modes::common_util::io::spawn_with_timeout(cmd, timeout_ms) {
Ok(o) => o, Ok(o) => o,
Err(e) => { Err(e) => { eprintln!("[ny-compiler] python harness failed: {}", e); return false; }
eprintln!("[ny-compiler] python harness failed to spawn: {}", e);
return false;
}
}; };
if out.status.success() { if !out.timed_out {
if let Ok(line) = String::from_utf8(out.stdout) if let Ok(line) = String::from_utf8(out.stdout)
.map(|s| s.lines().next().unwrap_or("").to_string()) .map(|s| s.lines().next().unwrap_or("").to_string())
{ {
@ -366,118 +367,18 @@ impl NyashRunner {
} }
}; };
if exe_path.exists() { if exe_path.exists() {
let mut cmd = std::process::Command::new(&exe_path);
// Prefer passing the original filename directly (parser EXE accepts positional path)
cmd.arg(filename);
// Gates
if std::env::var("NYASH_NY_COMPILER_MIN_JSON").ok().as_deref() == Some("1") {
cmd.arg("--min-json");
}
if std::env::var("NYASH_SELFHOST_READ_TMP").ok().as_deref() == Some("1") {
cmd.arg("--read-tmp");
}
if std::env::var("NYASH_NY_COMPILER_STAGE3").ok().as_deref() == Some("1") {
cmd.arg("--stage3");
}
if let Ok(raw) = std::env::var("NYASH_NY_COMPILER_CHILD_ARGS") {
for tok in raw.split_whitespace() {
cmd.arg(tok);
}
}
let timeout_ms: u64 = std::env::var("NYASH_NY_COMPILER_TIMEOUT_MS") let timeout_ms: u64 = std::env::var("NYASH_NY_COMPILER_TIMEOUT_MS")
.ok() .ok()
.and_then(|s| s.parse().ok()) .and_then(|s| s.parse().ok())
.unwrap_or(2000); .unwrap_or(2000);
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped()); if let Some(module) = super::modes::common_util::selfhost_exe::exe_try_parse_json_v0(filename, timeout_ms) {
let mut child = match cmd.spawn() {
Ok(c) => c,
Err(e) => {
eprintln!("[ny-compiler] exe spawn failed: {}", e);
return false;
}
};
let mut ch_stdout = child.stdout.take();
let mut ch_stderr = child.stderr.take();
let start = Instant::now();
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_status)) => {
break;
}
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) {
let _ = child.kill();
let _ = child.wait();
timed_out = true;
break;
}
sleep(Duration::from_millis(10));
}
Err(e) => {
eprintln!("[ny-compiler] exe wait error: {}", e);
return false;
}
}
}
let mut out_buf = Vec::new();
let mut err_buf = Vec::new();
if let Some(mut s) = ch_stdout {
let _ = s.read_to_end(&mut out_buf);
}
if let Some(mut s) = ch_stderr {
let _ = s.read_to_end(&mut err_buf);
}
if timed_out {
let head = String::from_utf8_lossy(&out_buf)
.chars()
.take(200)
.collect::<String>();
eprintln!(
"[ny-compiler] exe timeout after {} ms; stdout(head)='{}'",
timeout_ms,
head.replace('\n', "\\n")
);
return false;
}
let stdout = match String::from_utf8(out_buf) {
Ok(s) => s,
Err(_) => String::new(),
};
let mut json_line = String::new();
for line in stdout.lines() {
let t = line.trim();
if t.starts_with('{') && t.contains("\"version\"") && t.contains("\"kind\"") {
json_line = t.to_string();
break;
}
}
if json_line.is_empty() {
if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") {
let head: String = stdout.chars().take(200).collect();
let errh: String = String::from_utf8_lossy(&err_buf)
.chars()
.take(200)
.collect();
eprintln!("[ny-compiler] exe produced no JSON; stdout(head)='{}' stderr(head)='{}'", head.replace('\n', "\\n"), errh.replace('\n', "\\n"));
}
return false;
}
// Parse JSON v0 → MIR module
match super::json_v0_bridge::parse_json_v0_to_module(&json_line) {
Ok(module) => {
println!("🚀 Ny compiler EXE path (ny→json_v0) ON");
super::json_v0_bridge::maybe_dump_mir(&module); super::json_v0_bridge::maybe_dump_mir(&module);
let emit_only = std::env::var("NYASH_NY_COMPILER_EMIT_ONLY") let emit_only = std::env::var("NYASH_NY_COMPILER_EMIT_ONLY")
.unwrap_or_else(|_| "1".to_string()) .unwrap_or_else(|_| "1".to_string())
== "1"; == "1";
if emit_only { if emit_only { return false; }
return false; // Prefer PyVM when requested (reference semantics)
} else { if std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1") {
// Prefer PyVM when requested (reference semantics), regardless of BoxCall presence
let prefer_pyvm =
std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1");
if prefer_pyvm {
if let Ok(py3) = which::which("python3") { if let Ok(py3) = which::which("python3") {
let runner = std::path::Path::new("tools/pyvm_runner.py"); let runner = std::path::Path::new("tools/pyvm_runner.py");
if runner.exists() { if runner.exists() {
@ -488,44 +389,17 @@ impl NyashRunner {
eprintln!("❌ PyVM MIR JSON emit error: {}", e); eprintln!("❌ PyVM MIR JSON emit error: {}", e);
process::exit(1); process::exit(1);
} }
if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref() == Some("1") {
== Some("1") eprintln!("[Bridge] using PyVM (selfhost) → {}", mir_json_path.display());
{
eprintln!(
"[Bridge] using PyVM (selfhost) → {}",
mir_json_path.display()
);
} }
let entry = if module.functions.contains_key("Main.main") { let entry = if module.functions.contains_key("Main.main") { "Main.main" }
"Main.main" else if module.functions.contains_key("main") { "main" } else { "Main.main" };
} else if module.functions.contains_key("main") {
"main"
} else {
"Main.main"
};
let status = std::process::Command::new(py3) let status = std::process::Command::new(py3)
.args([ .args(["tools/pyvm_runner.py", "--in", &mir_json_path.display().to_string(), "--entry", entry])
"tools/pyvm_runner.py",
"--in",
&mir_json_path.display().to_string(),
"--entry",
entry,
])
.status() .status()
.map_err(|e| format!("spawn pyvm: {}", e)) .map_err(|e| format!("spawn pyvm: {}", e))
.unwrap(); .unwrap();
let code = status.code().unwrap_or(1); let code = status.code().unwrap_or(1);
if !status.success() {
if std::env::var("NYASH_CLI_VERBOSE").ok().as_deref()
== Some("1")
{
eprintln!(
"❌ PyVM (selfhost) failed (status={})",
code
);
}
}
// Harmonize with interpreter path for smokes: print Result then exit code
println!("Result: {}", code); println!("Result: {}", code);
std::process::exit(code); std::process::exit(code);
} }
@ -533,15 +407,11 @@ impl NyashRunner {
} }
self.execute_mir_module(&module); self.execute_mir_module(&module);
return true; return true;
} } else {
}
Err(e) => {
eprintln!("[ny-compiler] json parse error: {}", e);
return false; return false;
} }
} }
} }
}
// Fallback: inline VM run (embed source into a tiny wrapper that prints JSON) // Fallback: inline VM run (embed source into a tiny wrapper that prints JSON)
// This avoids CLI arg forwarding complexity and does not require FileBox. // This avoids CLI arg forwarding complexity and does not require FileBox.
@ -578,52 +448,15 @@ impl NyashRunner {
.ok() .ok()
.and_then(|s| s.parse().ok()) .and_then(|s| s.parse().ok())
.unwrap_or(2000); .unwrap_or(2000);
let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped()); let out = match super::modes::common_util::io::spawn_with_timeout(cmd, timeout_ms) {
let mut child = match cmd.spawn() { Ok(o) => o,
Ok(c) => c, Err(e) => { eprintln!("[ny-compiler] spawn inline vm failed: {}", e); return false; }
Err(e) => {
eprintln!("[ny-compiler] spawn inline vm failed: {}", e);
return false;
}
}; };
let mut ch_stdout = child.stdout.take(); if out.timed_out {
let mut ch_stderr = child.stderr.take(); let head = String::from_utf8_lossy(&out.stdout).chars().take(200).collect::<String>();
let start = Instant::now(); eprintln!("[ny-compiler] inline timeout after {} ms; stdout(head)='{}'", timeout_ms, head.replace('\n', "\\n"));
let mut timed_out = false;
loop {
match child.try_wait() {
Ok(Some(_)) => break,
Ok(None) => {
if start.elapsed() >= Duration::from_millis(timeout_ms) {
let _ = child.kill();
let _ = child.wait();
timed_out = true;
break;
} }
sleep(Duration::from_millis(10)); raw = String::from_utf8_lossy(&out.stdout).to_string();
}
Err(e) => {
eprintln!("[ny-compiler] inline wait error: {}", e);
break;
}
}
}
let mut out_buf = Vec::new();
if let Some(mut s) = ch_stdout {
let _ = s.read_to_end(&mut out_buf);
}
if timed_out {
let head = String::from_utf8_lossy(&out_buf)
.chars()
.take(200)
.collect::<String>();
eprintln!(
"[ny-compiler] inline timeout after {} ms; stdout(head)='{}'",
timeout_ms,
head.replace('\n', "\\n")
);
}
raw = String::from_utf8_lossy(&out_buf).to_string();
} }
let mut json_line = String::new(); let mut json_line = String::new();
for line in raw.lines() { for line in raw.lines() {

View File

@ -11,11 +11,13 @@ pub enum BarrierKind {
/// GC hooks that execution engines may call at key points. /// GC hooks that execution engines may call at key points.
/// Implementations must be Send + Sync for multi-thread preparation. /// Implementations must be Send + Sync for multi-thread preparation.
pub trait GcHooks: Send + Sync { pub trait GcHooks: Send + Sync + std::any::Any {
/// Safe point for cooperative GC (e.g., poll or yield). /// Safe point for cooperative GC (e.g., poll or yield).
fn safepoint(&self) {} fn safepoint(&self) {}
/// Memory barrier hint for loads/stores. /// Memory barrier hint for loads/stores.
fn barrier(&self, _kind: BarrierKind) {} fn barrier(&self, _kind: BarrierKind) {}
/// Allocation accounting (bytes are best-effort; may be 0 when unknown)
fn alloc(&self, _bytes: u64) {}
/// Optional counters snapshot for diagnostics. Default: None. /// Optional counters snapshot for diagnostics. Default: None.
fn snapshot_counters(&self) -> Option<(u64, u64, u64)> { fn snapshot_counters(&self) -> Option<(u64, u64, u64)> {
None None
@ -27,48 +29,35 @@ pub struct NullGc;
impl GcHooks for NullGc {} impl GcHooks for NullGc {}
use std::sync::atomic::{AtomicU64, Ordering}; /// CountingGc is now a thin wrapper around the unified GcController.
/// Simple counting GC (PoC): counts safepoints and barriers.
/// Useful to validate hook frequency without affecting semantics.
pub struct CountingGc { pub struct CountingGc {
pub safepoints: AtomicU64, inner: crate::runtime::gc_controller::GcController,
pub barrier_reads: AtomicU64,
pub barrier_writes: AtomicU64,
} }
impl CountingGc { impl CountingGc {
pub fn new() -> Self { pub fn new() -> Self {
// Default to rc+cycle mode for development metrics
let mode = crate::runtime::gc_mode::GcMode::RcCycle;
Self { Self {
safepoints: AtomicU64::new(0), inner: crate::runtime::gc_controller::GcController::new(mode),
barrier_reads: AtomicU64::new(0),
barrier_writes: AtomicU64::new(0),
} }
} }
pub fn snapshot(&self) -> (u64, u64, u64) { pub fn snapshot(&self) -> (u64, u64, u64) {
( self.inner.snapshot()
self.safepoints.load(Ordering::Relaxed),
self.barrier_reads.load(Ordering::Relaxed),
self.barrier_writes.load(Ordering::Relaxed),
)
} }
} }
impl GcHooks for CountingGc { impl GcHooks for CountingGc {
fn safepoint(&self) { fn safepoint(&self) {
self.safepoints.fetch_add(1, Ordering::Relaxed); self.inner.safepoint();
} }
fn barrier(&self, kind: BarrierKind) { fn barrier(&self, kind: BarrierKind) {
match kind { self.inner.barrier(kind);
BarrierKind::Read => {
self.barrier_reads.fetch_add(1, Ordering::Relaxed);
}
BarrierKind::Write => {
self.barrier_writes.fetch_add(1, Ordering::Relaxed);
}
} }
fn alloc(&self, bytes: u64) {
self.inner.alloc(bytes);
} }
fn snapshot_counters(&self) -> Option<(u64, u64, u64)> { fn snapshot_counters(&self) -> Option<(u64, u64, u64)> {
Some(self.snapshot()) Some(self.inner.snapshot())
} }
} }

View File

@ -0,0 +1,207 @@
//! Unified GC controller (skeleton)
//! Implements GcHooks and centralizes mode selection and metrics.
use std::sync::atomic::{AtomicU64, Ordering};
use super::gc::{BarrierKind, GcHooks};
use super::gc_mode::GcMode;
use crate::config::env;
use crate::runtime::gc_trace;
use std::collections::{HashSet, VecDeque};
pub struct GcController {
mode: GcMode,
safepoints: AtomicU64,
barrier_reads: AtomicU64,
barrier_writes: AtomicU64,
alloc_bytes: AtomicU64,
alloc_count: AtomicU64,
sp_since_last: AtomicU64,
bytes_since_last: AtomicU64,
collect_sp_interval: Option<u64>,
collect_alloc_bytes: Option<u64>,
// Diagnostics: last trial reachability counters
trial_nodes_last: AtomicU64,
trial_edges_last: AtomicU64,
// Diagnostics: collection counters and last duration/flags
collect_count_total: AtomicU64,
collect_by_sp: AtomicU64,
collect_by_alloc: AtomicU64,
trial_duration_last_ms: AtomicU64,
trial_reason_last: AtomicU64, // bitflags: 1=sp, 2=alloc
}
impl GcController {
pub fn new(mode: GcMode) -> Self {
Self {
mode,
safepoints: AtomicU64::new(0),
barrier_reads: AtomicU64::new(0),
barrier_writes: AtomicU64::new(0),
alloc_bytes: AtomicU64::new(0),
alloc_count: AtomicU64::new(0),
sp_since_last: AtomicU64::new(0),
bytes_since_last: AtomicU64::new(0),
collect_sp_interval: env::gc_collect_sp_interval(),
collect_alloc_bytes: env::gc_collect_alloc_bytes(),
trial_nodes_last: AtomicU64::new(0),
trial_edges_last: AtomicU64::new(0),
collect_count_total: AtomicU64::new(0),
collect_by_sp: AtomicU64::new(0),
collect_by_alloc: AtomicU64::new(0),
trial_duration_last_ms: AtomicU64::new(0),
trial_reason_last: AtomicU64::new(0),
}
}
pub fn mode(&self) -> GcMode {
self.mode
}
pub fn snapshot(&self) -> (u64, u64, u64) {
(
self.safepoints.load(Ordering::Relaxed),
self.barrier_reads.load(Ordering::Relaxed),
self.barrier_writes.load(Ordering::Relaxed),
)
}
}
impl GcHooks for GcController {
fn safepoint(&self) {
// Off mode: minimal overhead but still callable
if self.mode != GcMode::Off {
self.safepoints.fetch_add(1, Ordering::Relaxed);
let sp = self.sp_since_last.fetch_add(1, Ordering::Relaxed) + 1;
// Opportunistic collection trigger
let sp_hit = self
.collect_sp_interval
.map(|n| n > 0 && sp >= n)
.unwrap_or(false);
let bytes = self.bytes_since_last.load(Ordering::Relaxed);
let bytes_hit = self
.collect_alloc_bytes
.map(|n| n > 0 && bytes >= n)
.unwrap_or(false);
if sp_hit || bytes_hit {
// Record reason flags for diagnostics
let mut flags: u64 = 0;
if sp_hit { flags |= 1; self.collect_by_sp.fetch_add(1, Ordering::Relaxed); }
if bytes_hit { flags |= 2; self.collect_by_alloc.fetch_add(1, Ordering::Relaxed); }
self.trial_reason_last.store(flags, Ordering::Relaxed);
self.run_trial_collection();
}
}
// Future: per-mode collection/cooperation hooks
}
fn barrier(&self, kind: BarrierKind) {
if self.mode == GcMode::Off {
return;
}
match kind {
BarrierKind::Read => {
self.barrier_reads.fetch_add(1, Ordering::Relaxed);
}
BarrierKind::Write => {
self.barrier_writes.fetch_add(1, Ordering::Relaxed);
}
}
}
fn snapshot_counters(&self) -> Option<(u64, u64, u64)> {
Some(self.snapshot())
}
fn alloc(&self, bytes: u64) {
if self.mode == GcMode::Off {
return;
}
self.alloc_count.fetch_add(1, Ordering::Relaxed);
self.alloc_bytes.fetch_add(bytes, Ordering::Relaxed);
self.bytes_since_last
.fetch_add(bytes, Ordering::Relaxed);
}
}
impl GcController {
pub fn alloc_totals(&self) -> (u64, u64) {
(
self.alloc_count.load(Ordering::Relaxed),
self.alloc_bytes.load(Ordering::Relaxed),
)
}
}
impl GcController {
fn run_trial_collection(&self) {
// Reset windows
self.sp_since_last.store(0, Ordering::Relaxed);
self.bytes_since_last.store(0, Ordering::Relaxed);
// PoC: no object graph; report current handles as leak candidates and return.
if self.mode == GcMode::Off {
return;
}
// Only run for rc/rc+cycle/stw; rc+cycle is default.
match self.mode {
GcMode::Rc | GcMode::RcCycle | GcMode::STW => {
let started = std::time::Instant::now();
// Roots: JIT/AOT handle registry snapshot
let roots = crate::jit::rt::handles::snapshot_arcs();
let mut visited: HashSet<u64> = HashSet::new();
let mut q: VecDeque<std::sync::Arc<dyn crate::box_trait::NyashBox>> =
VecDeque::new();
for r in roots.into_iter() {
let id = r.box_id();
if visited.insert(id) {
q.push_back(r);
}
}
let mut nodes: u64 = visited.len() as u64;
let mut edges: u64 = 0;
while let Some(cur) = q.pop_front() {
gc_trace::trace_children(&*cur, &mut |child| {
edges += 1;
let id = child.box_id();
if visited.insert(id) {
nodes += 1;
q.push_back(child);
}
});
}
// Store last diagnostics (available for JSON metrics)
self.trial_nodes_last.store(nodes, Ordering::Relaxed);
self.trial_edges_last.store(edges, Ordering::Relaxed);
if (nodes + edges) > 0 && crate::config::env::gc_metrics() {
eprintln!(
"[GC] trial: reachable nodes={} edges={} (roots=jit_handles)",
nodes, edges
);
}
// Update counters
let dur = started.elapsed();
let ms = dur.as_millis() as u64;
self.trial_duration_last_ms.store(ms, Ordering::Relaxed);
self.collect_count_total.fetch_add(1, Ordering::Relaxed);
// Reason flags derive from current env thresholds vs last windows reaching triggers
// Note: we set flags in safepoint() where triggers were decided.
}
_ => {}
}
}
}
impl GcController {
pub fn trial_reachability_last(&self) -> (u64, u64) {
(
self.trial_nodes_last.load(Ordering::Relaxed),
self.trial_edges_last.load(Ordering::Relaxed),
)
}
pub fn collection_totals(&self) -> (u64, u64, u64) {
(
self.collect_count_total.load(Ordering::Relaxed),
self.collect_by_sp.load(Ordering::Relaxed),
self.collect_by_alloc.load(Ordering::Relaxed),
)
}
pub fn trial_duration_last_ms(&self) -> u64 {
self.trial_duration_last_ms.load(Ordering::Relaxed)
}
pub fn trial_reason_last_bits(&self) -> u64 { self.trial_reason_last.load(Ordering::Relaxed) }
}

34
src/runtime/gc_mode.rs Normal file
View File

@ -0,0 +1,34 @@
//! GC mode selection (user-facing)
use crate::config::env;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum GcMode {
RcCycle,
Minorgen,
STW,
Rc,
Off,
}
impl GcMode {
pub fn from_env() -> Self {
match env::gc_mode().to_ascii_lowercase().as_str() {
"auto" | "rc+cycle" => GcMode::RcCycle,
"minorgen" => GcMode::Minorgen,
"stw" => GcMode::STW,
"rc" => GcMode::Rc,
"off" => GcMode::Off,
_ => GcMode::RcCycle,
}
}
pub fn as_str(&self) -> &'static str {
match self {
GcMode::RcCycle => "rc+cycle",
GcMode::Minorgen => "minorgen",
GcMode::STW => "stw",
GcMode::Rc => "rc",
GcMode::Off => "off",
}
}
}

35
src/runtime/gc_trace.rs Normal file
View File

@ -0,0 +1,35 @@
//! Minimal GC tracing helpers (skeleton)
//!
//! Downcast-based child edge enumeration for builtin containers.
//! This is a non-invasive helper to support diagnostics and future collectors.
use std::sync::Arc;
use crate::box_trait::NyashBox;
/// Visit child boxes of a given object and invoke `visit(child)` for each.
/// This function recognizes builtin containers (ArrayBox/MapBox) and is a no-op otherwise.
pub fn trace_children(obj: &dyn NyashBox, visit: &mut dyn FnMut(Arc<dyn NyashBox>)) {
// ArrayBox
if let Some(arr) = obj.as_any().downcast_ref::<crate::boxes::array::ArrayBox>() {
if let Ok(items) = arr.items.read() {
for it in items.iter() {
// Convert Box<dyn NyashBox> to Arc<dyn NyashBox>
let arc: Arc<dyn NyashBox> = Arc::from(it.clone_box());
visit(arc);
}
}
return;
}
// MapBox
if let Some(map) = obj.as_any().downcast_ref::<crate::boxes::map_box::MapBox>() {
if let Ok(data) = map.get_data().read() {
for (_k, v) in data.iter() {
let arc: Arc<dyn NyashBox> = Arc::from(v.clone_box());
visit(arc);
}
}
return;
}
}

View File

@ -4,97 +4,74 @@ use once_cell::sync::OnceCell;
use std::sync::{Arc, RwLock}; use std::sync::{Arc, RwLock};
use super::scheduler::CancellationToken; use super::scheduler::CancellationToken;
use super::{gc::GcHooks, scheduler::Scheduler}; use super::{gc::BarrierKind, gc::GcHooks, scheduler::Scheduler};
static GLOBAL_GC: OnceCell<RwLock<Option<Arc<dyn GcHooks>>>> = OnceCell::new(); // Unified global runtime hooks state (single lock for consistency)
static GLOBAL_SCHED: OnceCell<RwLock<Option<Arc<dyn Scheduler>>>> = OnceCell::new(); struct GlobalHooksState {
// Phase 2 scaffold: current task group's cancellation token (no-op default) gc: Option<Arc<dyn GcHooks>>,
static GLOBAL_CUR_TOKEN: OnceCell<RwLock<Option<CancellationToken>>> = OnceCell::new(); sched: Option<Arc<dyn Scheduler>>,
// Phase 2 scaffold: current group's child futures registry (best-effort) cur_token: Option<CancellationToken>,
static GLOBAL_GROUP_FUTURES: OnceCell<RwLock<Vec<crate::boxes::future::FutureWeak>>> = futures: Vec<crate::boxes::future::FutureWeak>,
OnceCell::new(); strong: Vec<crate::boxes::future::FutureBox>,
// Strong ownership list for implicit group (pre-TaskGroup actualization) scope_depth: usize,
static GLOBAL_GROUP_STRONG: OnceCell<RwLock<Vec<crate::boxes::future::FutureBox>>> = group_stack: Vec<std::sync::Arc<crate::boxes::task_group_box::TaskGroupInner>>,
OnceCell::new(); }
// Simple scope depth counter for implicit group (join-at-scope-exit footing)
static TASK_SCOPE_DEPTH: OnceCell<RwLock<usize>> = OnceCell::new();
// TaskGroup scope stack (explicit group ownership per function scope)
static TASK_GROUP_STACK: OnceCell<
RwLock<Vec<std::sync::Arc<crate::boxes::task_group_box::TaskGroupInner>>>,
> = OnceCell::new();
fn gc_cell() -> &'static RwLock<Option<Arc<dyn GcHooks>>> { impl GlobalHooksState {
GLOBAL_GC.get_or_init(|| RwLock::new(None)) fn new() -> Self {
Self {
gc: None,
sched: None,
cur_token: None,
futures: Vec::new(),
strong: Vec::new(),
scope_depth: 0,
group_stack: Vec::new(),
}
}
} }
fn sched_cell() -> &'static RwLock<Option<Arc<dyn Scheduler>>> {
GLOBAL_SCHED.get_or_init(|| RwLock::new(None)) static GLOBAL_STATE: OnceCell<RwLock<GlobalHooksState>> = OnceCell::new();
}
fn token_cell() -> &'static RwLock<Option<CancellationToken>> { fn state() -> &'static RwLock<GlobalHooksState> {
GLOBAL_CUR_TOKEN.get_or_init(|| RwLock::new(None)) GLOBAL_STATE.get_or_init(|| RwLock::new(GlobalHooksState::new()))
}
fn futures_cell() -> &'static RwLock<Vec<crate::boxes::future::FutureWeak>> {
GLOBAL_GROUP_FUTURES.get_or_init(|| RwLock::new(Vec::new()))
}
fn strong_cell() -> &'static RwLock<Vec<crate::boxes::future::FutureBox>> {
GLOBAL_GROUP_STRONG.get_or_init(|| RwLock::new(Vec::new()))
}
fn scope_depth_cell() -> &'static RwLock<usize> {
TASK_SCOPE_DEPTH.get_or_init(|| RwLock::new(0))
}
fn group_stack_cell(
) -> &'static RwLock<Vec<std::sync::Arc<crate::boxes::task_group_box::TaskGroupInner>>> {
TASK_GROUP_STACK.get_or_init(|| RwLock::new(Vec::new()))
} }
pub fn set_from_runtime(rt: &crate::runtime::nyash_runtime::NyashRuntime) { pub fn set_from_runtime(rt: &crate::runtime::nyash_runtime::NyashRuntime) {
if let Ok(mut g) = gc_cell().write() { if let Ok(mut st) = state().write() {
*g = Some(rt.gc.clone()); st.gc = Some(rt.gc.clone());
st.sched = rt.scheduler.as_ref().cloned();
if st.cur_token.is_none() {
st.cur_token = Some(CancellationToken::new());
} }
if let Ok(mut s) = sched_cell().write() { st.futures.clear();
*s = rt.scheduler.as_ref().cloned(); st.strong.clear();
} st.scope_depth = 0;
// Optional: initialize a fresh token for the runtime's root group (Phase 2 wiring) st.group_stack.clear();
if let Ok(mut t) = token_cell().write() {
if t.is_none() {
*t = Some(CancellationToken::new());
}
}
// Reset group futures registry on new runtime
if let Ok(mut f) = futures_cell().write() {
f.clear();
}
if let Ok(mut s) = strong_cell().write() {
s.clear();
}
if let Ok(mut d) = scope_depth_cell().write() {
*d = 0;
}
if let Ok(mut st) = group_stack_cell().write() {
st.clear();
} }
} }
pub fn set_gc(gc: Arc<dyn GcHooks>) { pub fn set_gc(gc: Arc<dyn GcHooks>) {
if let Ok(mut g) = gc_cell().write() { if let Ok(mut st) = state().write() {
*g = Some(gc); st.gc = Some(gc);
} }
} }
pub fn set_scheduler(s: Arc<dyn Scheduler>) { pub fn set_scheduler(s: Arc<dyn Scheduler>) {
if let Ok(mut w) = sched_cell().write() { if let Ok(mut st) = state().write() {
*w = Some(s); st.sched = Some(s);
} }
} }
/// Set the current task group's cancellation token (scaffold). /// Set the current task group's cancellation token (scaffold).
pub fn set_current_group_token(tok: CancellationToken) { pub fn set_current_group_token(tok: CancellationToken) {
if let Ok(mut w) = token_cell().write() { if let Ok(mut st) = state().write() {
*w = Some(tok); st.cur_token = Some(tok);
} }
} }
/// Get the current task group's cancellation token (no-op default). /// Get the current task group's cancellation token (no-op default).
pub fn current_group_token() -> CancellationToken { pub fn current_group_token() -> CancellationToken {
if let Ok(r) = token_cell().read() { if let Ok(st) = state().read() {
if let Some(t) = r.as_ref() { if let Some(t) = st.cur_token.as_ref() {
return t.clone(); return t.clone();
} }
} }
@ -103,21 +80,17 @@ pub fn current_group_token() -> CancellationToken {
/// Register a Future into the current group's registry (best-effort; clones share state) /// Register a Future into the current group's registry (best-effort; clones share state)
pub fn register_future_to_current_group(fut: &crate::boxes::future::FutureBox) { pub fn register_future_to_current_group(fut: &crate::boxes::future::FutureBox) {
if let Ok(mut st) = state().write() {
// Prefer explicit current TaskGroup at top of stack // Prefer explicit current TaskGroup at top of stack
if let Ok(st) = group_stack_cell().read() { if let Some(inner) = st.group_stack.last() {
if let Some(inner) = st.last() {
if let Ok(mut v) = inner.strong.lock() { if let Ok(mut v) = inner.strong.lock() {
v.push(fut.clone()); v.push(fut.clone());
return; return;
} }
} }
}
// Fallback to implicit global group // Fallback to implicit global group
if let Ok(mut list) = futures_cell().write() { st.futures.push(fut.downgrade());
list.push(fut.downgrade()); st.strong.push(fut.clone());
}
if let Ok(mut s) = strong_cell().write() {
s.push(fut.clone());
} }
} }
@ -127,21 +100,11 @@ pub fn join_all_registered_futures(timeout_ms: u64) {
let deadline = Instant::now() + Duration::from_millis(timeout_ms); let deadline = Instant::now() + Duration::from_millis(timeout_ms);
loop { loop {
let mut all_ready = true; let mut all_ready = true;
// purge list of dropped or completed futures opportunistically // purge + readiness check under single state lock (short critical sections)
{ if let Ok(mut st) = state().write() {
// purge weak list: keep only upgradeable futures st.futures.retain(|fw| fw.is_ready().is_some());
if let Ok(mut list) = futures_cell().write() { st.strong.retain(|f| !f.ready());
list.retain(|fw| fw.is_ready().is_some()); for fw in st.futures.iter() {
}
// purge strong list: remove completed futures to reduce retention
if let Ok(mut s) = strong_cell().write() {
s.retain(|f| !f.ready());
}
}
// check readiness
{
if let Ok(list) = futures_cell().read() {
for fw in list.iter() {
if let Some(ready) = fw.is_ready() { if let Some(ready) = fw.is_ready() {
if !ready { if !ready {
all_ready = false; all_ready = false;
@ -150,7 +113,6 @@ pub fn join_all_registered_futures(timeout_ms: u64) {
} }
} }
} }
}
if all_ready { if all_ready {
break; break;
} }
@ -161,22 +123,18 @@ pub fn join_all_registered_futures(timeout_ms: u64) {
std::thread::yield_now(); std::thread::yield_now();
} }
// Final sweep // Final sweep
if let Ok(mut s) = strong_cell().write() { if let Ok(mut st) = state().write() {
s.retain(|f| !f.ready()); st.strong.retain(|f| !f.ready());
} st.futures.retain(|fw| matches!(fw.is_ready(), Some(false)));
if let Ok(mut list) = futures_cell().write() {
list.retain(|fw| matches!(fw.is_ready(), Some(false)));
} }
} }
/// Push a task scope (footing). On pop of the outermost scope, perform a best-effort join. /// Push a task scope (footing). On pop of the outermost scope, perform a best-effort join.
pub fn push_task_scope() { pub fn push_task_scope() {
if let Ok(mut d) = scope_depth_cell().write() { if let Ok(mut st) = state().write() {
*d += 1; st.scope_depth += 1;
}
// Push a new explicit TaskGroup for this scope // Push a new explicit TaskGroup for this scope
if let Ok(mut st) = group_stack_cell().write() { st.group_stack.push(std::sync::Arc::new(
st.push(std::sync::Arc::new(
crate::boxes::task_group_box::TaskGroupInner { crate::boxes::task_group_box::TaskGroupInner {
strong: std::sync::Mutex::new(Vec::new()), strong: std::sync::Mutex::new(Vec::new()),
}, },
@ -190,19 +148,13 @@ pub fn push_task_scope() {
pub fn pop_task_scope() { pub fn pop_task_scope() {
let mut do_join = false; let mut do_join = false;
let mut popped: Option<std::sync::Arc<crate::boxes::task_group_box::TaskGroupInner>> = None; let mut popped: Option<std::sync::Arc<crate::boxes::task_group_box::TaskGroupInner>> = None;
{ if let Ok(mut st) = state().write() {
if let Ok(mut d) = scope_depth_cell().write() { if st.scope_depth > 0 {
if *d > 0 { st.scope_depth -= 1;
*d -= 1;
}
if *d == 0 {
do_join = true;
}
}
} }
if st.scope_depth == 0 { do_join = true; }
// Pop explicit group for this scope // Pop explicit group for this scope
if let Ok(mut st) = group_stack_cell().write() { popped = st.group_stack.pop();
popped = st.pop();
} }
if do_join { if do_join {
let ms: u64 = std::env::var("NYASH_TASK_SCOPE_JOIN_MS") let ms: u64 = std::env::var("NYASH_TASK_SCOPE_JOIN_MS")
@ -240,13 +192,11 @@ pub fn pop_task_scope() {
/// Perform a runtime safepoint and poll the scheduler if available. /// Perform a runtime safepoint and poll the scheduler if available.
pub fn safepoint_and_poll() { pub fn safepoint_and_poll() {
if let Ok(g) = gc_cell().read() { if let Ok(st) = state().read() {
if let Some(gc) = g.as_ref() { if let Some(gc) = st.gc.as_ref() {
gc.safepoint(); gc.safepoint();
} }
} if let Some(sched) = st.sched.as_ref() {
if let Ok(s) = sched_cell().read() {
if let Some(sched) = s.as_ref() {
sched.poll(); sched.poll();
} }
} }
@ -255,8 +205,8 @@ pub fn safepoint_and_poll() {
/// Try to schedule a task on the global scheduler. Returns true if scheduled. /// Try to schedule a task on the global scheduler. Returns true if scheduled.
pub fn spawn_task(name: &str, f: Box<dyn FnOnce() + Send + 'static>) -> bool { pub fn spawn_task(name: &str, f: Box<dyn FnOnce() + Send + 'static>) -> bool {
// If a scheduler is registered, enqueue the task; otherwise run inline. // If a scheduler is registered, enqueue the task; otherwise run inline.
if let Ok(s) = sched_cell().read() { if let Ok(st) = state().read() {
if let Some(sched) = s.as_ref() { if let Some(sched) = st.sched.as_ref() {
sched.spawn(name, f); sched.spawn(name, f);
return true; return true;
} }
@ -272,8 +222,8 @@ pub fn spawn_task_with_token(
token: crate::runtime::scheduler::CancellationToken, token: crate::runtime::scheduler::CancellationToken,
f: Box<dyn FnOnce() + Send + 'static>, f: Box<dyn FnOnce() + Send + 'static>,
) -> bool { ) -> bool {
if let Ok(s) = sched_cell().read() { if let Ok(st) = state().read() {
if let Some(sched) = s.as_ref() { if let Some(sched) = st.sched.as_ref() {
sched.spawn_with_token(name, token, f); sched.spawn_with_token(name, token, f);
return true; return true;
} }
@ -284,8 +234,8 @@ pub fn spawn_task_with_token(
/// Spawn a delayed task via scheduler if available; returns true if scheduled. /// Spawn a delayed task via scheduler if available; returns true if scheduled.
pub fn spawn_task_after(delay_ms: u64, name: &str, f: Box<dyn FnOnce() + Send + 'static>) -> bool { pub fn spawn_task_after(delay_ms: u64, name: &str, f: Box<dyn FnOnce() + Send + 'static>) -> bool {
if let Ok(s) = sched_cell().read() { if let Ok(st) = state().read() {
if let Some(sched) = s.as_ref() { if let Some(sched) = st.sched.as_ref() {
sched.spawn_after(delay_ms, name, f); sched.spawn_after(delay_ms, name, f);
return true; return true;
} }
@ -297,3 +247,21 @@ pub fn spawn_task_after(delay_ms: u64, name: &str, f: Box<dyn FnOnce() + Send +
}); });
false false
} }
/// Forward a GC barrier event to the currently registered GC hooks (if any).
pub fn gc_barrier(kind: BarrierKind) {
if let Ok(st) = state().read() {
if let Some(gc) = st.gc.as_ref() {
gc.barrier(kind);
}
}
}
/// Report an allocation to the current GC hooks (best-effort)
pub fn gc_alloc(bytes: u64) {
if let Ok(st) = state().read() {
if let Some(gc) = st.gc.as_ref() {
gc.alloc(bytes);
}
}
}

View File

@ -4,6 +4,9 @@
pub mod box_registry; pub mod box_registry;
pub mod gc; pub mod gc;
pub mod gc_controller;
pub mod gc_mode;
pub mod gc_trace;
pub mod global_hooks; pub mod global_hooks;
pub mod leak_tracker; pub mod leak_tracker;
pub mod nyash_runtime; pub mod nyash_runtime;

View File

@ -1,4 +1,4 @@
use super::types::{LoadedPluginV2, PluginBoxMetadata, PluginBoxV2, PluginHandleInner}; use super::types::{LoadedPluginV2, NyashTypeBoxFfi, PluginBoxMetadata, PluginBoxV2, PluginHandleInner};
use crate::bid::{BidError, BidResult}; use crate::bid::{BidError, BidResult};
use crate::box_trait::NyashBox; use crate::box_trait::NyashBox;
use crate::config::nyash_toml_v2::{LibraryDefinition, NyashConfigV2}; use crate::config::nyash_toml_v2::{LibraryDefinition, NyashConfigV2};
@ -12,11 +12,15 @@ fn dbg_on() -> bool {
std::env::var("NYASH_DEBUG_PLUGIN").unwrap_or_default() == "1" std::env::var("NYASH_DEBUG_PLUGIN").unwrap_or_default() == "1"
} }
type BoxInvokeFn = extern "C" fn(u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32;
#[derive(Debug, Clone, Default)] #[derive(Debug, Clone, Default)]
struct LoadedBoxSpec { struct LoadedBoxSpec {
type_id: Option<u32>, type_id: Option<u32>,
methods: HashMap<String, MethodSpec>, methods: HashMap<String, MethodSpec>,
fini_method_id: Option<u32>, fini_method_id: Option<u32>,
// Optional Nyash ABI v2 per-box invoke entry (not yet used for calls)
invoke_id: Option<BoxInvokeFn>,
} }
#[derive(Debug, Clone, Copy)] #[derive(Debug, Clone, Copy)]
struct MethodSpec { struct MethodSpec {
@ -124,7 +128,7 @@ impl PluginLoaderV2 {
let lib = unsafe { Library::new(&lib_path) }.map_err(|_| BidError::PluginError)?; let lib = unsafe { Library::new(&lib_path) }.map_err(|_| BidError::PluginError)?;
let lib_arc = Arc::new(lib); let lib_arc = Arc::new(lib);
// Resolve required invoke symbol (TypeBox v2: nyash_plugin_invoke) // Resolve required invoke symbol (legacy library-level): nyash_plugin_invoke
unsafe { unsafe {
let invoke_sym: Symbol< let invoke_sym: Symbol<
unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32, unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32,
@ -152,6 +156,35 @@ impl PluginLoaderV2 {
.insert(lib_name.to_string(), Arc::new(loaded)); .insert(lib_name.to_string(), Arc::new(loaded));
} }
// Try to resolve Nyash ABI v2 per-box TypeBox symbols and record invoke_id
// Symbol pattern: nyash_typebox_<BoxType>
for box_type in &lib_def.boxes {
let sym_name = format!("nyash_typebox_{}\0", box_type);
unsafe {
if let Ok(tb_sym) = lib_arc.get::<Symbol<&NyashTypeBoxFfi>>(sym_name.as_bytes()) {
let st: &NyashTypeBoxFfi = &*tb_sym;
// Validate ABI tag 'TYBX' (0x54594258) and basic invariants
let abi_ok = st.abi_tag == 0x5459_4258
&& st.struct_size as usize >= std::mem::size_of::<NyashTypeBoxFfi>();
if !abi_ok {
continue;
}
// Remember invoke_id in box_specs for (lib_name, box_type)
if let Some(invoke_id) = st.invoke_id {
let key = (lib_name.to_string(), box_type.to_string());
let mut map = self.box_specs.write().map_err(|_| BidError::PluginError)?;
let entry = map.entry(key).or_insert(LoadedBoxSpec {
type_id: None,
methods: HashMap::new(),
fini_method_id: None,
invoke_id: None,
});
entry.invoke_id = Some(invoke_id);
}
}
}
}
Ok(()) Ok(())
} }

View File

@ -13,13 +13,16 @@ fi
echo "[bootstrap] c0 (rust) → c1 (ny) → c1' parity (JIT-only)" >&2 echo "[bootstrap] c0 (rust) → c1 (ny) → c1' parity (JIT-only)" >&2
# c0: baseline run (rust path) # c0: baseline run (rust path)
NYASH_DISABLE_PLUGINS=1 NYASH_CLI_VERBOSE=1 "$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c0.out timeout -s KILL 20s env NYASH_DISABLE_PLUGINS=1 NYASH_CLI_VERBOSE=1 \
"$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c0.out
# c1: try Ny compiler path (flagged); tolerate fallback to rust path # c1: try Ny compiler path (flagged); tolerate fallback to rust path
NYASH_DISABLE_PLUGINS=1 NYASH_USE_NY_COMPILER=1 NYASH_CLI_VERBOSE=1 "$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c1.out || true timeout -s KILL 20s env NYASH_DISABLE_PLUGINS=1 NYASH_USE_NY_COMPILER=1 NYASH_CLI_VERBOSE=1 \
"$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c1.out || true
# c1': re-run (simulated second pass) # c1': re-run (simulated second pass)
NYASH_DISABLE_PLUGINS=1 NYASH_USE_NY_COMPILER=1 NYASH_CLI_VERBOSE=1 "$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c1p.out || true timeout -s KILL 20s env NYASH_DISABLE_PLUGINS=1 NYASH_USE_NY_COMPILER=1 NYASH_CLI_VERBOSE=1 \
"$BIN" --backend vm "$ROOT_DIR/apps/examples/string_p0.nyash" > /tmp/nyash-c1p.out || true
H0=$(rg -n '^Result:\s*' /tmp/nyash-c0.out | sed 's/\s\+/ /g') H0=$(rg -n '^Result:\s*' /tmp/nyash-c0.out | sed 's/\s\+/ /g')
H1=$(rg -n '^Result:\s*' /tmp/nyash-c1.out | sed 's/\s\+/ /g' || true) H1=$(rg -n '^Result:\s*' /tmp/nyash-c1.out | sed 's/\s\+/ /g' || true)

View File

@ -15,7 +15,10 @@ mkdir -p dist/nyash_compiler/tmp
echo 'return 1+2*3' > dist/nyash_compiler/tmp/sample_exe_smoke.nyash echo 'return 1+2*3' > dist/nyash_compiler/tmp/sample_exe_smoke.nyash
echo "[3/4] Running parser EXE → JSON ..." echo "[3/4] Running parser EXE → JSON ..."
(cd dist/nyash_compiler && ./nyash_compiler tmp/sample_exe_smoke.nyash > sample.json) (cd dist/nyash_compiler && timeout -s KILL 60s ./nyash_compiler tmp/sample_exe_smoke.nyash > sample.json)
echo "[3.5/4] Validating JSON schema ..."
python3 tools/validate_mir_json.py dist/nyash_compiler/sample.json
if ! head -n1 dist/nyash_compiler/sample.json | grep -q '"kind":"Program"'; then if ! head -n1 dist/nyash_compiler/sample.json | grep -q '"kind":"Program"'; then
echo "error: JSON does not look like a Program" >&2 echo "error: JSON does not look like a Program" >&2
@ -26,7 +29,7 @@ echo "[4/4] Executing via bridge (pipe) to verify semantics ..."
# Keep core minimal and deterministic # Keep core minimal and deterministic
export NYASH_DISABLE_PLUGINS=1 export NYASH_DISABLE_PLUGINS=1
set +e set +e
cat dist/nyash_compiler/sample.json | ./target/release/nyash --ny-parser-pipe --backend vm >/dev/null timeout -s KILL 60s bash -c 'cat dist/nyash_compiler/sample.json | ./target/release/nyash --ny-parser-pipe --backend vm >/dev/null'
RC=$? RC=$?
set -e set -e
if [[ "$RC" -ne 7 ]]; then if [[ "$RC" -ne 7 ]]; then
@ -36,4 +39,3 @@ fi
echo "✅ EXE-first smoke passed (parser EXE + bridge run)" echo "✅ EXE-first smoke passed (parser EXE + bridge run)"
exit 0 exit 0

View File

@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""
Validate a MIR JSON file against the Nyash JSON v0 schema.
Usage:
python3 tools/validate_mir_json.py <file.json> [--schema docs/reference/mir/json_v0.schema.json]
Requires the 'jsonschema' Python package. Install via:
python3 -m pip install jsonschema
"""
import argparse
import json
import sys
from pathlib import Path
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument('json_file', help='MIR JSON file path')
ap.add_argument('--schema', default='docs/reference/mir/json_v0.schema.json', help='Schema JSON path')
args = ap.parse_args()
try:
import jsonschema # type: ignore
except Exception:
print('[schema] error: Python package "jsonschema" not found.\n'
'Install with: python3 -m pip install jsonschema', file=sys.stderr)
return 2
try:
with open(args.json_file, 'r', encoding='utf-8') as f:
data = json.load(f)
except Exception as e:
print(f'[schema] error: failed to read JSON: {e}', file=sys.stderr)
return 3
try:
with open(args.schema, 'r', encoding='utf-8') as f:
schema = json.load(f)
except Exception as e:
print(f'[schema] error: failed to read schema: {e}', file=sys.stderr)
return 4
try:
jsonschema.validate(instance=data, schema=schema)
except jsonschema.ValidationError as e: # type: ignore
# Show human-friendly context
path = '/'.join([str(p) for p in e.path])
print(f'[schema] validation failed at $.{path}: {e.message}', file=sys.stderr)
return 5
print('[schema] validation OK')
return 0
if __name__ == '__main__':
sys.exit(main())