ci: add GitHub Actions workflow; runner: prefer PyVM in selfhost paths; stage3: add LLVM + bridge acceptance smokes; docs: update env flags

2025-09-17 01:20:15 +09:00
parent 5c9213cd03
commit adbfbb2c76
11 changed files with 212 additions and 20 deletions
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@ -0,0 +1,51 @@
+name: nyash-ci
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  build-and-smoke:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install LLVM 18 (apt.llvm.org)
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y wget gnupg lsb-release
+          wget https://apt.llvm.org/llvm.sh
+          chmod +x llvm.sh
+          sudo ./llvm.sh 18
+
+      - name: Set LLVM env
+        run: echo "LLVM_SYS_180_PREFIX=$(llvm-config-18 --prefix)" >> "$GITHUB_ENV"
+
+      - name: Ensure Python3
+        run: sudo apt-get install -y python3
+
+      - name: Set up Rust
+        uses: dtolnay/rust-toolchain@stable
+
+      - name: Cache Rust build
+        uses: Swatinem/rust-cache@v2
+        with:
+          cache-targets: true
+
+      - name: Build (LLVM feature)
+        run: cargo build --release --features llvm
+
+      - name: Curated LLVM smokes
+        run: ./tools/smokes/curated_llvm.sh
+
+      - name: Curated LLVM Stage-3 smokes
+        run: ./tools/smokes/curated_llvm_stage3.sh
+
+      - name: Bridge Stage-3 acceptance (JSON v0 pipe)
+        run: ./tools/ny_stage3_bridge_accept_smoke.sh
+
+      - name: Curated LLVM Stage-3 smokes (PHI-off)
+        run: ./tools/smokes/curated_llvm_stage3.sh --phi-off
+
--- a/AGENTS.md
+++ b/AGENTS.md
@ -60,6 +60,7 @@ Selfhost 子プロセスの引数透過（開発者向け）
 - 親→子にスクリプト引数を渡す環境変数:
  - `NYASH_NY_COMPILER_MIN_JSON=1` → 子に `-- --min-json`
  - `NYASH_SELFHOST_READ_TMP=1`    → 子に `-- --read-tmp`（`tmp/ny_parser_input.ny` を FileBox で読み込む。CIでは未使用）
+  - `NYASH_NY_COMPILER_STAGE3=1`   → 子に `-- --stage3`（Stage‑3 構文受理: Break/Continue/Throw/Try）
  - `NYASH_NY_COMPILER_CHILD_ARGS` → スペース区切りで子にそのまま渡す
 - 子側（apps/selfhost-compiler/compiler.nyash）は `--read-tmp` を受理して `tmp/ny_parser_input.ny` を読む（plugins 必要）。

--- a/CURRENT_TASK.md
+++ b/CURRENT_TASK.md
@ -14,8 +14,15 @@ What Changed (today)
 - フィールドは box 先頭のみルールのリンタを Runner に追加（`NYASH_FIELDS_TOP_STRICT=1` でエラー）。
 - Syntax Torture スイートの実行正規化（末行比較）。一部テスト本文を Nyash 仕様に合わせて修正。
 - JSON v0 仕様に Stage‑3 ノード（Break/Continue/Throw/Try）を追記。Parser Stage‑3 設計メモの現状/残課題を更新。
- LLVM smoke に Stage‑3 loop サンプル（break/continue + throw/try/catch/finally 付き）を追加（`NYASH_LLVM_STAGE3_SMOKE=1`）。
- Bridge (`json_v0`) に Stage‑3 throw/try の実稼働ルートを追加（`NYASH_BRIDGE_THROW_ENABLE=1` / `NYASH_BRIDGE_TRY_ENABLE=1` で MIR Throw/Catch を生成）。
+- LLVM curated smokes を新設（`tools/smokes/curated_llvm.sh`）。core/async/loop/peek を10s/ケースで実行、PHI‑off も検証可（`--phi-off`）。
+- LLVM Stage‑3 受理スモークを追加（`tools/smokes/curated_llvm_stage3.sh`）。try/finally・dead throw を10s/ケースで実行。PHI‑off（`--phi-off`）/trap抑止（`NYASH_LLVM_TRAP_ON_THROW=0`）も確認。
+- 旧スモークは `tools/smokes/archive/` へ整理（JIT/Cranelift 系は当面対象外）。
+- Bridge/Builder に PHI 非生成モードを導入（`NYASH_MIR_NO_PHI=1`）。LLVM Resolver 合成と統一し、Verifier は緩和ゲート（`verify_allow_no_phi()`）を追加。
+- LLVM 側に PHI 合成トレースを追加（`NYASH_LLVM_TRACE_PHI=1`）。jump/finalize/resolve の観測を統一タグで出力。
+- LLVM Throw を最小降ろし（`llvm.trap`→`unreachable`、`NYASH_LLVM_TRAP_ON_THROW=0` で trap 抑止）。
+- 環境変数アクセスを `config::env` に集約（`mir_no_phi()`/`verify_allow_no_phi()`/`llvm_use_harness()`）。
+- dev プロファイル `tools/dev_env.sh phi_off` を追加。ルート清掃ユーティリティ `tools/clean_root_artifacts.sh` を追加。
+- CI（GH Actions）を curated LLVM（PHI‑on/off）実行に刷新。旧JITジョブは停止。

 Decision (Phase‑15 wrap‑up)
 - MIR13 移行（PHI 非生成）: Phase‑15 の締めとして、MIR 生成層（Bridge/Builder）は PHI を生成しない方針に切替。PHI 合成は LLVM 層（llvmlite/Resolver）に集約。
@ -26,11 +33,16 @@ Next Focus (Throw/Try — LLVM first)
 - ブリッジ設計: `emit_degraded_throw` の差し替え方針を策定し、JSON v0 `Try` ノード → MIR 変換の仕様を決める（Stage-3 例外モデル）。
 - MIR Builder/Runtime 調査: Rust VM/PyVM の `ControlFlow::Throw` 経路と既存 TryCatch 降格の挙動を整理。必要に応じて docs と CURRENT_TASK に反映。
 - PyVM 設計: 例外モデルをどこまで Python 側に実装するか決め、最小テスト計画を用意。
- LLVM 実装方針: Throw/Try の MIR 命令を LLVM 側がどう扱うか（panic扱い or fallback）を設計し、smoke 更新案を作る。
+- LLVM 実装方針: Throw/Try の MIR 命令を LLVM 側がどう扱うか（panic扱い or fallback）を設計し、smoke 更新案を作る（現状 Throw は trap/unreachable 最小降ろし完了）。
 - テスト計画: JSON フィクスチャと `tools/llvm_smoke.sh` を中心に Stage-3 例外用のスモーク/単体テストを整備。

 ※ Cranelift/JIT 系は当面対象外。ビルド時も LLVM のみを有効化（JIT 関連 feature/CI は無視）。

+Runner updates (2025‑09‑16)
+- Selfhost pipeline: PyVM 優先（`NYASH_VM_USE_PY=1`）を全分岐で適用（EXE/inline/child 経路の一貫性）。
+- 重複関数の整理: `modes/common.rs::try_run_ny_compiler_pipeline` は `selfhost.rs::try_run_selfhost_pipeline` に委譲（ドリフト防止）。
+- Stage‑3 受理導線: `NYASH_NY_COMPILER_STAGE3=1` で子プロセスに `--stage3` を付与。inline フォールバックは `stage3_enable(1)` を既定有効化。
+
 - llvmlite/AOT（本戦）強化 — コアコレクション配線とエントリ統一
  - Array/Map の BoxCall を NyRT ハンドルAPIに直結：
    - Array: `push`→`nyash.array.push_h`、`length/len`→`nyash.any.length_h`
@ -170,6 +182,18 @@ Smoke Policy (Phase‑15)
 - PyVM: 一部チェックのみ（async/nowait/await/GC/sync は対象外）
 - LLVM: フル対応（llvmlite harness）。`tools/smokes/curated_llvm.sh [--phi-off]` を利用
 - JIT: 未整備（JIT向けスモークは `tools/smokes/archive/` に移管）
+ - Stage‑3 acceptance（Bridge 経路）: `tools/ny_stage3_bridge_accept_smoke.sh`（Try/Break/Continue/Throw を JSON v0 で受理できることを確認）
+
+Next Phase — Selfhost Parser/Compiler in Nyash（着手準備完了）
+- 目的: Nyash スクリプトで Parser/Emitter を実装し、Ny → JSON v0 → Bridge → MIR 実行の自己ホスト路線に移行。
+- ステップ（最小 MVP）:
+  1) `apps/selfhost-compiler/` に ParserBox/EmitterBox を Nyash で実装（Stage‑2 構文、JSON v0 出力）。
+  2) ランナーに `NYASH_USE_NY_COMPILER=1` ゲートを追加し、子プロセス/pipe で JSON v0 を受け取って Bridge→MIR 実行。
+  3) curated LLVM スモークの一部を自己ホスト経路で通す（PyVMは非対象、LLVMで検証）。
+  4) CI に自己ホスト最小ジョブを追加（timeout/静音運用、PHI‑on 既定）。
+- ガード/ポリシー:
+  - 既存 Rust Parser/Emitter はフォールバックとして保持（`NYASH_SKIP_TOML_ENV=1` で隔離可能）。
+  - 仕様差が出た場合は LLVM 側の意味論に合わせて Nyash 実装を調整。

 MIR13 Plan（Phase‑15 終盤）
 - Bridge/Builder: PHI を生成しない（受理は維持）。If/Loop の合流は LLVM Resolver に任せる。
--- a/apps/selfhost-compiler/README.md
+++ b/apps/selfhost-compiler/README.md
@ -12,6 +12,7 @@ Run (behind flag)
 - `NYASH_USE_NY_COMPILER=1 target/release/nyash --backend vm <program.nyash>`
  - The runner writes the input to `tmp/ny_parser_input.ny` and invokes this program.
  - It captures a JSON v0 line from stdout and executes it via the JSON bridge.
+  - Stage‑3 syntax gate: set `NYASH_NY_COMPILER_STAGE3=1` to pass `--stage3` to the parser (accepts Break/Continue/Throw/Try in JSON v0).

 Notes
 - Early MVP emits a minimal JSON v0 (currently a placeholder: return 0). We will gradually wire lexer/parser/emitter.
--- a/apps/tests/stage3_throw_dead_branch.nyash
+++ b/apps/tests/stage3_throw_dead_branch.nyash
@ -0,0 +1,5 @@
+if false {
+  throw 1
+}
+return 0
+
--- a/apps/tests/stage3_try_finally_basic.nyash
+++ b/apps/tests/stage3_try_finally_basic.nyash
@ -0,0 +1,9 @@
+try {
+  local x = 1
+} catch (Error e) {
+  local y = 2
+} finally {
+  local z = 3
+}
+return 0
+
--- a/src/runner/modes/common.rs
+++ b/src/runner/modes/common.rs
@ -131,6 +131,8 @@ impl NyashRunner {

    /// Phase-15.3: Attempt Ny compiler pipeline (Ny -> JSON v0 via Ny program), then execute MIR
    pub(crate) fn try_run_ny_compiler_pipeline(&self, filename: &str) -> bool {
+        // Delegate to centralized selfhost pipeline to avoid drift
+        return self.try_run_selfhost_pipeline(filename);
        use std::io::Write;
        // Read input source
        let code = match fs::read_to_string(filename) {
@ -274,11 +276,9 @@ impl NyashRunner {
                        if emit_only {
                            return false;
                        } else {
-                            // Prefer PyVM when requested AND the module contains BoxCalls (Stage-2 semantics)
-                            let needs_pyvm = module.functions.values().any(|f| {
-                                f.blocks.values().any(|bb| bb.instructions.iter().any(|inst| matches!(inst, crate::mir::MirInstruction::BoxCall { .. })))
-                            });
-                            if needs_pyvm && std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1") {
+                            // Prefer PyVM when requested (reference semantics), regardless of BoxCall presence
+                            let prefer_pyvm = std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1");
+                            if prefer_pyvm {
                                if let Ok(py3) = which::which("python3") {
                                    let runner = std::path::Path::new("tools/pyvm_runner.py");
                                    if runner.exists() {
@ -476,11 +476,9 @@ impl NyashRunner {
                    // Do not execute; fall back to default path to keep final Result unaffected (Stage‑1 policy)
                    false
                } else {
-                    // Prefer PyVM when requested AND the module contains BoxCalls
-                    let needs_pyvm = module.functions.values().any(|f| {
-                        f.blocks.values().any(|bb| bb.instructions.iter().any(|inst| matches!(inst, crate::mir::MirInstruction::BoxCall { .. })))
-                    });
-                    if needs_pyvm && std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1") {
+                    // Prefer PyVM when requested (reference semantics)
+                    let prefer_pyvm = std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1");
+                    if prefer_pyvm {
                        if let Ok(py3) = which::which("python3") {
                            let runner = std::path::Path::new("tools/pyvm_runner.py");
                            if runner.exists() {
--- a/src/runner/selfhost.rs
+++ b/src/runner/selfhost.rs
@ -165,6 +165,7 @@ impl NyashRunner {
                // Gates
                if std::env::var("NYASH_NY_COMPILER_MIN_JSON").ok().as_deref() == Some("1") { cmd.arg("--min-json"); }
                if std::env::var("NYASH_SELFHOST_READ_TMP").ok().as_deref() == Some("1") { cmd.arg("--read-tmp"); }
+                if std::env::var("NYASH_NY_COMPILER_STAGE3").ok().as_deref() == Some("1") { cmd.arg("--stage3"); }
                if let Ok(raw) = std::env::var("NYASH_NY_COMPILER_CHILD_ARGS") { for tok in raw.split_whitespace() { cmd.arg(tok); } }
                let timeout_ms: u64 = std::env::var("NYASH_NY_COMPILER_TIMEOUT_MS").ok().and_then(|s| s.parse().ok()).unwrap_or(2000);
                let mut cmd = cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
@ -212,11 +213,9 @@ impl NyashRunner {
                        if emit_only {
                            return false;
                        } else {
-                            // Prefer PyVM when requested AND the module contains BoxCalls (Stage-2 semantics)
-                            let needs_pyvm = module.functions.values().any(|f| {
-                                f.blocks.values().any(|bb| bb.instructions.iter().any(|inst| matches!(inst, crate::mir::MirInstruction::BoxCall { .. })))
-                            });
-                            if needs_pyvm && std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1") {
+                            // Prefer PyVM when requested (reference semantics), regardless of BoxCall presence
+                            let prefer_pyvm = std::env::var("NYASH_VM_USE_PY").ok().as_deref() == Some("1");
+                            if prefer_pyvm {
                                if let Ok(py3) = which::which("python3") {
                                    let runner = std::path::Path::new("tools/pyvm_runner.py");
                                    if runner.exists() {
@ -272,7 +271,7 @@ impl NyashRunner {
            }
            let inline_path = std::path::Path::new("tmp").join("inline_selfhost_emit.nyash");
            let inline_code = format!(
-                "include \"apps/selfhost-compiler/boxes/parser_box.nyash\"\ninclude \"apps/selfhost-compiler/boxes/emitter_box.nyash\"\nstatic box Main {{\n  main(args) {{\n    local s = \"{}\"\n    local p = new ParserBox()\n    local json = p.parse_program2(s)\n    local e = new EmitterBox()\n    json = e.emit_program(json, \"[]\")\n    print(json)\n    return 0\n  }}\n}}\n",
+                "include \"apps/selfhost-compiler/boxes/parser_box.nyash\"\ninclude \"apps/selfhost-compiler/boxes/emitter_box.nyash\"\nstatic box Main {{\n  main(args) {{\n    local s = \"{}\"\n    local p = new ParserBox()\n    p.stage3_enable(1)\n    local json = p.parse_program2(s)\n    local e = new EmitterBox()\n    json = e.emit_program(json, \"[]\")\n    print(json)\n    return 0\n  }}\n}}\n",
                esc
            );
            if let Err(e) = std::fs::write(&inline_path, inline_code) {
--- a/tools/ny_stage3_bridge_accept_smoke.sh
+++ b/tools/ny_stage3_bridge_accept_smoke.sh
@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+set -euo pipefail
+[[ "${NYASH_CLI_VERBOSE:-0}" == "1" ]] && set -x
+
+ROOT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
+BIN="$ROOT_DIR/target/release/nyash"
+
+if [[ ! -x "$BIN" ]]; then
+  (cd "$ROOT_DIR" && cargo build --release >/dev/null)
+fi
+
+pass() { echo "✅ $1" >&2; }
+fail() { echo "❌ $1" >&2; echo "$2" >&2; exit 1; }
+
+run_json_expect_code() {
+  local name="$1"; shift
+  local json="$1"; shift
+  local expect_code="$1"; shift
+  set +e
+  # Feed JSON v0 directly via pipe. Prefer PyVM for parity.
+  OUT=$(printf '%s\n' "$json" | NYASH_PIPE_USE_PYVM=${NYASH_PIPE_USE_PYVM:-1} "$BIN" --ny-parser-pipe --backend vm 2>&1)
+  CODE=$?
+  set -e
+  if [[ "$CODE" == "$expect_code" ]]; then pass "$name"; else fail "$name" "$OUT"; fi
+}
+
+# A) try/catch/finally acceptance (degrade path unless NYASH_BRIDGE_TRY_ENABLE=1); final return 0
+JSON_A='{"version":0,"kind":"Program","body":[
+  {"type":"Try","try":[{"type":"Local","name":"x","expr":{"type":"Int","value":1}}],
+   "catches":[{"param":"e","typeHint":"Error","body":[{"type":"Local","name":"y","expr":{"type":"Int","value":2}}]}],
+   "finally":[{"type":"Local","name":"z","expr":{"type":"Int","value":3}}]
+  },
+  {"type":"Return","expr":{"type":"Int","value":0}}
+]}'
+run_json_expect_code "try/catch/finally (accept)" "$JSON_A" 0
+
+# B) break acceptance under dead branch (ignored when not in loop)
+JSON_B='{"version":0,"kind":"Program","body":[
+  {"type":"If","cond":{"type":"Bool","value":false},"then":[{"type":"Break"}]},
+  {"type":"Return","expr":{"type":"Int","value":0}}
+]}'
+run_json_expect_code "break in dead branch (accept)" "$JSON_B" 0
+
+# C) continue acceptance under dead branch (ignored when not in loop)
+JSON_C='{"version":0,"kind":"Program","body":[
+  {"type":"If","cond":{"type":"Bool","value":false},"then":[{"type":"Continue"}]},
+  {"type":"Return","expr":{"type":"Int","value":0}}
+]}'
+run_json_expect_code "continue in dead branch (accept)" "$JSON_C" 0
+
+# D) throw acceptance as expression (degrade path unless NYASH_BRIDGE_THROW_ENABLE=1)
+JSON_D='{"version":0,"kind":"Program","body":[
+  {"type":"Expr","expr":{"type":"Throw","expr":{"type":"Int","value":123}}},
+  {"type":"Return","expr":{"type":"Int","value":0}}
+]}'
+run_json_expect_code "throw (accept)" "$JSON_D" 0
+
+echo "All Stage-3 bridge acceptance smokes PASS" >&2
+exit 0
+
--- a/tools/selfhost_stage2_smoke.sh
+++ b/tools/selfhost_stage2_smoke.sh
@ -12,6 +12,10 @@ fi
 TMP="$ROOT_DIR/tmp"
 mkdir -p "$TMP"

+# Default to PyVM reference unless explicitly disabled by caller
+: "${NYASH_VM_USE_PY:=1}"
+export NYASH_VM_USE_PY
+
 pass() { echo "✅ $1" >&2; }
 fail() { echo "❌ $1" >&2; echo "$2" >&2; exit 1; }

--- a/tools/smokes/curated_llvm_stage3.sh
+++ b/tools/smokes/curated_llvm_stage3.sh
@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Curated LLVM Stage-3 acceptance smoke (llvmlite harness)
+# Usage: tools/smokes/curated_llvm_stage3.sh [--phi-off]
+
+ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
+BIN="$ROOT_DIR/target/release/nyash"
+
+if ! [ -x "$BIN" ]; then
+  echo "[curated-llvm-stage3] building nyash (release, features=llvm)" >&2
+  cargo build --release --features llvm >/dev/null
+fi
+
+export NYASH_LLVM_USE_HARNESS=1
+
+if [[ "${1:-}" == "--phi-off" ]]; then
+  export NYASH_MIR_NO_PHI=1
+  export NYASH_VERIFY_ALLOW_NO_PHI=1
+  echo "[curated-llvm-stage3] PHI-off (edge-copy) enabled" >&2
+fi
+
+run_ok() {
+  local path="$1"
+  echo "[curated-llvm-stage3] RUN --backend llvm: ${path}"
+  timeout 10s "$BIN" --backend llvm "$path" >/dev/null
+}
+
+# A) try/catch/finally without actual throw
+run_ok "$ROOT_DIR/apps/tests/stage3_try_finally_basic.nyash"
+
+# B) throw in dead branch (acceptance only)
+run_ok "$ROOT_DIR/apps/tests/stage3_throw_dead_branch.nyash"
+
+# C) repeat with trap disabled (robustness; should be no-op here)
+NYASH_LLVM_TRAP_ON_THROW=0 run_ok "$ROOT_DIR/apps/tests/stage3_try_finally_basic.nyash"
+NYASH_LLVM_TRAP_ON_THROW=0 run_ok "$ROOT_DIR/apps/tests/stage3_throw_dead_branch.nyash"
+
+echo "[curated-llvm-stage3] OK"
+