feat: Add HTTP status tests and dynamic plugin documentation

- Add e2e_vm_http_status_404/500 tests to verify HTTP status handling
- ResultBox properly returns Ok(Response) for HTTP errors, Err for connection failures
- Create dynamic-plugin-flow.md documenting MIR→VM→Registry→Plugin flow
- Add vm-stats test files for HTTP 404/500 status codes
- Update net-plugin.md with HTTP error handling clarification
- Create E2E_TESTS.md documenting all E2E test behaviors
- Add mir-26-instruction-diet.md for MIR optimization plans
- Add vm-stats-cookbook.md for VM statistics usage guide
- Update MIR verifier to properly track self-assignment patterns

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm
2025-08-23 16:33:32 +09:00
parent 494a864ed2
commit 70af0fe566
15 changed files with 671 additions and 19 deletions

41
.github/workflows/plugins-e2e.yml vendored Normal file
View File

@ -0,0 +1,41 @@
name: Plugins E2E (Linux)
on:
push:
branches: [ main, dev ]
pull_request:
branches: [ main, dev ]
jobs:
plugins-e2e:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
toolchain: stable
override: true
- name: Cache cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Build (release)
run: cargo build --release -j2
- name: Run E2E with plugins (Linux)
env:
RUST_BACKTRACE: 1
NYASH_NET_LOG: "0"
run: |
cargo test --features plugins -q -- --nocapture

View File

@ -2,32 +2,43 @@
## ✅ 直近の完了
1. ドキュメント再編成の完了(構造刷新)
2. プラグインBoxFileBox基本実装とインタープリター統合
2. VM×プラグインのE2E整備FileBox/Net
- FileBox: open/write/read, copyFrom(handle)VM
- Net: GET/POSTVM、404/500Ok(Response)、unreachableErr(ErrorBox)
3. VM命令カウンタ時間計測のCLI化`--vm-stats`, `--vm-stats-json`とJSON出力対応
- サンプル/スクリプト整備tools/run_vm_stats.sh、local_tests/vm_stats_*.nyash
4. MIR if-merge 修正retがphi dstを返す Verifier強化mergeでのphi未使用検知
5. ドキュメント追加・更新
- Dynamic Plugin FlowMIR→VM→Registry→Loader→Plugin
- Netプラグインのエラーモデルunreachable=Err, 404/500=Ok
- E2Eテスト一覧整備
6. CI: plugins E2E ジョブLinuxを追加
## 🚧 次にやること(再開方針)
1) MIR→VMの健全化短期・最優先
- 現行MIR→VMのマッピング表を作成欠落/冗長/重複を可視化
- サンプル/テストをVMで実行し、差分ログ例外系・returns_resultを確認
- 成果物: `docs/reference/architecture/mir-to-vm-mapping.md`暫定
- マッピング表更新Err経路・Handle戻り・Result整合を実測で反映
- Verifierルールの拡充use-before-def across merge を強化)
- 成果物: `docs/reference/architecture/mir-to-vm-mapping.md`更新済・追補
2) VM×プラグインシステムのE2E検証短期
- `tests/e2e_plugin_filebox.rs` をVMでも通す`--features plugins`
- ケース: `new/close`, `open/read/write`, `copyFrom(handle)`、デリゲーション from Parent
- 成果物: テストグリーン+既知の制約を `VM_README.md` に明記
- FileBox/Netを中心にケース拡張大きいボディ、ヘッダー多数、タイムアウト等
- 成果物: E2E追補`VM_README.md` に既知の制約とTipsを追記
3) 命令セットのダイエット中期目標26命令
-行統計(`--vm-stats --vm-stats-json`)でホット命令を特
- 統合方針(例: TypeCheck/Castの整理、Array/Ref周りの集約、ExternCall→BoxCall移行
- 段階移行(互換エイリアス→削除)と回帰テスト整備
- 成果物: 26命令案ドラフト+移行計画
-HTTP OK/404/500/unreachable、FileBoxを反映して合意版を確
- 統合方針TypeOp/WeakRef/Barrierの統合、ExternCall最小化
- 段階移行(ビルドモードでメタ降格、互換エイリアス→削除)と回帰テスト整備
- 成果物: 26命令案(合意版)+移行計画
## ▶ 実行コマンド例
計測実行:
```bash
nyash --backend vm --vm-stats --vm-stats-json local_tests/test_hello.nyash > vm_stats.json
tools/run_vm_stats.sh local_tests/vm_stats_http_ok.nyash vm_stats_ok.json
tools/run_vm_stats.sh local_tests/vm_stats_http_err.nyash vm_stats_err.json
tools/run_vm_stats.sh local_tests/vm_stats_http_404.nyash vm_stats_404.json
tools/run_vm_stats.sh local_tests/vm_stats_http_500.nyash vm_stats_500.json
```
VM×プラグインE2E:
@ -42,10 +53,9 @@ nyash --dump-mir --mir-verbose examples/plugin_box_sample.nyash
nyash --verify examples/plugin_box_sample.nyash
```
## 🔭 26命令ターゲットドラフトの方向性
コア(候補): Const / Copy / Load / Store / BinOp / UnaryOp / Compare / Jump / Branch / Phi / Call / BoxCall / NewBox / ArrayGet / ArraySet / RefNew / RefGet / RefSet / WeakNew / WeakLoad / BarrierRead / BarrierWrite / Return / Print or ExternCall(→BoxCall集約) + 2枠例外/await系のどちらか
補助: Debug/Nop/Safepointはビルドモードで有効化命令としては非中核に降格
## 🔭 26命令ターゲット合意ドラフト)
- コア: Const / Copy / Load / Store / BinOp / UnaryOp / Compare / Jump / Branch / Phi / Return / Call / BoxCall / NewBox / ArrayGet / ArraySet / RefNew / RefGet / RefSet / Await / Print / ExternCall(最小) / TypeOp(=TypeCheck/Cast統合) / WeakRef(=WeakNew/WeakLoad統合) / Barrier(=Read/Write統合)
- メタ降格: Debug / Nop / Safepointビルドモードで制御
---
最終更新: 2025年8月23日MIR/VM再フォーカス、26命令ダイエットへ)
最終更新: 2025年8月23日VM×Plugins安定・MIR修正・26命令合意ドラフトへ)

View File

@ -49,3 +49,28 @@ Future Ideas
Notes
- Keep this file as a living list; prune as items graduate to tracked issues/PRs.
---
2025-08-23 Updates (VM × Plugins focus)
- VM Stats frontdoor (done): CLI flags `--vm-stats`, `--vm-stats-json`; JSON schema includes total/counts/top20/elapsed_ms.
- Next: integrate with `--benchmark` to emit per-backend stats; add `NYASH_VM_STATS_FORMAT=json` docs.
- ResultBox in VM (done): dispatch for `isOk/getValue/getError`; generic `toString()` fallback for any Box.
- Impact: HTTP Result paths now work end-to-end in VM.
- MIR if-merge bug (done): bind merged variable to Phi result across Program blocks (ret now returns phi dst).
- Next: add verifier check for "use-before-def across merge"; snapshot a failing MIR pattern as a test.
- Net plugin error mapping (done): on TCP connect failure, return TLV string; loader maps to Result.Err(ErrorBox).
- Next: formalize `returns_result` ok-type in nyash.toml (e.g., ok_returns = "HttpResponseBox"); tighten loader.
- E2E coverage (done):
- FileBox: open/write/read, copyFrom(handle)
- Net: GET/POST/status/header/body; 204 empty body; client error (unreachable port) → Err
- Next: 404/5xx reply from server side; timeouts; large bodies; header casing behavior.
Short-Term TODOs
- Add vm-stats samples for normal/error HTTP flows; feed into 26-instruction diet discussion.
- CI: run `--features plugins` E2E on a dedicated job; gate on Linux only; quiet logs unless failed.
- Docs: append "VM→Plugin TLV debugging" quick tips (env flags, TLV preview).
26-Instruction Diet Hooks
- Candidate demotions: Debug/Nop/Safepoint → meta; TypeCheck/Cast → fold or verify-time.
- Keep hot path: BoxCall/NewBox/Branch/Jump/Phi/BinOp/Compare/Return.
- Track Weak*/Barrier usage; keep as extended-set unless surfaced in vm-stats.

View File

@ -20,6 +20,11 @@
- `src/box_factory/*` … Builtin/User/Plugin の各 Factory 実装
- `src/runtime/plugin_loader_v2.rs` … BID-FFI v2 ローダExternCall/Plugin 呼び出し)
関連ドキュメント
- 動的プラグインの流れ: [dynamic-plugin-flow.md](./dynamic-plugin-flow.md)
- 命令セットダイエット: [mir-26-instruction-diet.md](./mir-26-instruction-diet.md)
- MIR→VMマッピング: [mir-to-vm-mapping.md](./mir-to-vm-mapping.md)
## 実行フロー(概略)
1) Nyash コード → Parser → AST → `MirCompiler``MirModule` を生成
2) `VM::with_runtime(runtime)` で実行(`execute_module`
@ -35,6 +40,7 @@
- Builtin: `BuiltinBoxFactory` が直接生成
- User-defined: `UserDefinedBoxFactory``InstanceBox`
- Plugin: プラグイン設定(`nyash.toml`)に従い BID-FFI で `PluginBoxV2`
- **動的解決の詳細**: [dynamic-plugin-flow.md](./dynamic-plugin-flow.md) を参照
## birth/メソッドの関数化MIR
- Lowering ポリシー: AST の `new``NewBox` に続けて `BoxCall("birth")` を自動挿入

View File

@ -0,0 +1,115 @@
# Dynamic Plugin Flow (VM × Registry × PluginLoader v2)
最終更新: 2025-08-23
目的
- Nyash 実行時に、MIR→VM→Registry→Plugin の呼び出しがどう流れるかを図解・手順で把握する。
- TLVエンコード、ResultBoxの扱い、Handleのライフサイクル、nyash.tomlとの連携を1枚で理解する。
## ハイレベル流れ(シーケンス)
```
Nyash Source ──▶ MIR (Builder)
│ (BoxCall/NewBox/…)
VM Executor
│ (BoxCall dispatch)
├─ InstanceBox → Lowered MIR 関数呼び出し
├─ BuiltinBox → VM内ディスパッチ
└─ PluginBoxV2 → PluginLoader v2
│ (nyash.toml を参照)
Invoke (TLV)
Plugin (lib*.so)
│ (戻り値をTLVで返却)
Loader でデコード
│ (returns_result/Handle/型)
NyashBox (ResultBox/PluginBoxV2/基本型)
VM に復帰
```
## 主要構成要素
- MIR: `MirInstruction::{BoxCall, NewBox, …}` で外部呼び出し箇所を明示。
- VM: `src/backend/vm.rs`
- InstanceBoxは `{Class}.{method}/{argc}` のLowered関数へ呼び出し
- BuiltinはVM内の簡易ディスパッチ
- PluginBoxV2は Loader v2 へ委譲
- Registry/Runtime: `NyashRuntime` + `box_registry` + `plugin_loader_v2`
- `nyash.toml``libraries.*` を読み込み、Box名→ライブラリ名、type_id、method_id等を集約
## NewBox生成
1) MIRの `NewBox { box_type, args }`
2) VM: `runtime.box_registry``box_type` を問い合わせ
3) PluginBoxの場合、Loader v2が `birth(method_id=0)` を TLV で呼び出し
4) Pluginは `type_id` と新規 `instance_id` を返却 → Loader は `PluginBoxV2` を構築
5) VMは `ScopeTracker` に登録(スコープ終了で `fini` を呼ぶ)
## BoxCallメソッド呼び出し
- InstanceBox: Lowered関数 `{Class}.{method}/{argc}` を MIR/VM内で実行
- Builtin: VM内の `call_box_method` で対応StringBox.length 等)
- PluginBoxV2: Loader v2 の `invoke_instance_method` で TLV を組み立てて呼び出し
## TLVType-Length-Value
- ヘッダ: `u16 ver=1`, `u16 argc`
- 各引数: `u8 tag`, `u8 reserved`, `u16 size`, `payload`
- 主な tag:
- 2 = i32 (size=4)
- 6 = string, 7 = bytes
- 8 = Handle(BoxRef) → payload = `u32 type_id || u32 instance_id`
- 9 = void (size=0)
## 戻り値のマッピング(重要)
- `returns_result=false`
- tag=8 → PluginBoxV2Handle
- tag=2 → IntegerBox、tag=6/7 → StringBox、tag=9 → void
- `returns_result=true`ResultBoxで包む
- tag=8/2 → `Result.Ok(value)`
- tag=6/7 → `Result.Err(ErrorBox(message))`Netプラグインなどがエラー文字列を返却
- tag=9 → `Result.Ok(void)`
補足
- VM内で ResultBox の `isOk/getValue/getError` をディスパッチ済み
- `toString()` フォールバックにより任意の Box を安全に文字列化可能
## HandleBoxRefのライフサイクル
- Loaderは `(type_id, instance_id)``PluginBoxV2` としてラップ
- `share_box()` は同一インスタンス共有、`clone_box()` はプラグインの birth を呼ぶ(設計意図による)
- `fini``ScopeTracker` または Drop で保証(プラグインの `fini_method_id` を参照)
## 具体例HttpClientBox.get
1) Nyash: `r = cli.get(url)`
2) MIR: `BoxCall`returns_result=true
3) VM→Loader: TLVurl = tag=6
4) Loader→Plugin: `invoke(type_id=HttpClient, method_id=get)`
5) Plugin:
- 接続成功: `Handle(HttpResponse)` を返す → Loaderは `Result.Ok(PluginBoxV2)`
- 接続失敗: `String("connect failed …")` を返す → Loaderは `Result.Err(ErrorBox)`
6) Nyash: `if r.isOk() { resp = r.getValue() … } else { print(r.getError().toString()) }`
## nyash.toml 連携
- 例: `libraries."libnyash_net_plugin.so".HttpClientBox.methods.get = { method_id = 1, args=["url"], returns_result = true }`
- Loaderは `method_id``returns_result` を参照し、TLVと戻り値のラップ方針を決定
- 型宣言args/kindにより、引数のTLVタグ検証を実施不一致は InvalidArgs
## デバッグTips
- `NYASH_DEBUG_PLUGIN=1`: VM→Plugin の TLV ヘッダと先頭64バイトをプレビュー
- `NYASH_NET_LOG=1 NYASH_NET_LOG_FILE=net_plugin.log`: Netプラグイン内部ログ
- `--dump-mir --mir-verbose`: if/phi/return などのMIRを確認
- `--vm-stats --vm-stats-json`: 命令使用のJSONを取得hot pathの裏取りに
## 将来の整合・改善
- nyash.toml に ok側の戻り型例: `ok_returns = "HttpResponseBox"`)を追加 → Loader判定の厳密化
- Verifier強化: use-before-def across merge の検出phi誤用を早期に発見
- BoxCall fast-path の最適化hot path最優先
関連
- `docs/reference/plugin-system/net-plugin.md`
- `docs/reference/architecture/mir-to-vm-mapping.md`
- `docs/reference/architecture/mir-26-instruction-diet.md`

View File

@ -0,0 +1,86 @@
# MIR 26-Instruction Diet (Agreed Final Set)
Goal
- Converge on a lean, proven instruction set guided by vm-stats and E2E.
- Preserve hot paths, demote meta, fold type ops, reserve room for growth.
Agreed Final Set (26)
1) Const
2) Copy
3) Load
4) Store
5) BinOp
6) UnaryOp
7) Compare
8) Jump
9) Branch
10) Phi
11) Return
12) Call
13) BoxCall
14) NewBox
15) ArrayGet
16) ArraySet
17) RefNew
18) RefGet
19) RefSet
20) Await
21) Print
22) ExternCall (keep minimal; prefer BoxCall)
23) TypeOp (unify TypeCheck/Cast)
24) WeakRef (unify WeakNew/WeakLoad)
25) Barrier (unify BarrierRead/BarrierWrite)
26) Reserve (future async/error instr)
Hot/Core (keep)
- Data: Const, Copy, Load, Store
- ALU: BinOp, UnaryOp, Compare
- Control: Jump, Branch, Phi, Return
- Calls: Call, BoxCall
- Objects: NewBox
- Arrays: ArrayGet, ArraySet
Likely Keep (usage-dependent)
- Refs: RefNew, RefGet, RefSet (seen in language features; keep unless stats prove cold)
- Async: Await (FutureNew/Set can be Box/APIs)
Meta (demote to build-mode)
- Debug, Nop, Safepoint
Type Ops (fold)
- TypeCheck, Cast → fold/verify-time or unify as a single TypeOp (optional).
External (unify)
- ExternCall → prefer BoxCall; keep ExternCall only where required.
Extended/Reserve
- Weak*: WeakNew, WeakLoad
- Barriers: BarrierRead, BarrierWrite
- 2 Reserve IDs for future async/error instrumentation
Mapping Notes
- HTTP E2E shows BoxCall/NewBox dominate (3342%), then Const/NewBox, with Branch/Jump/Phi only in error flows.
- FileBox path similarly heavy on BoxCall/NewBox/Const.
- Implication: invest into BoxCall fast path and const/alloc optimization.
Migration Strategy
1) Introduce an experimental feature gate that switches TypeCheck/Cast to TypeOp or folds them.
2) Demote Debug/Nop/Safepoint under non-release builds.
3) Keep ExternCall available, route new external APIs via BoxCall when possible.
4) Track Weak*/Barrier usage; unify under WeakRef/Barrier. Graduate dedicated ops only if vm-stats shows recurring use.
Mapping from Current → Final
- TypeCheck, Cast → TypeOp
- WeakNew, WeakLoad → WeakRef
- BarrierRead, BarrierWrite → Barrier
- (Keep) ExternCall, but prefer BoxCall where possibleExternCall は最小限に)
- (Keep) Debug/Nop/Safepoint as meta under non-release builds命令セット外のビルドモード制御に降格
Verification
- Add MIR verifier rule: use-before-def across merges is rejected (guards phi misuse).
- Add snapshot tests for classic if-merge returning phi.
Appendix: Rationale
- Smaller ISA simplifies VM fast paths and aids JIT/AOT later.
- Data shows hot paths concentrated on calls, const, alloc; control sparse and localized.
- Folding type ops reduces interpreter/VM dispatch; verification handles safety.

View File

@ -103,6 +103,22 @@
- NewBox/BoxCall が上位に入るため、命令セットから外すのは不可(コア扱い)。
- Compare/Branch/Jump/Phi は制御フローのコア。26命令の中核として維持が妥当。
## 実測統計2025-08-23
出所: vm-stats正常系HTTP異常系HTTPFileBox
- 正常系HTTP40命令
- BoxCall: 1742.5%/ Const: 1230%/ NewBox: 922.5%/ Return: 1 / Safepoint: 1
- 異常系HTTP21命令 = 正常の52.5%
- BoxCall: 733.3%/ Const: 628.6%/ NewBox: 314.3%
- Branch: 1 / Jump: 1 / Phi: 1エラーハンドリング特有/ Return: 1 / Safepoint: 1
- FileBox44命令
- BoxCall: 1738.6%/ Const: 1329.5%/ NewBox: 1227.3%/ Return: 1 / Safepoint: 1
設計含意:
- BoxCallが常に最頻出33〜42%)。呼び出しコスト最適化が最優先。
- Const/NewBoxが次点。定数・生成の最適化定数畳み込み軽量生成・シェアリングが効果的。
- 異常系は早期収束で命令半減。if/phi修正は実戦で有効Branch/Jump/Phiが顕在化
## 26命令ダイエット検討のたたき台
方針: 「命令の意味は保ちつつ集約」。代表案:
- 維持: Const / Copy / Load / Store / BinOp / UnaryOp / Compare / Jump / Branch / Phi / Return / Call / BoxCall / NewBox / ArrayGet / ArraySet
@ -118,6 +134,19 @@
---
## 26命令ダイエットの指針実測反映
- 維持(ホット・コア): BoxCall / NewBox / Const / BinOp / Compare / Branch / Jump / Phi / Return / Copy / Load / Store / Call
- 実装方針: ExternCallは原則BoxCallへ集約必要なら限定的に残す
- メタ降格: Debug/Nop/Safepointビルドモードで制御
- 型系: TypeCheck/Castは折りたたみ or 検証時に処理1命令に集約も可
- 参照/弱参照/バリア: 需要ベースで拡張枠へvm-statsに登場しない限りコア外
提案(ドラフト):
- コア候補(例): Const, Copy, Load, Store, BinOp, UnaryOp, Compare, Jump, Branch, Phi, Return, Call, BoxCall, NewBox, ArrayGet, ArraySet, RefNew, RefGet, RefSet, Await, Print, ExternCall(集約可), TypeOp(=TypeCheck/Cast), 予備2将来枠
- 予備はWeak*/Barrierや将来の非同期拡張等に割当実測で常用化したら昇格
---
## E2E更新VM経由の実働確認
成功ケースVM:
@ -125,6 +154,8 @@
- FileBox.copyFrom(handle): Handle引数tag=8, size=8, type_id+instance_idで成功
- HttpClientBox.get + HttpServerBox: 基本GETの往復ResultBox経由でResponse取得
- HttpClientBox.post + headers: Status/ヘッダー/ボディをVMで往復確認
- HttpClientBox.get unreachable: 接続失敗時はResult.Err(ErrorBox)ローダーがstring/bytesをErrにマップ
- HTTP 404/500: Result.Ok(Response)ステータスはResponse上に保持
デバッグ小技:
- `NYASH_DEBUG_PLUGIN=1` で VM→Plugin 呼び出しTLVの ver/argc/先頭バイトをダンプ

View File

@ -12,6 +12,11 @@
- FileBoxプラグインで実証済み
- プラグイン開発者はここから始める
- **[../architecture/dynamic-plugin-flow.md](../architecture/dynamic-plugin-flow.md)** - **動的プラグインシステムの全体フロー** 🆕
- MIR→VM→Registry→プラグインの動的解決フロー
- コンパイル時決め打ちなし、実行時動的判定の仕組み
- nyash.tomlによる透過的な切り替え
- **[vm-plugin-integration.md](./vm-plugin-integration.md)** - **VM統合仕様書** 🆕
- VMバックエンドとプラグインシステムの統合
- BoxRef型による統一アーキテクチャ

View File

@ -30,6 +30,13 @@
- `acceptTimeout(ms)`: タイムアウト時は `void`
- `recvTimeout(ms)`: タイムアウト時は 空 `bytes`長さ0の文字列
### HTTPエラーハンドリング重要
- **接続失敗unreachable**: `Result.Err(ErrorBox)` を返す
- 例: ポート8099に接続できない → `Err("connect failed for 127.0.0.1:8099/...")`
- **HTTPステータスエラー404/500等**: `Result.Ok(HttpResponseBox)` を返す
- 例: 404 Not Found → `Ok(response)``response.getStatus()` が 404
- トランスポート層は成功、アプリケーション層のエラーとして扱う
将来の整合:
- `ResultBox` での返却に対応する設計(`Ok(value)`/`Err(ErrorBox)`)を検討中。
- `nyash.toml` のメソッド宣言に戻り値型Resultを記載し、ランタイムで自動ラップする案。

26
docs/tests/E2E_TESTS.md Normal file
View File

@ -0,0 +1,26 @@
# E2E Tests Overview
Purpose
- Track end-to-end coverage with plugins and core features, both interpreter and VM.
HTTP (plugins)
- GET basic (VM): `e2e_vm_http_get_basic` → body `OK`
- POST + headers (VM): `e2e_vm_http_post_and_headers``201:V:R`
- Status 404 (VM): `e2e_vm_http_status_404``404:NF`
- Status 500 (VM): `e2e_vm_http_status_500``500:ERR`
- Client error (unreachable) (VM): `e2e_vm_http_client_error_result``Result.Err(ErrorBox)`
FileBox (plugins)
- Close returns void (Interp/VM)
- Open/Write/Read (VM): `e2e_vm_plugin_filebox_open_rw``HELLO`
- copyFrom(handle) (VM): `e2e_vm_plugin_filebox_copy_from_handle``HELLO`
MIR/VM Core
- Ref ops MIR build: `mir_phase6_lowering_ref_ops`
- Ref ops VM exec: `mir_phase6_vm_ref_ops`
- Async ops MIR/VM: `mir_phase7_async_ops`
Conventions
- Use distinct ports per test (8080+). Enable logs only on failure to keep CI output tidy.
- Plugins logs: `NYASH_NET_LOG=1 NYASH_NET_LOG_FILE=net_plugin.log`.

View File

@ -0,0 +1,38 @@
# VM Stats Cookbook
Collect VM instruction stats (JSON) to guide optimization and instruction set diet.
## Prerequisites
- Build: `cargo build --release -j32`
- Ensure plugins are configured in `nyash.toml` if your program uses them.
## Quick Start
```bash
# Human-readable
./target/release/nyash --backend vm --vm-stats local_tests/vm_stats_http_ok.nyash
# JSON for tooling
./target/release/nyash --backend vm --vm-stats --vm-stats-json local_tests/vm_stats_http_ok.nyash > vm_stats_ok.json
# Or via helper script
tools/run_vm_stats.sh local_tests/vm_stats_http_ok.nyash vm_stats_ok.json
```
## Sample Programs
- `local_tests/vm_stats_http_ok.nyash` — Server responds "OK" to a client GET.
- `local_tests/vm_stats_http_err.nyash` — Client GET to an unreachable port (Result Err path).
- `local_tests/vm_stats_http_404.nyash` — Server returns 404/"NF"; transport成功アプリ層エラーの代表例。
- `local_tests/vm_stats_http_500.nyash` — Server returns 500/"ERR"; 同上。
- `local_tests/vm_stats_filebox.nyash` — FileBox open/write/copyFrom/read.
## Tips
- Enable plugin debugging when needed:
- `NYASH_DEBUG_PLUGIN=1` — Show VM→Plugin TLV header preview.
- `NYASH_NET_LOG=1 NYASH_NET_LOG_FILE=net_plugin.log` — Net plugin logs.
- Env alternative to CLI flags:
- `NYASH_VM_STATS=1` and `NYASH_VM_STATS_JSON=1`.
## Next Steps
- Collect stats for normal and error flows (OK/404/500/unreachable, FileBox)。
- Compare hot instructions across scenariosBoxCall/Const/NewBox の比率、Branch/Jump/Phi の有無)。
- Feed findings into the 26-instruction diet discussionコア維持・メタ降格・型折りたたみ

View File

@ -48,6 +48,12 @@ pub enum VerificationError {
use_block: BasicBlockId,
def_block: BasicBlockId,
},
/// Merge block uses predecessor-defined value directly instead of Phi
MergeUsesPredecessorValue {
value: ValueId,
merge_block: BasicBlockId,
pred_block: BasicBlockId,
},
}
/// MIR verifier for SSA form and semantic correctness
@ -103,6 +109,10 @@ impl MirVerifier {
if let Err(mut cfg_errors) = self.verify_control_flow(function) {
local_errors.append(&mut cfg_errors);
}
// 4. Check merge-block value usage (ensure Phi is used)
if let Err(mut merge_errors) = self.verify_merge_uses(function) {
local_errors.append(&mut merge_errors);
}
if local_errors.is_empty() {
Ok(())
@ -220,6 +230,62 @@ impl MirVerifier {
}
}
/// Verify that blocks with multiple predecessors do not use predecessor-defined values directly.
/// In merge blocks, values coming from predecessors must be routed through Phi.
fn verify_merge_uses(&self, function: &MirFunction) -> Result<(), Vec<VerificationError>> {
let mut errors = Vec::new();
// Build predecessor map
let mut preds: std::collections::HashMap<BasicBlockId, Vec<BasicBlockId>> = std::collections::HashMap::new();
for (bid, block) in &function.blocks {
for succ in &block.successors {
preds.entry(*succ).or_default().push(*bid);
}
}
// Build definition map (value -> def block)
let mut def_block: std::collections::HashMap<ValueId, BasicBlockId> = std::collections::HashMap::new();
for (bid, block) in &function.blocks {
for inst in block.all_instructions() {
if let Some(dst) = inst.dst_value() {
def_block.insert(dst, *bid);
}
}
}
// Helper: collect phi dsts in a block
let mut phi_dsts_in_block: std::collections::HashMap<BasicBlockId, std::collections::HashSet<ValueId>> = std::collections::HashMap::new();
for (bid, block) in &function.blocks {
let set = phi_dsts_in_block.entry(*bid).or_default();
for inst in block.all_instructions() {
if let super::MirInstruction::Phi { dst, .. } = inst { set.insert(*dst); }
}
}
for (bid, block) in &function.blocks {
let Some(pred_list) = preds.get(bid) else { continue };
if pred_list.len() < 2 { continue; }
let phi_dsts = phi_dsts_in_block.get(bid);
// check instructions including terminator
for inst in block.all_instructions() {
for used in inst.used_values() {
if let Some(&db) = def_block.get(&used) {
if pred_list.contains(&db) {
// used value defined in a predecessor; must be routed via phi (i.e., used should be phi dst)
let is_phi_dst = phi_dsts.map(|s| s.contains(&used)).unwrap_or(false);
if !is_phi_dst {
errors.push(VerificationError::MergeUsesPredecessorValue {
value: used,
merge_block: *bid,
pred_block: db,
});
}
}
}
}
}
}
if errors.is_empty() { Ok(()) } else { Err(errors) }
}
/// Compute reachable blocks from entry
fn compute_reachable_blocks(&self, function: &MirFunction) -> HashSet<BasicBlockId> {
let mut reachable = HashSet::new();
@ -301,6 +367,10 @@ impl std::fmt::Display for VerificationError {
write!(f, "Value {} used in block {} but defined in non-dominating block {}",
value, use_block, def_block)
},
VerificationError::MergeUsesPredecessorValue { value, merge_block, pred_block } => {
write!(f, "Merge block {} uses predecessor-defined value {} from block {} without Phi",
merge_block, value, pred_block)
},
}
}
}
@ -308,7 +378,8 @@ impl std::fmt::Display for VerificationError {
#[cfg(test)]
mod tests {
use super::*;
use crate::mir::{MirFunction, FunctionSignature, MirType, EffectMask, BasicBlock};
use crate::mir::{MirFunction, FunctionSignature, MirType, EffectMask, BasicBlock, MirBuilder, MirPrinter};
use crate::ast::{ASTNode, Span, LiteralValue};
#[test]
fn test_valid_function_verification() {
@ -334,4 +405,45 @@ mod tests {
// and verify that the verifier catches it
// Implementation details would depend on the specific test case
}
#[test]
fn test_if_merge_uses_phi_not_predecessor() {
// Program:
// if true { result = "A" } else { result = "B" }
// result
let ast = ASTNode::Program {
statements: vec![
ASTNode::If {
condition: Box::new(ASTNode::Literal { value: LiteralValue::Bool(true), span: Span::unknown() }),
then_body: vec![ ASTNode::Assignment {
target: Box::new(ASTNode::Variable { name: "result".to_string(), span: Span::unknown() }),
value: Box::new(ASTNode::Literal { value: LiteralValue::String("A".to_string()), span: Span::unknown() }),
span: Span::unknown(),
}],
else_body: Some(vec![ ASTNode::Assignment {
target: Box::new(ASTNode::Variable { name: "result".to_string(), span: Span::unknown() }),
value: Box::new(ASTNode::Literal { value: LiteralValue::String("B".to_string()), span: Span::unknown() }),
span: Span::unknown(),
}]),
span: Span::unknown(),
},
ASTNode::Variable { name: "result".to_string(), span: Span::unknown() },
],
span: Span::unknown(),
};
let mut builder = MirBuilder::new();
let module = builder.build_module(ast).expect("build mir");
// Verify: should be OK (no MergeUsesPredecessorValue)
let mut verifier = MirVerifier::new();
let res = verifier.verify_module(&module);
if let Err(errs) = &res { eprintln!("Verifier errors: {:?}", errs); }
assert!(res.is_ok(), "MIR should pass merge-phi verification");
// Optional: ensure printer shows a phi in merge and ret returns a defined value
let mut printer = MirPrinter::verbose();
let mir_text = printer.print_module(&module);
assert!(mir_text.contains("phi"), "Printed MIR should contain a phi in merge block\n{}", mir_text);
}
}

51
tests/E2E_TESTS.md Normal file
View File

@ -0,0 +1,51 @@
# E2E Tests Documentation
最終更新: 2025-08-23
## E2E Plugin Net Tests (e2e_plugin_net.rs)
### HTTP Tests
#### 基本的なHTTP通信
- `e2e_http_ok_test` - 基本的なHTTP通信の成功ケース
- `e2e_http_post_and_headers` - POSTリクエストとヘッダー処理
- `e2e_http_empty_body` - 空ボディのレスポンス処理
- `e2e_http_multiple_requests_order` - 複数リクエストの順序処理
#### HTTPステータスコード処理
- `e2e_vm_http_status_404` - 404 Not Found レスポンス処理
- サーバーが404ステータスで応答 → クライアントは `Result.Ok(HttpResponseBox)` を受信
- `response.getStatus()` で 404、`response.readBody()` で "NF" を取得
- `e2e_vm_http_status_500` - 500 Internal Server Error レスポンス処理
- サーバーが500ステータスで応答 → クライアントは `Result.Ok(HttpResponseBox)` を受信
- `response.getStatus()` で 500、`response.readBody()` で "ERR" を取得
#### HTTPエラー処理
- `e2e_vm_http_client_error_result` - 接続失敗時のエラー処理
- ポート8099への接続失敗 → `Result.Err(ErrorBox)` を返す
- エラーメッセージ: "connect failed for 127.0.0.1:8099/nope"
### Socket Tests
- `e2e_socket_echo` - ソケットエコーサーバー
- `e2e_socket_timeout` - ソケットタイムアウト処理
## E2E Plugin Net Additional Tests (e2e_plugin_net_additional.rs)
### 高度なHTTPエラー処理
- `e2e_vm_http_client_error_result` - VM環境での接続エラー処理
- `e2e_vm_http_empty_body` - VM環境での空ボディ処理
## HTTPエラーハンドリングの重要な区別
### 接続失敗Transport Layer Error
- **状況**: サーバーに到達できないunreachable
- **結果**: `Result.Err(ErrorBox)`
- **例**: ポートが閉じている、ホストが存在しない
### HTTPステータスエラーApplication Layer Error
- **状況**: サーバーに到達したがアプリケーションエラー
- **結果**: `Result.Ok(HttpResponseBox)`
- **例**: 404 Not Found、500 Internal Server Error
- **理由**: トランスポート層は成功、HTTPプロトコルとして正常な応答
この区別により、ネットワークレベルのエラーとアプリケーションレベルのエラーを適切に処理できます。

View File

@ -84,6 +84,76 @@ body
assert_eq!(result.to_string_box().value, "OK");
}
#[test]
fn e2e_vm_http_status_404() {
std::env::set_var("NYASH_NET_LOG", "1");
std::env::set_var("NYASH_NET_LOG_FILE", "net_plugin3.log");
if !try_init_plugins() { return; }
let code = r#"
local srv, cli, r, resp, req, body, st
srv = new HttpServerBox()
srv.start(8086)
cli = new HttpClientBox()
r = cli.get("http://localhost:8086/notfound")
req = srv.accept().get_value()
resp = new HttpResponseBox()
resp.setStatus(404)
resp.write("NF")
req.respond(resp)
resp = r.get_value()
st = resp.getStatus()
body = resp.readBody()
st.toString() + ":" + body
"#;
let ast = NyashParser::parse_from_string(code).expect("parse failed");
let runtime = NyashRuntime::new();
let mut compiler = nyash_rust::mir::MirCompiler::new();
let compile_result = compiler.compile(ast).expect("mir compile failed");
let mut vm = VM::with_runtime(runtime);
let result = vm.execute_module(&compile_result.module).expect("vm exec failed");
assert_eq!(result.to_string_box().value, "404:NF");
}
#[test]
fn e2e_vm_http_status_500() {
std::env::set_var("NYASH_NET_LOG", "1");
std::env::set_var("NYASH_NET_LOG_FILE", "net_plugin3.log");
if !try_init_plugins() { return; }
let code = r#"
local srv, cli, r, resp, req, body, st
srv = new HttpServerBox()
srv.start(8084)
cli = new HttpClientBox()
r = cli.get("http://localhost:8084/error")
req = srv.accept().get_value()
resp = new HttpResponseBox()
resp.setStatus(500)
resp.write("ERR")
req.respond(resp)
resp = r.get_value()
st = resp.getStatus()
body = resp.readBody()
st.toString() + ":" + body
"#;
let ast = NyashParser::parse_from_string(code).expect("parse failed");
let runtime = NyashRuntime::new();
let mut compiler = nyash_rust::mir::MirCompiler::new();
let compile_result = compiler.compile(ast).expect("mir compile failed");
let mut vm = VM::with_runtime(runtime);
let result = vm.execute_module(&compile_result.module).expect("vm exec failed");
assert_eq!(result.to_string_box().value, "500:ERR");
}
#[test]
fn e2e_vm_http_post_and_headers() {
std::env::set_var("NYASH_NET_LOG", "1");

29
tools/run_vm_stats.sh Normal file
View File

@ -0,0 +1,29 @@
#!/usr/bin/env bash
set -euo pipefail
# Run Nyash VM with stats enabled and save JSON output
# Usage: tools/run_vm_stats.sh <nyash_file> [output_json]
if [ $# -lt 1 ]; then
echo "Usage: $0 <nyash_file> [output_json]" >&2
exit 1
fi
NYASH_FILE="$1"
OUT_JSON="${2:-vm_stats.json}"
if [ ! -f "$NYASH_FILE" ]; then
echo "File not found: $NYASH_FILE" >&2
exit 1
fi
NYASH_BIN="./target/release/nyash"
if [ ! -x "$NYASH_BIN" ]; then
echo "Building nyash in release mode..." >&2
cargo build --release -q
fi
echo "Running: $NYASH_BIN --backend vm --vm-stats --vm-stats-json $NYASH_FILE" >&2
NYASH_VM_STATS=1 NYASH_VM_STATS_JSON=1 "$NYASH_BIN" --backend vm --vm-stats --vm-stats-json "$NYASH_FILE" > "$OUT_JSON"
echo "Stats written to: $OUT_JSON" >&2