diff --git a/CURRENT_TASK.md b/CURRENT_TASK.md index bac400cf..528eaa09 100644 --- a/CURRENT_TASK.md +++ b/CURRENT_TASK.md @@ -5,6 +5,55 @@ Summary - PyVM is the semantic reference engine; llvmlite is used for AOT and parity checks. - GC: user modes defined; controller実装（rc+cycle skeleton + metrics/diagnostics）に移行。LLVM safepoint輸出/NyRT配線と自動挿入（envゲートON）を完了。 +Refactoring — Code Quality (High Priority, 2025‑09‑17 夜間) +- MIR instruction meta de‑boilerplate: + - Added `inst_meta!` macro and migrated major instructions (Unary/Compare/Load/Cast/TypeOp/Array{Get,Set}/Return/Branch/Jump/Print/Debug/Barrier*/Ref*/Weak*/Future*/Phi/NewBox). + - File `src/mir/instruction_kinds/mod.rs` shrank ~100 lines; behavior unchanged (introspection only). +- Tests safety pass (unwrap elimination): + - `src/tests/typebox_tlv_diff.rs` now uses `with_host` + `EnvGuard` + `inv_{ok,some,void}` helpers. + - Removed all `.unwrap()` from the file; failures carry context via `expect` or helper panic messages. +- Tokenizer duplication cut: + - Centralized single‑char token mapping via `single_char_token()`; kept longest‑match logic for multi‑char ops. +- Box operators de‑duplicate: + - Added `impl_static_numeric_ops!` for Integer/Float static ops (Add/Sub/Mul/Div), preserved zero‑division checks. + - Introduced `concat_result` and `can_repeat`; simplified Dynamic* impls and `OperatorResolver` via helper functions. +- Net plugin modularization (unsafe/TLV/state split): + - New: `plugins/nyash-net-plugin/src/ffi.rs` (CStr/ptr helpers), `tlv.rs` (encode/decode), `state.rs` (global maps/IDs). + - `lib.rs` delegates to these modules; unsafe is centralized, TLV logic unified, globals encapsulated. +— Python plugin refactor (Phase‑15) + - ffi 分離: `plugins/nyash-python-plugin/src/ffi.rs` に CPython ローダ/シンボル/`ensure_cpython()` を移設（挙動等価）。 + - GILGuard 適用拡大: `eval/import/getattr/str/call/callKw` の全パスを `gil::GILGuard` で保護。手動 Ensure/Release を撤去。 + - pytypes 導入: `plugins/nyash-python-plugin/src/pytypes.rs` + - TLV→Py 変換を集約: `count_tlv_args/tuple_from_tlv/kwargs_from_tlv`（内部 `fill_*` で unsafe を局在化）。 + - `take_py_error_string` を移設し、lib 側からの呼び出しを置換。 + - 参照ヘルパ（`incref/decref`）と CString/CStr ヘルパを用意（段階移行用）。 + - 旧ロジックの削除/整理: + - `lib.rs` の `fill_kwargs_from_tlv` を削除（pytypes へ移行済み）。 + - 旧 `tlv_count_args` と `fill_tuple_from_tlv` は廃止（コメント化→撤去予定）。 + - ビルド: `cargo check -p nyash-python-plugin` 警告ゼロ、Net プラグインも警告ゼロを維持。 + +Done (2025‑09‑18) +- Python plugin refactor 完了 + - RAII: `PyOwned`/`PyBorrowed` 導入＋ `OBJ_PTRS` 撤去。参照は型で管理（Drop=DecRef）。 + - autodecode を `pytypes` に移設（`DecodedValue`）。呼び出し側は TLV 書き戻しのみ。 + - CString/CStr/bytes 変換を `pytypes` に統一。 + - ログ出力を `NYASH_PY_LOG` でガードし既定静音化。 + - クリーンアップ: 過剰 `allow(dead_code)` の縮小とコメント整理。 +- ny-llvmc: EXE 出力対応 + - `--emit exe/obj`、`--nyrt `、`--libs` を実装。`.o` 生成後に NyRT とリンクして実行可能に。 + - 代表ケースで EXE 実行（exit code）を確認。 + +Next (Immediate) +- tools/build_llvm.sh の crate→EXE 統合 + - `NYASH_LLVM_COMPILER=crate` かつ `NYASH_LLVM_EMIT=exe` の場合、`ny-llvmc --emit exe` を呼び出し、手動リンクをスキップ。 + - `NYASH_LLVM_NYRT`/`NYASH_LLVM_LIBS` 環境変数でリンク先/追加フラグを指定可能に（Linux 既定は `-ldl -lpthread -lm`）。 +- Self‑host/EXE スモークの整備 + - 代表3ケース（const/binop/branch など）の JSON→ny-llvmc→EXE→実行をワンショットで検証するスクリプト（Linux）。 + - 既存 `exe_first_smoke.sh`/`build_compiler_exe.sh` の補助として crate 直結経路を並行維持。 +- CI 追補 + - Linux ジョブに crate‑EXE ルートの最小スモークを追加（exit code 判定のみ）。 + + What Changed (recent) - MIR13 default enabled - `mir_no_phi()` default set to true (can disable via `NYASH_MIR_NO_PHI=0`). @@ -173,28 +222,31 @@ Plugin ABI v2 updates (2025‑09‑17) - `tools/build_llvm.sh` に `NYASH_LLVM_COMPILER=crate|harness` を追加（`crate` は `ny-llvmc`。JSON は `NYASH_LLVM_MIR_JSON` 指定） - JSON スキーマ検証を可能なら実行（`tools/validate_mir_json.py`） -Plugin ABI v2 updates (2025‑09‑17 — Python family + Net smoke + JSON emit) -- v2 migration（Python 系 完了） - - `plugins/nyash-python-plugin`: `nyash_typebox_PyRuntimeBox` / `nyash_typebox_PyObjectBox` を追加（resolve/invoke_id 実装。既存 v1 は残存） - - `plugins/nyash-python-parser-plugin`: `nyash_typebox_PythonParserBox` を追加（birth/parse/fini） - - `plugins/nyash-python-compiler-plugin`: `nyash_typebox_PythonCompilerBox` を追加（birth/compile/fini） - - `nyash.toml` に Python 系 3 ライブラリを登録（type_id: 40/41/60/61） -- Net 往復スモーク（最小） - - 追加: `apps/tests/net_roundtrip.nyash`（Server.start→Client.get→Server.accept/respond→Client.readBody） - - 追加: `tools/plugin_v2_smoke.sh` に Net 機能スモークを条件付きで実行（CI常時ジョブに内包） -- nyash → MIR JSON emit フラグ - - CLI `--emit-mir-json ` を追加（`src/cli.rs`）。`runner.execute_mir_module` でファイル出力→即終了を実装。 - - これにより `ny-llvmc` へ JSON を直結しやすくなった（次の CI 経路で使用予定） +Plugin ABI v2 updates — 完了報告（2025‑09‑17） +- v2 migration（全 first‑party 完了） + - Python 系: `PyRuntimeBox`/`PyObjectBox`/`PythonParserBox`/`PythonCompilerBox` を v2 化 + - 既存 first‑party（File/Path/Math/Time/Regex/Net/String/Array/Map/Integer/Console）を v2 化 + - Encoding/TOML も v2 追加（`EncodingBox`/`TOMLBox`）し `nyash.toml` に登録 +- Legacy 撤去 + - 旧ローダ（`src/runtime/plugin_loader_legacy.rs`）と旧C‑ABI FileBox を削除 + - 全プラグインの v1 エクスポート（abi/init/invoke）を物理削除（v2専用化） +- スモーク/CI + - v2 ロード＋機能（Regex/Response/Net往復）スモークを常時 + - `ny-llvmc`（crate）で .o 生成するCIジョブを追加（Linux） +- nyash → MIR JSON emit + - CLI `--emit-mir-json ` を追加し、`ny-llvmc` 直結導線を整備 -Plan after restart（次の計画） -- Python 系プラグインの v2 化（parser/compiler/python-plugin） -- Docs 追記（Net/Regex のメソッド表、型/戻りTLVの簡易表） -- スモーク強化 - - Net: `ServerBox.start -> Client.get -> Request.respond -> Response.readBody` の往復最小ケースを追加 - - 主要 v2 Box の軽機能（String/Array/Map/Regex/Path/Math/Time）を 1 ジョブで走らせる -- LLVM 共通化 - - `nyash` からの JSON emit コマンド/フラグ導入（`--emit-mir-json ` など）→ `ny-llvmc` 直結 - - CI に `ny-llvmc` 実 JSON 経路を追加（Linux 常時） -- NyRT 整理（軽） - - TLV/エラー定数を `include/nyash_abi.h` と整合させる（ヘッダ経由参照） - - （必要時）`nyrt_last_error()` の追加検討 +Next — Self‑Hosting/EXE（crate 直結） +- ny-llvmc 機能拡張（.exe 出力） + - `ny-llvmc --emit exe --out ` を実装（`.o` + NyRT リンク）。`--nyrt `/`--libs ` を受理 + - 既存 `tools/build_llvm.sh` の crate 経路と統合（env: `NYASH_LLVM_COMPILER=crate`） + - Linux でのリンクフラグ最小化（`-Wl,--whole-archive -lnyrt -Wl,--no-whole-archive -ldl -lpthread -lm`） +- CI 拡張 + - `.o` 生成に加え、`.exe` 生成＋実行（exit code 検証）ジョブを追加（Linux） + - 代表3ケース（const/binop/branch）で EXE を起動し `0` 戻りを確認 +- Self‑host pipeline 寄り + - nyash CLI `--emit-mir-json` を EXE-first パスにも活用（JSON → ny-llvmc → exe → 実行） + - 将来: PyVM/llvmlite パリティベンチ（小規模）→ EXE でも同値を継続確認 +- Docs/Guides 更新 + - `docs/LLVM_HARNESS.md` に ny-llvmc の exe 出力手順を追記 + - `docs/guides/selfhost-pilot.md` に crate 直結（.o/.exe）手順とトラブルシュート diff --git a/crates/nyash-llvm-compiler/src/main.rs b/crates/nyash-llvm-compiler/src/main.rs index aca7d1b4..2cf676d4 100644 --- a/crates/nyash-llvm-compiler/src/main.rs +++ b/crates/nyash-llvm-compiler/src/main.rs @@ -7,13 +7,16 @@ use anyhow::{bail, Context, Result}; use clap::{ArgAction, Parser}; #[derive(Parser, Debug)] -#[command(name = "ny-llvmc", about = "Nyash LLVM compiler (llvmlite harness wrapper)")] +#[command( + name = "ny-llvmc", + about = "Nyash LLVM compiler (llvmlite harness wrapper)" +)] struct Args { /// MIR JSON input file path (use '-' to read from stdin). When omitted with --dummy, a dummy ny_main is emitted. #[arg(long = "in", value_name = "FILE", default_value = "-")] infile: String, - /// Output object file (.o) + /// Output path. For `--emit obj`, this is an object (.o). For `--emit exe`, this is an executable path. #[arg(long, value_name = "FILE")] out: PathBuf, @@ -24,6 +27,18 @@ struct Args { /// Path to Python harness script (defaults to tools/llvmlite_harness.py in CWD) #[arg(long, value_name = "FILE")] harness: Option, + + /// Emit kind: 'obj' (default) or 'exe'. + #[arg(long, value_name = "{obj|exe}", default_value = "obj")] + emit: String, + + /// Path to directory containing libnyrt.a when emitting an executable. If omitted, searches target/release then crates/nyrt/target/release. + #[arg(long, value_name = "DIR")] + nyrt: Option, + + /// Extra linker libs/flags appended when emitting an executable (single string, space-separated). + #[arg(long, value_name = "FLAGS")] + libs: Option, } fn main() -> Result<()> { @@ -41,10 +56,27 @@ fn main() -> Result<()> { PathBuf::from("tools/llvmlite_harness.py") }; + // Determine emit kind + let emit_exe = matches!(args.emit.as_str(), "exe" | "EXE"); + if args.dummy { - run_harness_dummy(&harness_path, &args.out) + // Dummy ny_main: always go through harness to produce an object then link if requested + let obj_path = if emit_exe { + // derive a temporary .o path next to output + let mut p = args.out.clone(); + p.set_extension("o"); + p + } else { + args.out.clone() + }; + run_harness_dummy(&harness_path, &obj_path) .with_context(|| "failed to run harness in dummy mode")?; - println!("[ny-llvmc] dummy object written: {}", args.out.display()); + if emit_exe { + link_executable(&obj_path, &args.out, args.nyrt.as_ref(), args.libs.as_deref())?; + println!("[ny-llvmc] executable written: {}", args.out.display()); + } else { + println!("[ny-llvmc] dummy object written: {}", obj_path.display()); + } return Ok(()); } @@ -56,8 +88,8 @@ fn main() -> Result<()> { .read_to_string(&mut buf) .context("reading MIR JSON from stdin")?; // Basic sanity check that it's JSON - let _: serde_json::Value = serde_json::from_str(&buf) - .context("stdin does not contain valid JSON")?; + let _: serde_json::Value = + serde_json::from_str(&buf).context("stdin does not contain valid JSON")?; let tmp = std::env::temp_dir().join("ny_llvmc_stdin.json"); let mut f = File::create(&tmp).context("create temp json file")?; f.write_all(buf.as_bytes()).context("write temp json")?; @@ -71,9 +103,27 @@ fn main() -> Result<()> { bail!("input JSON not found: {}", input_path.display()); } - run_harness_in(&harness_path, &input_path, &args.out) - .with_context(|| format!("failed to compile MIR JSON via harness: {}", input_path.display()))?; - println!("[ny-llvmc] object written: {}", args.out.display()); + // Produce object first + let obj_path = if emit_exe { + let mut p = args.out.clone(); + p.set_extension("o"); + p + } else { + args.out.clone() + }; + + run_harness_in(&harness_path, &input_path, &obj_path).with_context(|| { + format!( + "failed to compile MIR JSON via harness: {}", + input_path.display() + ) + })?; + if emit_exe { + link_executable(&obj_path, &args.out, args.nyrt.as_ref(), args.libs.as_deref())?; + println!("[ny-llvmc] executable written: {}", args.out.display()); + } else { + println!("[ny-llvmc] object written: {}", obj_path.display()); + } // Cleanup temp file if used if let Some(p) = temp_path { @@ -120,3 +170,39 @@ fn ensure_python() -> Result<()> { } } +fn link_executable(obj: &Path, out_exe: &Path, nyrt_dir_opt: Option<&PathBuf>, extra_libs: Option<&str>) -> Result<()> { + // Resolve nyRT static lib + let nyrt_dir = if let Some(dir) = nyrt_dir_opt { + dir.clone() + } else { + // try target/release then crates/nyrt/target/release + let a = PathBuf::from("target/release"); + let b = PathBuf::from("crates/nyrt/target/release"); + if a.join("libnyrt.a").exists() { a } else { b } + }; + let libnyrt = nyrt_dir.join("libnyrt.a"); + if !libnyrt.exists() { + bail!("libnyrt.a not found in {} (use --nyrt to specify)", nyrt_dir.display()); + } + + // Choose a C linker + let linker = ["cc", "clang", "gcc"].into_iter().find(|c| Command::new(c).arg("--version").output().map(|o| o.status.success()).unwrap_or(false)).unwrap_or("cc"); + + let mut cmd = Command::new(linker); + cmd.arg("-o").arg(out_exe); + cmd.arg(obj); + // Whole-archive libnyrt to ensure all objects are linked + cmd.arg("-Wl,--whole-archive").arg(&libnyrt).arg("-Wl,--no-whole-archive"); + // Common libs on Linux + cmd.arg("-ldl").arg("-lpthread").arg("-lm"); + if let Some(extras) = extra_libs { + for tok in extras.split_whitespace() { + cmd.arg(tok); + } + } + let status = cmd.status().context("failed to invoke system linker")?; + if !status.success() { + bail!("linker exited with status: {:?}", status.code()); + } + Ok(()) +} diff --git a/crates/nyrt/src/lib.rs b/crates/nyrt/src/lib.rs index f9b5a033..4dc9bc18 100644 --- a/crates/nyrt/src/lib.rs +++ b/crates/nyrt/src/lib.rs @@ -677,7 +677,9 @@ pub extern "C" fn main() -> i32 { // Choose GC hooks based on env (default dev: Counting for observability unless explicitly off) let mut rt_builder = nyash_rust::runtime::NyashRuntimeBuilder::new(); let gc_mode = nyash_rust::runtime::gc_mode::GcMode::from_env(); - let controller = std::sync::Arc::new(nyash_rust::runtime::gc_controller::GcController::new(gc_mode)); + let controller = std::sync::Arc::new(nyash_rust::runtime::gc_controller::GcController::new( + gc_mode, + )); rt_builder = rt_builder.with_gc_hooks(controller); let rt_hooks = rt_builder.build(); nyash_rust::runtime::global_hooks::set_from_runtime(&rt_hooks); @@ -720,16 +722,23 @@ pub extern "C" fn main() -> i32 { let want_json = std::env::var("NYASH_GC_METRICS_JSON").ok().as_deref() == Some("1"); let want_text = std::env::var("NYASH_GC_METRICS").ok().as_deref() == Some("1"); if want_json || want_text { - let (sp, br, bw) = rt_hooks - .gc - .snapshot_counters() - .unwrap_or((0, 0, 0)); + let (sp, br, bw) = rt_hooks.gc.snapshot_counters().unwrap_or((0, 0, 0)); let handles = nyash_rust::jit::rt::handles::len(); let gc_mode_s = gc_mode.as_str(); // Include allocation totals if controller is used let any_gc: &dyn std::any::Any = &*rt_hooks.gc; - let (alloc_count, alloc_bytes, trial_nodes, trial_edges, collect_total, collect_sp, collect_alloc, last_ms, last_reason) = if let Some(ctrl) = any_gc - .downcast_ref::() + let ( + alloc_count, + alloc_bytes, + trial_nodes, + trial_edges, + collect_total, + collect_sp, + collect_alloc, + last_ms, + last_reason, + ) = if let Some(ctrl) = + any_gc.downcast_ref::() { let (ac, ab) = ctrl.alloc_totals(); let (tn, te) = ctrl.trial_reachability_last(); @@ -741,9 +750,18 @@ pub extern "C" fn main() -> i32 { (0, 0, 0, 0, 0, 0, 0, 0, 0) }; // Settings snapshot (env) - let sp_interval = std::env::var("NYASH_GC_COLLECT_SP").ok().and_then(|s| s.parse::().ok()).unwrap_or(0); - let alloc_thresh = std::env::var("NYASH_GC_COLLECT_ALLOC").ok().and_then(|s| s.parse::().ok()).unwrap_or(0); - let auto_sp = std::env::var("NYASH_LLVM_AUTO_SAFEPOINT").ok().map(|v| v == "1").unwrap_or(true); + let sp_interval = std::env::var("NYASH_GC_COLLECT_SP") + .ok() + .and_then(|s| s.parse::().ok()) + .unwrap_or(0); + let alloc_thresh = std::env::var("NYASH_GC_COLLECT_ALLOC") + .ok() + .and_then(|s| s.parse::().ok()) + .unwrap_or(0); + let auto_sp = std::env::var("NYASH_LLVM_AUTO_SAFEPOINT") + .ok() + .map(|v| v == "1") + .unwrap_or(true); if want_json { // Minimal JSON assembly to avoid extra deps in nyrt println!( diff --git a/crates/nyrt/src/plugin/invoke.rs b/crates/nyrt/src/plugin/invoke.rs index 226ab575..c173d08e 100644 --- a/crates/nyrt/src/plugin/invoke.rs +++ b/crates/nyrt/src/plugin/invoke.rs @@ -23,13 +23,11 @@ pub extern "C" fn nyash_plugin_invoke3_i64( let nargs = argc.max(0) as usize; let mut buf = nyash_rust::runtime::plugin_ffi_common::encode_tlv_header(nargs as u16); // Encode legacy VM arg at position into provided buffer (avoid capturing &mut buf) - let mut encode_from_legacy_into = |dst: &mut Vec, arg_pos: usize| { - nyrt_encode_from_legacy_at(dst, arg_pos) - }; + let mut encode_from_legacy_into = + |dst: &mut Vec, arg_pos: usize| nyrt_encode_from_legacy_at(dst, arg_pos); // Encode argument value or fallback to legacy slot (avoid capturing &mut buf) - let mut encode_arg_into = |dst: &mut Vec, val: i64, pos: usize| { - nyrt_encode_arg_or_legacy(dst, val, pos) - }; + let mut encode_arg_into = + |dst: &mut Vec, val: i64, pos: usize| nyrt_encode_arg_or_legacy(dst, val, pos); if nargs >= 1 { encode_arg_into(&mut buf, a1, 1); } @@ -203,9 +201,8 @@ pub extern "C" fn nyash_plugin_invoke3_f64( } }); }; - let mut encode_arg = |val: i64, pos: usize| { - crate::encode::nyrt_encode_arg_or_legacy(&mut buf, val, pos) - }; + let mut encode_arg = + |val: i64, pos: usize| crate::encode::nyrt_encode_arg_or_legacy(&mut buf, val, pos); if nargs >= 1 { encode_arg(a1, 1); } @@ -218,16 +215,17 @@ pub extern "C" fn nyash_plugin_invoke3_f64( } } // Invoke via shared helper - let (mut tag_ret, mut sz_ret, mut payload_ret): (u8, usize, Vec) = match invoke_core::plugin_invoke_call( - invoke.unwrap(), - type_id as u32, - method_id as u32, - instance_id, - &buf, - ) { - Some((t, s, p)) => (t, s, p), - None => return 0.0, - }; + let (mut tag_ret, mut sz_ret, mut payload_ret): (u8, usize, Vec) = + match invoke_core::plugin_invoke_call( + invoke.unwrap(), + type_id as u32, + method_id as u32, + instance_id, + &buf, + ) { + Some((t, s, p)) => (t, s, p), + None => return 0.0, + }; if let Some((tag, sz, payload)) = Some((tag_ret, sz_ret, payload_ret.as_slice())) { if let Some(f) = invoke_core::decode_entry_to_f64(tag, sz, payload) { return f; @@ -425,10 +423,17 @@ fn nyash_plugin_invoke_name_common_i64(method: &str, argc: i64, a0: i64, a1: i64 &mut out_len, ) }; - if rc != 0 { return 0; } + if rc != 0 { + return 0; + } let out_slice = &out[..out_len]; - if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(out_slice) { - if let Some(v) = super::invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; } + if let Some((tag, sz, payload)) = + nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(out_slice) + { + if let Some(v) = super::invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) + { + return v; + } } 0 } @@ -667,8 +672,12 @@ pub extern "C" fn nyash_plugin_invoke3_tagged_i64( if rc != 0 { return 0; } - if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) { - if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; } + if let Some((tag, sz, payload)) = + nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) + { + if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { + return v; + } } 0 } @@ -759,8 +768,12 @@ pub extern "C" fn nyash_plugin_invoke_tagged_v_i64( if rc != 0 { return 0; } - if let Some((tag, sz, payload)) = nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) { - if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { return v; } + if let Some((tag, sz, payload)) = + nyash_rust::runtime::plugin_ffi_common::decode::tlv_first(&out[..out_len]) + { + if let Some(v) = invoke_core::decode_entry_to_i64(tag, sz, payload, invoke.unwrap()) { + return v; + } } 0 } diff --git a/crates/nyrt/src/plugin/invoke_core.rs b/crates/nyrt/src/plugin/invoke_core.rs index 63426449..2c84806c 100644 --- a/crates/nyrt/src/plugin/invoke_core.rs +++ b/crates/nyrt/src/plugin/invoke_core.rs @@ -8,8 +8,7 @@ use nyash_rust::runtime::plugin_loader_v2::PluginBoxV2; pub struct Receiver { pub instance_id: u32, pub real_type_id: u32, - pub invoke: - unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32, + pub invoke: unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32, } /// Resolve receiver from a0: prefer handle registry; fallback to legacy VM args when allowed. @@ -27,9 +26,7 @@ pub fn resolve_receiver_for_a0(a0: i64) -> Option { } } // 2) Legacy VM args (index by a0) unless handle-only is enforced - if a0 >= 0 - && std::env::var("NYASH_JIT_ARGS_HANDLE_ONLY").ok().as_deref() != Some("1") - { + if a0 >= 0 && std::env::var("NYASH_JIT_ARGS_HANDLE_ONLY").ok().as_deref() != Some("1") { nyash_rust::jit::rt::with_legacy_vm_args(|args| { let idx = a0 as usize; if let Some(nyash_rust::backend::vm::VMValue::BoxRef(b)) = args.get(idx) { @@ -104,7 +101,15 @@ pub fn decode_entry_to_i64( tag: u8, sz: usize, payload: &[u8], - fallback_invoke: unsafe extern "C" fn(u32, u32, u32, *const u8, usize, *mut u8, *mut usize) -> i32, + fallback_invoke: unsafe extern "C" fn( + u32, + u32, + u32, + *const u8, + usize, + *mut u8, + *mut usize, + ) -> i32, ) -> Option { match tag { 2 => nyash_rust::runtime::plugin_ffi_common::decode::i32(payload).map(|v| v as i64), @@ -154,8 +159,10 @@ pub fn decode_entry_to_i64( } None } - 1 => nyash_rust::runtime::plugin_ffi_common::decode::bool(payload) - .map(|b| if b { 1 } else { 0 }), + 1 => { + nyash_rust::runtime::plugin_ffi_common::decode::bool(payload) + .map(|b| if b { 1 } else { 0 }) + } 5 => { if std::env::var("NYASH_JIT_NATIVE_F64").ok().as_deref() == Some("1") && sz == 8 { let mut b = [0u8; 8]; @@ -192,8 +199,13 @@ pub fn decode_entry_to_f64(tag: u8, sz: usize, payload: &[u8]) -> Option { } None } - 1 => nyash_rust::runtime::plugin_ffi_common::decode::bool(payload) - .map(|b| if b { 1.0 } else { 0.0 }), + 1 => nyash_rust::runtime::plugin_ffi_common::decode::bool(payload).map(|b| { + if b { + 1.0 + } else { + 0.0 + } + }), _ => None, } } diff --git a/plugins/nyash-encoding-plugin/src/lib.rs b/plugins/nyash-encoding-plugin/src/lib.rs index 980b998a..2142e680 100644 --- a/plugins/nyash-encoding-plugin/src/lib.rs +++ b/plugins/nyash-encoding-plugin/src/lib.rs @@ -146,9 +146,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -158,7 +158,9 @@ unsafe impl Sync for NyashTypeBoxFfi {} use std::ffi::CStr; extern "C" fn encoding_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "toUtf8Bytes" => M_TO_UTF8_BYTES, @@ -184,42 +186,91 @@ extern "C" fn encoding_invoke_id( unsafe { match method_id { M_BIRTH => { - if result_len.is_null() { return E_ARGS; } - if preflight(result, result_len, 4) { return E_SHORT; } + if result_len.is_null() { + return E_ARGS; + } + if preflight(result, result_len, 4) { + return E_SHORT; + } let id = NEXT_ID.fetch_add(1, Ordering::Relaxed); - if let Ok(mut m) = INST.lock() { m.insert(id, EncInstance); } else { return E_PLUGIN; } + if let Ok(mut m) = INST.lock() { + m.insert(id, EncInstance); + } else { + return E_PLUGIN; + } let b = id.to_le_bytes(); std::ptr::copy_nonoverlapping(b.as_ptr(), result, 4); - *result_len = 4; OK + *result_len = 4; + OK + } + M_FINI => { + if let Ok(mut m) = INST.lock() { + m.remove(&instance_id); + OK + } else { + E_PLUGIN + } } - M_FINI => { if let Ok(mut m) = INST.lock() { m.remove(&instance_id); OK } else { E_PLUGIN } } M_TO_UTF8_BYTES => { - let s = match read_arg_string(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; + let s = match read_arg_string(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; write_tlv_bytes(s.as_bytes(), result, result_len) } M_FROM_UTF8_BYTES => { - let bytes = match read_arg_bytes(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; - match String::from_utf8(bytes) { Ok(s) => write_tlv_string(&s, result, result_len), Err(_) => write_tlv_string("", result, result_len) } + let bytes = match read_arg_bytes(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; + match String::from_utf8(bytes) { + Ok(s) => write_tlv_string(&s, result, result_len), + Err(_) => write_tlv_string("", result, result_len), + } } M_BASE64_ENC => { - if let Some(b) = read_arg_bytes(args, args_len, 0) { let s = base64::encode(b); return write_tlv_string(&s, result, result_len); } - let s = match read_arg_string(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; + if let Some(b) = read_arg_bytes(args, args_len, 0) { + let s = base64::encode(b); + return write_tlv_string(&s, result, result_len); + } + let s = match read_arg_string(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; let enc = base64::encode(s.as_bytes()); write_tlv_string(&enc, result, result_len) } M_BASE64_DEC => { - let s = match read_arg_string(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; - match base64::decode(s.as_bytes()) { Ok(b) => write_tlv_bytes(&b, result, result_len), Err(_) => write_tlv_bytes(&[], result, result_len) } + let s = match read_arg_string(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; + match base64::decode(s.as_bytes()) { + Ok(b) => write_tlv_bytes(&b, result, result_len), + Err(_) => write_tlv_bytes(&[], result, result_len), + } } M_HEX_ENC => { - if let Some(b) = read_arg_bytes(args, args_len, 0) { let s = hex::encode(b); return write_tlv_string(&s, result, result_len); } - let s = match read_arg_string(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; + if let Some(b) = read_arg_bytes(args, args_len, 0) { + let s = hex::encode(b); + return write_tlv_string(&s, result, result_len); + } + let s = match read_arg_string(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; let enc = hex::encode(s.as_bytes()); write_tlv_string(&enc, result, result_len) } M_HEX_DEC => { - let s = match read_arg_string(args, args_len, 0) { Some(v) => v, None => return E_ARGS }; - match hex::decode(s.as_bytes()) { Ok(b) => write_tlv_bytes(&b, result, result_len), Err(_) => write_tlv_bytes(&[], result, result_len) } + let s = match read_arg_string(args, args_len, 0) { + Some(v) => v, + None => return E_ARGS, + }; + match hex::decode(s.as_bytes()) { + Ok(b) => write_tlv_bytes(&b, result, result_len), + Err(_) => write_tlv_bytes(&[], result, result_len), + } } _ => E_METHOD, } diff --git a/plugins/nyash-filebox-plugin/src/lib.rs b/plugins/nyash-filebox-plugin/src/lib.rs index a59164da..6b15a467 100644 --- a/plugins/nyash-filebox-plugin/src/lib.rs +++ b/plugins/nyash-filebox-plugin/src/lib.rs @@ -681,7 +681,11 @@ extern "C" fn filebox_invoke_id( if let Ok(mut map) = INSTANCES.lock() { map.insert( id, - FileBoxInstance { file: None, path: String::new(), buffer: None }, + FileBoxInstance { + file: None, + path: String::new(), + buffer: None, + }, ); } else { return NYB_E_PLUGIN_ERROR; @@ -900,7 +904,11 @@ extern "C" fn filebox_invoke_id( if let Ok(mut map) = INSTANCES.lock() { map.insert( new_id, - FileBoxInstance { file: None, path: String::new(), buffer: None }, + FileBoxInstance { + file: None, + path: String::new(), + buffer: None, + }, ); } // Return Handle TLV (type_id from config resolves host-side; we encode (6,new_id) here if needed) diff --git a/plugins/nyash-math-plugin/src/lib.rs b/plugins/nyash-math-plugin/src/lib.rs index 50da03af..e6cee37f 100644 --- a/plugins/nyash-math-plugin/src/lib.rs +++ b/plugins/nyash-math-plugin/src/lib.rs @@ -115,9 +115,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (resolve/invoke_id per box) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -127,7 +127,9 @@ unsafe impl Sync for NyashTypeBoxFfi {} use std::ffi::CStr; extern "C" fn mathbox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "sqrt" => M_SQRT, @@ -140,7 +142,9 @@ extern "C" fn mathbox_resolve(name: *const std::os::raw::c_char) -> u32 { } } extern "C" fn timebox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "now" => T_NOW, diff --git a/plugins/nyash-net-plugin/src/ffi.rs b/plugins/nyash-net-plugin/src/ffi.rs new file mode 100644 index 00000000..f6f2f7fe --- /dev/null +++ b/plugins/nyash-net-plugin/src/ffi.rs @@ -0,0 +1,19 @@ +use std::ffi::CStr; +use std::os::raw::c_char; + +// Safe wrapper: convert C string pointer to owned String. +// Safety details are contained within; caller gets a safe String. +pub fn cstr_to_string(ptr: *const c_char) -> String { + if ptr.is_null() { + return String::new(); + } + unsafe { CStr::from_ptr(ptr) } + .to_string_lossy() + .into_owned() +} + +// Re-export a safe view over a raw byte slice pointer. +// This function is unsafe since it trusts the pointer/length. +pub unsafe fn slice<'a>(p: *const u8, len: usize) -> &'a [u8] { + std::slice::from_raw_parts(p, len) +} diff --git a/plugins/nyash-net-plugin/src/lib.rs b/plugins/nyash-net-plugin/src/lib.rs index 077b1dfd..37f69d2b 100644 --- a/plugins/nyash-net-plugin/src/lib.rs +++ b/plugins/nyash-net-plugin/src/lib.rs @@ -4,11 +4,11 @@ use once_cell::sync::Lazy; use std::collections::{HashMap, VecDeque}; +use std::io::Read; use std::io::Write as IoWrite; -use std::io::{Read, Write}; use std::net::{TcpListener, TcpStream}; use std::sync::{ - atomic::{AtomicBool, AtomicU32, Ordering}, + atomic::{AtomicBool, Ordering}, Arc, Mutex, }; use std::time::Duration; @@ -43,21 +43,21 @@ macro_rules! netlog { // Error codes const OK: i32 = 0; const E_SHORT: i32 = -1; -const E_INV_TYPE: i32 = -2; +const _E_INV_TYPE: i32 = -2; const E_INV_METHOD: i32 = -3; const E_INV_ARGS: i32 = -4; const E_ERR: i32 = -5; const E_INV_HANDLE: i32 = -8; // Type IDs -const T_SERVER: u32 = 20; +const _T_SERVER: u32 = 20; const T_REQUEST: u32 = 21; const T_RESPONSE: u32 = 22; -const T_CLIENT: u32 = 23; +const _T_CLIENT: u32 = 23; // Socket -const T_SOCK_SERVER: u32 = 30; +const _T_SOCK_SERVER: u32 = 30; const T_SOCK_CONN: u32 = 31; -const T_SOCK_CLIENT: u32 = 32; +const _T_SOCK_CLIENT: u32 = 32; // Methods const M_BIRTH: u32 = 0; @@ -103,23 +103,7 @@ const M_CONN_CLOSE: u32 = 3; // -> void const M_CONN_RECV_TIMEOUT: u32 = 4; // ms -> bytes (empty if timeout) // Global State -static SERVER_INSTANCES: Lazy>> = - Lazy::new(|| Mutex::new(HashMap::new())); -static SERVER_START_SEQ: AtomicU32 = AtomicU32::new(1); -static ACTIVE_SERVER_ID: Lazy>> = Lazy::new(|| Mutex::new(None)); -static LAST_ACCEPTED_REQ: Lazy>> = Lazy::new(|| Mutex::new(None)); -static REQUESTS: Lazy>> = Lazy::new(|| Mutex::new(HashMap::new())); -static RESPONSES: Lazy>> = - Lazy::new(|| Mutex::new(HashMap::new())); -static CLIENTS: Lazy>> = Lazy::new(|| Mutex::new(HashMap::new())); - -static SERVER_ID: AtomicU32 = AtomicU32::new(1); -static REQUEST_ID: AtomicU32 = AtomicU32::new(1); -static RESPONSE_ID: AtomicU32 = AtomicU32::new(1); -static CLIENT_ID: AtomicU32 = AtomicU32::new(1); -static SOCK_SERVER_ID: AtomicU32 = AtomicU32::new(1); -static SOCK_CONN_ID: AtomicU32 = AtomicU32::new(1); -static SOCK_CLIENT_ID: AtomicU32 = AtomicU32::new(1); +// moved to state.rs struct ServerState { running: Arc, @@ -201,9 +185,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (per-Box resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -211,10 +195,12 @@ pub struct NyashTypeBoxFfi { } unsafe impl Sync for NyashTypeBoxFfi {} -use std::ffi::CStr; +mod ffi; extern "C" fn responsebox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "setStatus" => M_RESP_SET_STATUS, "setHeader" => M_RESP_SET_HEADER, @@ -228,8 +214,10 @@ extern "C" fn responsebox_resolve(name: *const std::os::raw::c_char) -> u32 { } } extern "C" fn clientbox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "get" => M_CLIENT_GET, "post" => M_CLIENT_POST, @@ -285,8 +273,10 @@ pub static nyash_typebox_ClientBox: NyashTypeBoxFfi = NyashTypeBoxFfi { // --- ServerBox --- extern "C" fn serverbox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "start" => M_SERVER_START, "stop" => M_SERVER_STOP, @@ -319,8 +309,10 @@ pub static nyash_typebox_ServerBox: NyashTypeBoxFfi = NyashTypeBoxFfi { // --- SockServerBox --- extern "C" fn sockserver_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "start" => M_SRV_START, "stop" => M_SRV_STOP, @@ -354,8 +346,10 @@ pub static nyash_typebox_SockServerBox: NyashTypeBoxFfi = NyashTypeBoxFfi { // --- SockClientBox --- extern "C" fn sockclient_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "connect" => M_SC_CONNECT, "birth" => M_SC_BIRTH, @@ -386,8 +380,10 @@ pub static nyash_typebox_SockClientBox: NyashTypeBoxFfi = NyashTypeBoxFfi { // --- SockConnBox --- extern "C" fn sockconn_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "send" => M_CONN_SEND, "recv" => M_CONN_RECV, @@ -419,8 +415,10 @@ pub static nyash_typebox_SockConnBox: NyashTypeBoxFfi = NyashTypeBoxFfi { capabilities: 0, }; extern "C" fn requestbox_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); + if name.is_null() { + return 0; + } + let s = ffi::cstr_to_string(name); match s.as_ref() { "path" => M_REQ_PATH, "readBody" => M_REQ_READ_BODY, @@ -462,8 +460,8 @@ unsafe fn server_invoke( ) -> i32 { match m { M_BIRTH => { - let id = SERVER_ID.fetch_add(1, Ordering::Relaxed); - SERVER_INSTANCES.lock().unwrap().insert( + let id = state::next_server_id(); + state::SERVER_INSTANCES.lock().unwrap().insert( id, ServerState { running: Arc::new(AtomicBool::new(false)), @@ -473,14 +471,14 @@ unsafe fn server_invoke( start_seq: 0, }, ); - write_u32(id, res, res_len) + tlv::write_u32(id, res, res_len) } M_SERVER_START => { // args: TLV string/int (port) - let port = tlv_parse_i32(slice(args, args_len)).unwrap_or(0); - if let Some(s) = SERVER_INSTANCES.lock().unwrap().get_mut(&id) { + let port = tlv::tlv_parse_i32(slice(args, args_len)).unwrap_or(0); + if let Some(s) = state::SERVER_INSTANCES.lock().unwrap().get_mut(&id) { s.port = port; - s.start_seq = SERVER_START_SEQ.fetch_add(1, Ordering::Relaxed); + s.start_seq = state::next_server_start_seq(); let running = s.running.clone(); let pending = s.pending.clone(); running.store(true, Ordering::SeqCst); @@ -494,11 +492,10 @@ unsafe fn server_invoke( Err(e) => { netlog!("http:bind error {} err={:?}", addr, e); running.store(false, Ordering::SeqCst); - return write_tlv_void(res, res_len); + return tlv::write_tlv_void(res, res_len); } }; // Spawn HTTP listener thread (real TCP) - let server_id_copy = id; let handle = std::thread::spawn(move || { let _ = listener.set_nonblocking(true); loop { @@ -513,16 +510,16 @@ unsafe fn server_invoke( read_http_request(&mut stream) { // Store stream for later respond() - let conn_id = SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CONNS.lock().unwrap().insert( + let conn_id = state::next_sock_conn_id(); + state::SOCK_CONNS.lock().unwrap().insert( conn_id, SockConnState { stream: Mutex::new(stream), }, ); - let req_id = REQUEST_ID.fetch_add(1, Ordering::Relaxed); - REQUESTS.lock().unwrap().insert( + let req_id = state::next_request_id(); + state::REQUESTS.lock().unwrap().insert( req_id, RequestState { path, @@ -549,36 +546,36 @@ unsafe fn server_invoke( *s.handle.lock().unwrap() = Some(handle); } // mark active server - *ACTIVE_SERVER_ID.lock().unwrap() = Some(id); - write_tlv_void(res, res_len) + *state::ACTIVE_SERVER_ID.lock().unwrap() = Some(id); + tlv::write_tlv_void(res, res_len) } M_SERVER_STOP => { - if let Some(s) = SERVER_INSTANCES.lock().unwrap().get_mut(&id) { + if let Some(s) = state::SERVER_INSTANCES.lock().unwrap().get_mut(&id) { s.running.store(false, Ordering::SeqCst); if let Some(h) = s.handle.lock().unwrap().take() { let _ = h.join(); } } // clear active if this server was active - let mut active = ACTIVE_SERVER_ID.lock().unwrap(); + let mut active = state::ACTIVE_SERVER_ID.lock().unwrap(); if active.map(|v| v == id).unwrap_or(false) { *active = None; } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_SERVER_ACCEPT => { // wait up to ~5000ms for a request to arrive for _ in 0..1000 { // Prefer TCP-backed requests (server_conn_id=Some) over stub ones if let Some(req_id) = { - let mut map = SERVER_INSTANCES.lock().unwrap(); + let mut map = state::SERVER_INSTANCES.lock().unwrap(); if let Some(s) = map.get_mut(&id) { let mut q = s.pending.lock().unwrap(); // Find first index with TCP backing let mut chosen: Option = None; for i in 0..q.len() { if let Some(rid) = q.get(i).copied() { - if let Some(rq) = REQUESTS.lock().unwrap().get(&rid) { + if let Some(rq) = state::REQUESTS.lock().unwrap().get(&rid) { if rq.server_conn_id.is_some() { chosen = Some(i); break; @@ -596,12 +593,12 @@ unsafe fn server_invoke( } } { netlog!("server.accept: return req_id={} srv_id={}", req_id, id); - *LAST_ACCEPTED_REQ.lock().unwrap() = Some(req_id); - return write_tlv_handle(T_REQUEST, req_id, res, res_len); + *state::LAST_ACCEPTED_REQ.lock().unwrap() = Some(req_id); + return tlv::write_tlv_handle(T_REQUEST, req_id, res, res_len); } std::thread::sleep(Duration::from_millis(5)); } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } _ => E_INV_METHOD, } @@ -617,8 +614,8 @@ unsafe fn request_invoke( ) -> i32 { match m { M_BIRTH => { - let id = REQUEST_ID.fetch_add(1, Ordering::Relaxed); - REQUESTS.lock().unwrap().insert( + let id = state::next_request_id(); + state::REQUESTS.lock().unwrap().insert( id, RequestState { path: String::new(), @@ -628,25 +625,25 @@ unsafe fn request_invoke( responded: false, }, ); - write_u32(id, res, res_len) + tlv::write_u32(id, res, res_len) } M_REQ_PATH => { - if let Some(rq) = REQUESTS.lock().unwrap().get(&id) { - write_tlv_string(&rq.path, res, res_len) + if let Some(rq) = state::REQUESTS.lock().unwrap().get(&id) { + tlv::write_tlv_string(&rq.path, res, res_len) } else { E_INV_HANDLE } } M_REQ_READ_BODY => { - if let Some(rq) = REQUESTS.lock().unwrap().get(&id) { - write_tlv_bytes(&rq.body, res, res_len) + if let Some(rq) = state::REQUESTS.lock().unwrap().get(&id) { + tlv::write_tlv_bytes(&rq.body, res, res_len) } else { E_INV_HANDLE } } M_REQ_RESPOND => { // args: TLV Handle(Response) - let (t, provided_resp_id) = tlv_parse_handle(slice(_args, _args_len)) + let (t, provided_resp_id) = tlv::tlv_parse_handle(slice(_args, _args_len)) .map_err(|_| ()) .or(Err(())) .unwrap_or((0, 0)); @@ -654,7 +651,7 @@ unsafe fn request_invoke( return E_INV_ARGS; } // Acquire request - let mut rq_map = REQUESTS.lock().unwrap(); + let mut rq_map = state::REQUESTS.lock().unwrap(); if let Some(rq) = rq_map.get_mut(&id) { netlog!( "Request.respond: req_id={} provided_resp_id={} server_conn_id={:?} response_id_hint={:?}", @@ -665,7 +662,7 @@ unsafe fn request_invoke( drop(rq_map); // Read response content from provided response handle let (status, headers, body) = { - let resp_map = RESPONSES.lock().unwrap(); + let resp_map = state::RESPONSES.lock().unwrap(); if let Some(src) = resp_map.get(&provided_resp_id) { netlog!( "Request.respond: Reading response id={}, status={}, body_len={}", @@ -714,7 +711,7 @@ unsafe fn request_invoke( "Request.respond: Sending HTTP response, buf_len={}", buf.len() ); - if let Some(conn) = SOCK_CONNS.lock().unwrap().remove(&conn_id) { + if let Some(conn) = state::SOCK_CONNS.lock().unwrap().remove(&conn_id) { if let Ok(mut s) = conn.stream.lock() { let _ = s.write_all(&buf); let _ = s.flush(); @@ -728,10 +725,10 @@ unsafe fn request_invoke( } // Also mirror to paired client Response handle to avoid race on immediate read if let Some(target_id) = { - let rq_map2 = REQUESTS.lock().unwrap(); + let rq_map2 = state::REQUESTS.lock().unwrap(); rq_map2.get(&id).and_then(|rq2| rq2.response_id) } { - let mut resp_map = RESPONSES.lock().unwrap(); + let mut resp_map = state::RESPONSES.lock().unwrap(); let dst = resp_map.entry(target_id).or_insert(ResponseState { status: 200, headers: HashMap::new(), @@ -746,19 +743,19 @@ unsafe fn request_invoke( } // mark responded { - let mut rq_map3 = REQUESTS.lock().unwrap(); + let mut rq_map3 = state::REQUESTS.lock().unwrap(); if let Some(rq3) = rq_map3.get_mut(&id) { rq3.responded = true; } } - return write_tlv_void(res, res_len); + return tlv::write_tlv_void(res, res_len); } // Not backed by a socket: attempt reroute to last accepted or latest TCP-backed unresponded request drop(rq_map); let candidate_req = { - if let Some(last_id) = *LAST_ACCEPTED_REQ.lock().unwrap() { - if let Some(r) = REQUESTS.lock().unwrap().get(&last_id) { + if let Some(last_id) = *state::LAST_ACCEPTED_REQ.lock().unwrap() { + if let Some(r) = state::REQUESTS.lock().unwrap().get(&last_id) { if r.server_conn_id.is_some() && !r.responded { Some(last_id) } else { @@ -772,7 +769,7 @@ unsafe fn request_invoke( } } .or_else(|| { - REQUESTS + state::REQUESTS .lock() .unwrap() .iter() @@ -787,12 +784,12 @@ unsafe fn request_invoke( }); if let Some(target_req_id) = candidate_req { let (conn_id_alt, resp_hint_alt) = { - let map = REQUESTS.lock().unwrap(); + let map = state::REQUESTS.lock().unwrap(); let r = map.get(&target_req_id).unwrap(); (r.server_conn_id.unwrap(), r.response_id) }; let (status, headers, body) = { - let resp_map = RESPONSES.lock().unwrap(); + let resp_map = state::RESPONSES.lock().unwrap(); if let Some(src) = resp_map.get(&provided_resp_id) { (src.status, src.headers.clone(), src.body.clone()) } else { @@ -829,14 +826,14 @@ unsafe fn request_invoke( target_req_id, conn_id_alt ); - if let Some(conn) = SOCK_CONNS.lock().unwrap().remove(&conn_id_alt) { + if let Some(conn) = state::SOCK_CONNS.lock().unwrap().remove(&conn_id_alt) { if let Ok(mut s) = conn.stream.lock() { let _ = s.write_all(&buf); let _ = s.flush(); } } if let Some(target_id) = resp_hint_alt { - let mut resp_map = RESPONSES.lock().unwrap(); + let mut resp_map = state::RESPONSES.lock().unwrap(); let dst = resp_map.entry(target_id).or_insert(ResponseState { status: 200, headers: HashMap::new(), @@ -849,10 +846,10 @@ unsafe fn request_invoke( dst.body = body.clone(); netlog!("Request.respond: mirrored client handle id={} body_len={} headers={} status={}", target_id, dst.body.len(), dst.headers.len(), dst.status); } - if let Some(rq4) = REQUESTS.lock().unwrap().get_mut(&target_req_id) { + if let Some(rq4) = state::REQUESTS.lock().unwrap().get_mut(&target_req_id) { rq4.responded = true; } - return write_tlv_void(res, res_len); + return tlv::write_tlv_void(res, res_len); } netlog!("Request.respond: no suitable TCP-backed request found for reroute; invalid handle"); return E_INV_HANDLE; @@ -873,8 +870,8 @@ unsafe fn response_invoke( ) -> i32 { match m { M_BIRTH => { - let id = RESPONSE_ID.fetch_add(1, Ordering::Relaxed); - RESPONSES.lock().unwrap().insert( + let id = state::next_response_id(); + state::RESPONSES.lock().unwrap().insert( id, ResponseState { status: 200, @@ -885,40 +882,40 @@ unsafe fn response_invoke( }, ); netlog!("Response.birth: new id={}", id); - write_u32(id, res, res_len) + tlv::write_u32(id, res, res_len) } M_RESP_SET_STATUS => { - let code = tlv_parse_i32(slice(args, args_len)).unwrap_or(200); - if let Some(rp) = RESPONSES.lock().unwrap().get_mut(&id) { + let code = tlv::tlv_parse_i32(slice(args, args_len)).unwrap_or(200); + if let Some(rp) = state::RESPONSES.lock().unwrap().get_mut(&id) { rp.status = code; } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_RESP_SET_HEADER => { - if let Ok((name, value)) = tlv_parse_two_strings(slice(args, args_len)) { - if let Some(rp) = RESPONSES.lock().unwrap().get_mut(&id) { + if let Ok((name, value)) = tlv::tlv_parse_two_strings(slice(args, args_len)) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get_mut(&id) { rp.headers.insert(name, value); } - return write_tlv_void(res, res_len); + return tlv::write_tlv_void(res, res_len); } E_INV_ARGS } M_RESP_WRITE => { // Accept String or Bytes - let bytes = tlv_parse_bytes(slice(args, args_len)).unwrap_or_default(); + let bytes = tlv::tlv_parse_bytes(slice(args, args_len)).unwrap_or_default(); netlog!("HttpResponse.write: id={} bytes_len={}", id, bytes.len()); - if let Some(rp) = RESPONSES.lock().unwrap().get_mut(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get_mut(&id) { rp.body.extend_from_slice(&bytes); netlog!("HttpResponse.write: body now has {} bytes", rp.body.len()); } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_RESP_READ_BODY => { netlog!("HttpResponse.readBody: enter id={}", id); // If bound to a client connection, lazily read and parse (with short retries) for _ in 0..50 { let need_parse = { - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { rp.client_conn_id } else { return E_INV_HANDLE; @@ -931,13 +928,13 @@ unsafe fn response_invoke( break; } } - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { netlog!( "HttpResponse.readBody: id={} body_len={}", id, rp.body.len() ); - write_tlv_bytes(&rp.body, res, res_len) + tlv::write_tlv_bytes(&rp.body, res, res_len) } else { E_INV_HANDLE } @@ -945,7 +942,7 @@ unsafe fn response_invoke( M_RESP_GET_STATUS => { for _ in 0..50 { let need_parse = { - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { rp.client_conn_id } else { return E_INV_HANDLE; @@ -958,17 +955,17 @@ unsafe fn response_invoke( break; } } - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { - write_tlv_i32(rp.status, res, res_len) + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { + tlv::write_tlv_i32(rp.status, res, res_len) } else { E_INV_HANDLE } } M_RESP_GET_HEADER => { - if let Ok(name) = tlv_parse_string(slice(args, args_len)) { + if let Ok(name) = tlv::tlv_parse_string(slice(args, args_len)) { for _ in 0..50 { let need_parse = { - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { rp.client_conn_id } else { return E_INV_HANDLE; @@ -981,9 +978,9 @@ unsafe fn response_invoke( break; } } - if let Some(rp) = RESPONSES.lock().unwrap().get(&id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get(&id) { let v = rp.headers.get(&name).cloned().unwrap_or_default(); - return write_tlv_string(&v, res, res_len); + return tlv::write_tlv_string(&v, res, res_len); } else { return E_INV_HANDLE; } @@ -996,7 +993,7 @@ unsafe fn response_invoke( unsafe fn client_invoke( m: u32, - id: u32, + _id: u32, args: *const u8, args_len: usize, res: *mut u8, @@ -1004,40 +1001,33 @@ unsafe fn client_invoke( ) -> i32 { match m { M_BIRTH => { - let id = CLIENT_ID.fetch_add(1, Ordering::Relaxed); - CLIENTS.lock().unwrap().insert(id, ClientState); - write_u32(id, res, res_len) + let id = state::next_client_id(); + state::CLIENTS.lock().unwrap().insert(id, ClientState); + tlv::write_u32(id, res, res_len) } M_CLIENT_GET => { // args: TLV String(url) - let url = tlv_parse_string(slice(args, args_len)).unwrap_or_default(); + let url = tlv::tlv_parse_string(slice(args, args_len)).unwrap_or_default(); let port = parse_port(&url).unwrap_or(80); let host = parse_host(&url).unwrap_or_else(|| "127.0.0.1".to_string()); let path = parse_path(&url); // Create client response handle first, so we can include it in header - let resp_id = RESPONSE_ID.fetch_add(1, Ordering::Relaxed); + let resp_id = state::next_response_id(); let (_h, _p, req_bytes) = build_http_request("GET", &url, None, resp_id); // Try TCP connect (best effort) let mut tcp_ok = false; if let Ok(mut stream) = TcpStream::connect(format!("{}:{}", host, port)) { let _ = stream.write_all(&req_bytes); let _ = stream.flush(); - let conn_id = SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CONNS.lock().unwrap().insert( + let conn_id = state::next_sock_conn_id(); + state::SOCK_CONNS.lock().unwrap().insert( conn_id, SockConnState { stream: Mutex::new(stream), }, ); - // Map to server_id by port if available - let server_id_for_port = { - let servers = SERVER_INSTANCES.lock().unwrap(); - servers - .iter() - .find(|(_, s)| s.port == port) - .map(|(sid, _)| *sid) - }; - RESPONSES.lock().unwrap().insert( + // Map to server_id by port if available (not used; reserved) + state::RESPONSES.lock().unwrap().insert( resp_id, ResponseState { status: 0, @@ -1055,14 +1045,8 @@ unsafe fn client_invoke( conn_id ); } else { - let server_id_for_port = { - let servers = SERVER_INSTANCES.lock().unwrap(); - servers - .iter() - .find(|(_, s)| s.port == port) - .map(|(sid, _)| *sid) - }; - RESPONSES.lock().unwrap().insert( + // Map to server_id by port if available (not used; reserved) + state::RESPONSES.lock().unwrap().insert( resp_id, ResponseState { status: 0, @@ -1076,7 +1060,7 @@ unsafe fn client_invoke( } // No stub enqueue in TCP-only design if tcp_ok { - write_tlv_handle(T_RESPONSE, resp_id, res, res_len) + tlv::write_tlv_handle(T_RESPONSE, resp_id, res, res_len) } else { // Encode error string; loader interprets returns_result=true methods' string payload as Err let msg = format!( @@ -1085,20 +1069,20 @@ unsafe fn client_invoke( port, if path.is_empty() { "" } else { &path } ); - write_tlv_string(&msg, res, res_len) + tlv::write_tlv_string(&msg, res, res_len) } } M_CLIENT_POST => { // args: TLV String(url), Bytes body let data = slice(args, args_len); - let (_, argc, mut pos) = tlv_parse_header(data) + let (_, argc, mut pos) = tlv::tlv_parse_header(data) .map_err(|_| ()) .or(Err(())) .unwrap_or((1, 0, 4)); if argc < 2 { return E_INV_ARGS; } - let (_t1, s1, p1) = tlv_parse_entry_hdr(data, pos) + let (_t1, s1, p1) = tlv::tlv_parse_entry_hdr(data, pos) .map_err(|_| ()) .or(Err(())) .unwrap_or((0, 0, 0)); @@ -1111,7 +1095,7 @@ unsafe fn client_invoke( .unwrap_or("") .to_string(); pos = p1 + s1; - let (t2, s2, p2) = tlv_parse_entry_hdr(data, pos) + let (t2, s2, p2) = tlv::tlv_parse_entry_hdr(data, pos) .map_err(|_| ()) .or(Err(())) .unwrap_or((0, 0, 0)); @@ -1124,27 +1108,21 @@ unsafe fn client_invoke( let path = parse_path(&url); let body_len = body.len(); // Create client response handle - let resp_id = RESPONSE_ID.fetch_add(1, Ordering::Relaxed); + let resp_id = state::next_response_id(); let (_h, _p, req_bytes) = build_http_request("POST", &url, Some(&body), resp_id); let mut tcp_ok = false; if let Ok(mut stream) = TcpStream::connect(format!("{}:{}", host, port)) { let _ = stream.write_all(&req_bytes); let _ = stream.flush(); - let conn_id = SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CONNS.lock().unwrap().insert( + let conn_id = state::next_sock_conn_id(); + state::SOCK_CONNS.lock().unwrap().insert( conn_id, SockConnState { stream: Mutex::new(stream), }, ); - let server_id_for_port = { - let servers = SERVER_INSTANCES.lock().unwrap(); - servers - .iter() - .find(|(_, s)| s.port == port) - .map(|(sid, _)| *sid) - }; - RESPONSES.lock().unwrap().insert( + // Map to server_id by port if available (not used; reserved) + state::RESPONSES.lock().unwrap().insert( resp_id, ResponseState { status: 0, @@ -1163,14 +1141,8 @@ unsafe fn client_invoke( body.len() ); } else { - let server_id_for_port = { - let servers = SERVER_INSTANCES.lock().unwrap(); - servers - .iter() - .find(|(_, s)| s.port == port) - .map(|(sid, _)| *sid) - }; - RESPONSES.lock().unwrap().insert( + // Map to server_id by port if available (not used; reserved) + state::RESPONSES.lock().unwrap().insert( resp_id, ResponseState { status: 0, @@ -1189,7 +1161,7 @@ unsafe fn client_invoke( } // No stub enqueue in TCP-only design if tcp_ok { - write_tlv_handle(T_RESPONSE, resp_id, res, res_len) + tlv::write_tlv_handle(T_RESPONSE, resp_id, res, res_len) } else { let msg = format!( "connect failed for {}:{}{} (body_len={})", @@ -1198,7 +1170,7 @@ unsafe fn client_invoke( if path.is_empty() { "" } else { &path }, body_len ); - write_tlv_string(&msg, res, res_len) + tlv::write_tlv_string(&msg, res, res_len) } } _ => E_INV_METHOD, @@ -1238,176 +1210,8 @@ fn parse_port(url: &str) -> Option { } // ===== Helpers ===== -unsafe fn slice<'a>(p: *const u8, len: usize) -> &'a [u8] { - std::slice::from_raw_parts(p, len) -} - -fn write_u32(v: u32, res: *mut u8, res_len: *mut usize) -> i32 { - unsafe { - if res_len.is_null() { - return E_INV_ARGS; - } - if res.is_null() || *res_len < 4 { - *res_len = 4; - return E_SHORT; - } - let b = v.to_le_bytes(); - std::ptr::copy_nonoverlapping(b.as_ptr(), res, 4); - *res_len = 4; - } - OK -} - -fn write_tlv_result(payloads: &[(u8, &[u8])], res: *mut u8, res_len: *mut usize) -> i32 { - if res_len.is_null() { - return E_INV_ARGS; - } - let mut buf = Vec::with_capacity(4 + payloads.iter().map(|(_, p)| 4 + p.len()).sum::()); - buf.extend_from_slice(&1u16.to_le_bytes()); - buf.extend_from_slice(&(payloads.len() as u16).to_le_bytes()); - for (tag, p) in payloads { - buf.push(*tag); - buf.push(0); - buf.extend_from_slice(&(p.len() as u16).to_le_bytes()); - buf.extend_from_slice(p); - } - unsafe { - let need = buf.len(); - if res.is_null() || *res_len < need { - *res_len = need; - return E_SHORT; - } - std::ptr::copy_nonoverlapping(buf.as_ptr(), res, need); - *res_len = need; - } - OK -} - -fn write_tlv_void(res: *mut u8, res_len: *mut usize) -> i32 { - write_tlv_result(&[(9u8, &[])], res, res_len) -} -fn write_tlv_string(s: &str, res: *mut u8, res_len: *mut usize) -> i32 { - write_tlv_result(&[(6u8, s.as_bytes())], res, res_len) -} -fn write_tlv_bytes(b: &[u8], res: *mut u8, res_len: *mut usize) -> i32 { - write_tlv_result(&[(7u8, b)], res, res_len) -} -fn write_tlv_i32(v: i32, res: *mut u8, res_len: *mut usize) -> i32 { - write_tlv_result(&[(2u8, &v.to_le_bytes())], res, res_len) -} -fn write_tlv_handle(t: u32, id: u32, res: *mut u8, res_len: *mut usize) -> i32 { - let mut payload = [0u8; 8]; - payload[0..4].copy_from_slice(&t.to_le_bytes()); - payload[4..8].copy_from_slice(&id.to_le_bytes()); - write_tlv_result(&[(8u8, &payload)], res, res_len) -} - -fn tlv_parse_header(data: &[u8]) -> Result<(u16, u16, usize), ()> { - if data.len() < 4 { - return Err(()); - } - let ver = u16::from_le_bytes([data[0], data[1]]); - let argc = u16::from_le_bytes([data[2], data[3]]); - if ver != 1 { - return Err(()); - } - Ok((ver, argc, 4)) -} -fn tlv_parse_string(data: &[u8]) -> Result { - let (_, argc, mut pos) = tlv_parse_header(data)?; - if argc < 1 { - return Err(()); - } - let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; - if tag != 6 { - return Err(()); - } - Ok(std::str::from_utf8(&data[p..p + size]) - .map_err(|_| ())? - .to_string()) -} -fn tlv_parse_two_strings(data: &[u8]) -> Result<(String, String), ()> { - let (_, argc, mut pos) = tlv_parse_header(data)?; - if argc < 2 { - return Err(()); - } - let (tag1, size1, p1) = tlv_parse_entry_hdr(data, pos)?; - if tag1 != 6 { - return Err(()); - } - let s1 = std::str::from_utf8(&data[p1..p1 + size1]) - .map_err(|_| ())? - .to_string(); - pos = p1 + size1; - let (tag2, size2, p2) = tlv_parse_entry_hdr(data, pos)?; - if tag2 != 6 { - return Err(()); - } - let s2 = std::str::from_utf8(&data[p2..p2 + size2]) - .map_err(|_| ())? - .to_string(); - Ok((s1, s2)) -} -fn tlv_parse_bytes(data: &[u8]) -> Result, ()> { - let (_, argc, mut pos) = tlv_parse_header(data)?; - if argc < 1 { - return Err(()); - } - let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; - if tag != 6 && tag != 7 { - return Err(()); - } - Ok(data[p..p + size].to_vec()) -} -fn tlv_parse_i32(data: &[u8]) -> Result { - let (_, argc, mut pos) = tlv_parse_header(data)?; - if argc < 1 { - return Err(()); - } - let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; - match (tag, size) { - (2, 4) => { - let mut b = [0u8; 4]; - b.copy_from_slice(&data[p..p + 4]); - Ok(i32::from_le_bytes(b)) - } - (5, 8) => { - // accept i64 - let mut b = [0u8; 8]; - b.copy_from_slice(&data[p..p + 8]); - Ok(i64::from_le_bytes(b) as i32) - } - _ => Err(()), - } -} -fn tlv_parse_handle(data: &[u8]) -> Result<(u32, u32), ()> { - let (_, argc, mut pos) = tlv_parse_header(data)?; - if argc < 1 { - return Err(()); - } - let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; - if tag != 8 || size != 8 { - return Err(()); - } - let mut t = [0u8; 4]; - let mut i = [0u8; 4]; - t.copy_from_slice(&data[p..p + 4]); - i.copy_from_slice(&data[p + 4..p + 8]); - Ok((u32::from_le_bytes(t), u32::from_le_bytes(i))) -} -fn tlv_parse_entry_hdr(data: &[u8], pos: usize) -> Result<(u8, usize, usize), ()> { - if pos + 4 > data.len() { - return Err(()); - } - let tag = data[pos]; - let _rsv = data[pos + 1]; - let size = u16::from_le_bytes([data[pos + 2], data[pos + 3]]) as usize; - let p = pos + 4; - if p + size > data.len() { - return Err(()); - } - Ok((tag, size, p)) -} +use ffi::slice; +mod tlv; // ===== HTTP helpers ===== fn parse_host(url: &str) -> Option { @@ -1526,7 +1330,7 @@ fn parse_client_response_into(resp_id: u32, conn_id: u32) { let mut body: Vec = Vec::new(); // Keep the connection until parsing succeeds; do not remove up front let mut should_remove = false; - if let Ok(mut map) = SOCK_CONNS.lock() { + if let Ok(mut map) = state::SOCK_CONNS.lock() { if let Some(conn) = map.get(&conn_id) { if let Ok(mut s) = conn.stream.lock() { let _ = s.set_read_timeout(Some(Duration::from_millis(4000))); @@ -1589,7 +1393,7 @@ fn parse_client_response_into(resp_id: u32, conn_id: u32) { map.remove(&conn_id); } } - if let Some(rp) = RESPONSES.lock().unwrap().get_mut(&resp_id) { + if let Some(rp) = state::RESPONSES.lock().unwrap().get_mut(&resp_id) { rp.status = status; rp.headers = headers; rp.body = body; @@ -1599,12 +1403,7 @@ fn parse_client_response_into(resp_id: u32, conn_id: u32) { } // ===== Socket implementation ===== -static SOCK_SERVERS: Lazy>> = - Lazy::new(|| Mutex::new(HashMap::new())); -static SOCK_CONNS: Lazy>> = - Lazy::new(|| Mutex::new(HashMap::new())); -static SOCK_CLIENTS: Lazy>> = - Lazy::new(|| Mutex::new(HashMap::new())); +// moved to state.rs unsafe fn sock_server_invoke( m: u32, @@ -1617,8 +1416,8 @@ unsafe fn sock_server_invoke( match m { M_SRV_BIRTH => { netlog!("sock:birth server"); - let id = SOCK_SERVER_ID.fetch_add(1, Ordering::Relaxed); - SOCK_SERVERS.lock().unwrap().insert( + let id = state::next_sock_server_id(); + state::SOCK_SERVERS.lock().unwrap().insert( id, SockServerState { running: Arc::new(AtomicBool::new(false)), @@ -1626,12 +1425,12 @@ unsafe fn sock_server_invoke( handle: Mutex::new(None), }, ); - write_u32(id, res, res_len) + tlv::write_u32(id, res, res_len) } M_SRV_START => { - let port = tlv_parse_i32(slice(args, args_len)).unwrap_or(0); + let port = tlv::tlv_parse_i32(slice(args, args_len)).unwrap_or(0); netlog!("sock:start server id={} port={}", id, port); - if let Some(ss) = SOCK_SERVERS.lock().unwrap().get(&id) { + if let Some(ss) = state::SOCK_SERVERS.lock().unwrap().get(&id) { let running = ss.running.clone(); let pending = ss.pending.clone(); running.store(true, Ordering::SeqCst); @@ -1644,8 +1443,8 @@ unsafe fn sock_server_invoke( match listener.accept() { Ok((stream, _)) => { stream.set_nonblocking(false).ok(); - let conn_id = SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CONNS.lock().unwrap().insert( + let conn_id = state::next_sock_conn_id(); + state::SOCK_CONNS.lock().unwrap().insert( conn_id, SockConnState { stream: Mutex::new(stream), @@ -1664,40 +1463,42 @@ unsafe fn sock_server_invoke( }); *ss.handle.lock().unwrap() = Some(handle); } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_SRV_STOP => { netlog!("sock:stop server id={}", id); - if let Some(ss) = SOCK_SERVERS.lock().unwrap().get(&id) { + if let Some(ss) = state::SOCK_SERVERS.lock().unwrap().get(&id) { ss.running.store(false, Ordering::SeqCst); if let Some(h) = ss.handle.lock().unwrap().take() { let _ = h.join(); } } - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_SRV_ACCEPT => { - if let Some(ss) = SOCK_SERVERS.lock().unwrap().get(&id) { + if let Some(ss) = state::SOCK_SERVERS.lock().unwrap().get(&id) { // wait up to ~5000ms for _ in 0..1000 { if let Some(cid) = ss.pending.lock().unwrap().pop_front() { netlog!("sock:accept returned conn_id={}", cid); - return write_tlv_handle(T_SOCK_CONN, cid, res, res_len); + return tlv::write_tlv_handle(T_SOCK_CONN, cid, res, res_len); } std::thread::sleep(std::time::Duration::from_millis(5)); } } netlog!("sock:accept timeout id={}", id); - write_tlv_void(res, res_len) + tlv::write_tlv_void(res, res_len) } M_SRV_ACCEPT_TIMEOUT => { - let timeout_ms = tlv_parse_i32(slice(args, args_len)).unwrap_or(0).max(0) as u64; - if let Some(ss) = SOCK_SERVERS.lock().unwrap().get(&id) { + let timeout_ms = tlv::tlv_parse_i32(slice(args, args_len)) + .unwrap_or(0) + .max(0) as u64; + if let Some(ss) = state::SOCK_SERVERS.lock().unwrap().get(&id) { let deadline = std::time::Instant::now() + Duration::from_millis(timeout_ms); loop { if let Some(cid) = ss.pending.lock().unwrap().pop_front() { netlog!("sock:acceptTimeout returned conn_id={}", cid); - return write_tlv_handle(T_SOCK_CONN, cid, res, res_len); + return tlv::write_tlv_handle(T_SOCK_CONN, cid, res, res_len); } if std::time::Instant::now() >= deadline { break; @@ -1715,7 +1516,7 @@ unsafe fn sock_server_invoke( unsafe fn sock_client_invoke( m: u32, - id: u32, + _id: u32, args: *const u8, args_len: usize, res: *mut u8, @@ -1723,21 +1524,24 @@ unsafe fn sock_client_invoke( ) -> i32 { match m { M_SC_BIRTH => { - let id = SOCK_CLIENT_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CLIENTS.lock().unwrap().insert(id, SockClientState); - write_u32(id, res, res_len) + let id = state::next_sock_client_id(); + state::SOCK_CLIENTS + .lock() + .unwrap() + .insert(id, SockClientState); + tlv::write_u32(id, res, res_len) } M_SC_CONNECT => { // args: host(string), port(i32) let data = slice(args, args_len); - let (_, argc, mut pos) = tlv_parse_header(data) + let (_, argc, mut pos) = tlv::tlv_parse_header(data) .map_err(|_| ()) .or(Err(())) .unwrap_or((1, 0, 4)); if argc < 2 { return E_INV_ARGS; } - let (_t1, s1, p1) = tlv_parse_entry_hdr(data, pos) + let (_t1, s1, p1) = tlv::tlv_parse_entry_hdr(data, pos) .map_err(|_| ()) .or(Err(())) .unwrap_or((0, 0, 0)); @@ -1750,7 +1554,7 @@ unsafe fn sock_client_invoke( .unwrap_or("") .to_string(); pos = p1 + s1; - let (_t2, _s2, p2) = tlv_parse_entry_hdr(data, pos) + let (_t2, _s2, p2) = tlv::tlv_parse_entry_hdr(data, pos) .map_err(|_| ()) .or(Err(())) .unwrap_or((0, 0, 0)); @@ -1764,17 +1568,17 @@ unsafe fn sock_client_invoke( }; let addr = format!("{}:{}", host, port); match TcpStream::connect(addr) { - Ok(mut stream) => { + Ok(stream) => { stream.set_nonblocking(false).ok(); - let conn_id = SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed); - SOCK_CONNS.lock().unwrap().insert( + let conn_id = state::next_sock_conn_id(); + state::SOCK_CONNS.lock().unwrap().insert( conn_id, SockConnState { stream: Mutex::new(stream), }, ); netlog!("sock:connect ok conn_id={}", conn_id); - write_tlv_handle(T_SOCK_CONN, conn_id, res, res_len) + tlv::write_tlv_handle(T_SOCK_CONN, conn_id, res, res_len) } Err(e) => { netlog!("sock:connect error: {:?}", e); @@ -1797,38 +1601,40 @@ unsafe fn sock_conn_invoke( match m { M_CONN_BIRTH => { // not used directly - write_u32(0, res, res_len) + tlv::write_u32(0, res, res_len) } M_CONN_SEND => { - let bytes = tlv_parse_bytes(slice(args, args_len)).unwrap_or_default(); - if let Some(conn) = SOCK_CONNS.lock().unwrap().get(&id) { + let bytes = tlv::tlv_parse_bytes(slice(args, args_len)).unwrap_or_default(); + if let Some(conn) = state::SOCK_CONNS.lock().unwrap().get(&id) { if let Ok(mut s) = conn.stream.lock() { let _ = s.write_all(&bytes); } netlog!("sock:send id={} n={}", id, bytes.len()); - return write_tlv_void(res, res_len); + return tlv::write_tlv_void(res, res_len); } E_INV_HANDLE } M_CONN_RECV => { - if let Some(conn) = SOCK_CONNS.lock().unwrap().get(&id) { + if let Some(conn) = state::SOCK_CONNS.lock().unwrap().get(&id) { if let Ok(mut s) = conn.stream.lock() { let mut buf = vec![0u8; 4096]; match s.read(&mut buf) { Ok(n) => { buf.truncate(n); netlog!("sock:recv id={} n={}", id, n); - return write_tlv_bytes(&buf, res, res_len); + return tlv::write_tlv_bytes(&buf, res, res_len); } - Err(_) => return write_tlv_bytes(&[], res, res_len), + Err(_) => return tlv::write_tlv_bytes(&[], res, res_len), } } } E_INV_HANDLE } M_CONN_RECV_TIMEOUT => { - let timeout_ms = tlv_parse_i32(slice(args, args_len)).unwrap_or(0).max(0) as u64; - if let Some(conn) = SOCK_CONNS.lock().unwrap().get(&id) { + let timeout_ms = tlv::tlv_parse_i32(slice(args, args_len)) + .unwrap_or(0) + .max(0) as u64; + if let Some(conn) = state::SOCK_CONNS.lock().unwrap().get(&id) { if let Ok(mut s) = conn.stream.lock() { let _ = s.set_read_timeout(Some(Duration::from_millis(timeout_ms))); let mut buf = vec![0u8; 4096]; @@ -1838,7 +1644,7 @@ unsafe fn sock_conn_invoke( Ok(n) => { buf.truncate(n); netlog!("sock:recvTimeout id={} n={} ms={}", id, n, timeout_ms); - return write_tlv_bytes(&buf, res, res_len); + return tlv::write_tlv_bytes(&buf, res, res_len); } Err(e) => { netlog!( @@ -1856,9 +1662,10 @@ unsafe fn sock_conn_invoke( } M_CONN_CLOSE => { // Drop the stream by removing entry - SOCK_CONNS.lock().unwrap().remove(&id); - write_tlv_void(res, res_len) + state::SOCK_CONNS.lock().unwrap().remove(&id); + tlv::write_tlv_void(res, res_len) } _ => E_INV_METHOD, } } +mod state; diff --git a/plugins/nyash-net-plugin/src/state.rs b/plugins/nyash-net-plugin/src/state.rs new file mode 100644 index 00000000..cba15c76 --- /dev/null +++ b/plugins/nyash-net-plugin/src/state.rs @@ -0,0 +1,78 @@ +use once_cell::sync::Lazy; +use std::collections::HashMap; +use std::sync::{ + atomic::{AtomicU32, Ordering}, + Mutex, +}; + +use super::{ + ClientState, RequestState, ResponseState, ServerState, SockClientState, SockConnState, + SockServerState, +}; + +pub(crate) static SERVER_INSTANCES: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); +pub(crate) static SERVER_START_SEQ: AtomicU32 = AtomicU32::new(1); +pub(crate) static ACTIVE_SERVER_ID: Lazy>> = Lazy::new(|| Mutex::new(None)); +pub(crate) static LAST_ACCEPTED_REQ: Lazy>> = Lazy::new(|| Mutex::new(None)); +pub(crate) static REQUESTS: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); +pub(crate) static RESPONSES: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); +pub(crate) static CLIENTS: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); + +pub(crate) static SERVER_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static REQUEST_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static RESPONSE_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static CLIENT_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static SOCK_SERVER_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static SOCK_CONN_ID: AtomicU32 = AtomicU32::new(1); +pub(crate) static SOCK_CLIENT_ID: AtomicU32 = AtomicU32::new(1); + +pub(crate) static SOCK_SERVERS: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); +pub(crate) static SOCK_CONNS: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); +pub(crate) static SOCK_CLIENTS: Lazy>> = + Lazy::new(|| Mutex::new(HashMap::new())); + +#[inline] +pub(crate) fn next_server_id() -> u32 { + SERVER_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_server_start_seq() -> u32 { + SERVER_START_SEQ.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_request_id() -> u32 { + REQUEST_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_response_id() -> u32 { + RESPONSE_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_client_id() -> u32 { + CLIENT_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_sock_server_id() -> u32 { + SOCK_SERVER_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_sock_conn_id() -> u32 { + SOCK_CONN_ID.fetch_add(1, Ordering::Relaxed) +} + +#[inline] +pub(crate) fn next_sock_client_id() -> u32 { + SOCK_CLIENT_ID.fetch_add(1, Ordering::Relaxed) +} diff --git a/plugins/nyash-net-plugin/src/tlv.rs b/plugins/nyash-net-plugin/src/tlv.rs new file mode 100644 index 00000000..5335a263 --- /dev/null +++ b/plugins/nyash-net-plugin/src/tlv.rs @@ -0,0 +1,181 @@ +use super::{E_INV_ARGS, E_SHORT, OK}; + +#[inline] +fn tlv_result_size(payloads: &[(u8, &[u8])]) -> usize { + 4 + payloads.iter().map(|(_, p)| 4 + p.len()).sum::() +} + +#[inline] +pub fn ensure_result_capacity(res: *mut u8, res_len: *mut usize, need: usize) -> Result<(), i32> { + if res_len.is_null() { + return Err(E_INV_ARGS); + } + unsafe { + if res.is_null() || *res_len < need { + *res_len = need; + return Err(E_SHORT); + } + } + Ok(()) +} + +#[inline] +unsafe fn write_bytes_unchecked(bytes: &[u8], res: *mut u8, res_len: *mut usize) { + std::ptr::copy_nonoverlapping(bytes.as_ptr(), res, bytes.len()); + *res_len = bytes.len(); +} + +pub fn write_u32(v: u32, res: *mut u8, res_len: *mut usize) -> i32 { + let bytes = v.to_le_bytes(); + if let Err(err) = ensure_result_capacity(res, res_len, bytes.len()) { + return err; + } + unsafe { + write_bytes_unchecked(&bytes, res, res_len); + } + OK +} + +pub fn write_tlv_result(payloads: &[(u8, &[u8])], res: *mut u8, res_len: *mut usize) -> i32 { + let need = tlv_result_size(payloads); + if let Err(err) = ensure_result_capacity(res, res_len, need) { + return err; + } + let mut buf = Vec::with_capacity(need); + buf.extend_from_slice(&1u16.to_le_bytes()); + buf.extend_from_slice(&(payloads.len() as u16).to_le_bytes()); + for (tag, p) in payloads { + buf.push(*tag); + buf.push(0); + buf.extend_from_slice(&(p.len() as u16).to_le_bytes()); + buf.extend_from_slice(p); + } + unsafe { + write_bytes_unchecked(&buf, res, res_len); + } + OK +} + +pub fn write_tlv_void(res: *mut u8, res_len: *mut usize) -> i32 { + write_tlv_result(&[(9u8, &[])], res, res_len) +} +pub fn write_tlv_string(s: &str, res: *mut u8, res_len: *mut usize) -> i32 { + write_tlv_result(&[(6u8, s.as_bytes())], res, res_len) +} +pub fn write_tlv_bytes(b: &[u8], res: *mut u8, res_len: *mut usize) -> i32 { + write_tlv_result(&[(7u8, b)], res, res_len) +} +pub fn write_tlv_i32(v: i32, res: *mut u8, res_len: *mut usize) -> i32 { + write_tlv_result(&[(2u8, &v.to_le_bytes())], res, res_len) +} +pub fn write_tlv_handle(t: u32, id: u32, res: *mut u8, res_len: *mut usize) -> i32 { + let mut payload = [0u8; 8]; + payload[0..4].copy_from_slice(&t.to_le_bytes()); + payload[4..8].copy_from_slice(&id.to_le_bytes()); + write_tlv_result(&[(8u8, &payload)], res, res_len) +} + +pub fn tlv_parse_header(data: &[u8]) -> Result<(u16, u16, usize), ()> { + if data.len() < 4 { + return Err(()); + } + let ver = u16::from_le_bytes([data[0], data[1]]); + let argc = u16::from_le_bytes([data[2], data[3]]); + if ver != 1 { + return Err(()); + } + Ok((ver, argc, 4)) +} +pub fn tlv_parse_string(data: &[u8]) -> Result { + let (_, argc, pos) = tlv_parse_header(data)?; + if argc < 1 { + return Err(()); + } + let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; + if tag != 6 { + return Err(()); + } + Ok(std::str::from_utf8(&data[p..p + size]) + .map_err(|_| ())? + .to_string()) +} +pub fn tlv_parse_two_strings(data: &[u8]) -> Result<(String, String), ()> { + let (_, argc, mut pos) = tlv_parse_header(data)?; + if argc < 2 { + return Err(()); + } + let (tag1, size1, p1) = tlv_parse_entry_hdr(data, pos)?; + if tag1 != 6 { + return Err(()); + } + let s1 = std::str::from_utf8(&data[p1..p1 + size1]) + .map_err(|_| ())? + .to_string(); + pos = p1 + size1; + let (tag2, size2, p2) = tlv_parse_entry_hdr(data, pos)?; + if tag2 != 6 { + return Err(()); + } + let s2 = std::str::from_utf8(&data[p2..p2 + size2]) + .map_err(|_| ())? + .to_string(); + Ok((s1, s2)) +} +pub fn tlv_parse_bytes(data: &[u8]) -> Result, ()> { + let (_, argc, pos) = tlv_parse_header(data)?; + if argc < 1 { + return Err(()); + } + let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; + if tag != 6 && tag != 7 { + return Err(()); + } + Ok(data[p..p + size].to_vec()) +} +pub fn tlv_parse_i32(data: &[u8]) -> Result { + let (_, argc, pos) = tlv_parse_header(data)?; + if argc < 1 { + return Err(()); + } + let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; + match (tag, size) { + (2, 4) => { + let mut b = [0u8; 4]; + b.copy_from_slice(&data[p..p + 4]); + Ok(i32::from_le_bytes(b)) + } + (5, 8) => { + let mut b = [0u8; 8]; + b.copy_from_slice(&data[p..p + 8]); + Ok(i64::from_le_bytes(b) as i32) + } + _ => Err(()), + } +} +pub fn tlv_parse_handle(data: &[u8]) -> Result<(u32, u32), ()> { + let (_, argc, pos) = tlv_parse_header(data)?; + if argc < 1 { + return Err(()); + } + let (tag, size, p) = tlv_parse_entry_hdr(data, pos)?; + if tag != 8 || size != 8 { + return Err(()); + } + let mut t = [0u8; 4]; + let mut i = [0u8; 4]; + t.copy_from_slice(&data[p..p + 4]); + i.copy_from_slice(&data[p + 4..p + 8]); + Ok((u32::from_le_bytes(t), u32::from_le_bytes(i))) +} +pub fn tlv_parse_entry_hdr(data: &[u8], pos: usize) -> Result<(u8, usize, usize), ()> { + if pos + 4 > data.len() { + return Err(()); + } + let tag = data[pos]; + let size = u16::from_le_bytes([data[pos + 2], data[pos + 3]]) as usize; + let p = pos + 4; + if p + size > data.len() { + return Err(()); + } + Ok((tag, size, p)) +} diff --git a/plugins/nyash-path-plugin/src/lib.rs b/plugins/nyash-path-plugin/src/lib.rs index f0d0b619..c9e6c02b 100644 --- a/plugins/nyash-path-plugin/src/lib.rs +++ b/plugins/nyash-path-plugin/src/lib.rs @@ -150,9 +150,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, // C string pub resolve: Option u32>, pub invoke_id: Option i32>, diff --git a/plugins/nyash-python-compiler-plugin/src/lib.rs b/plugins/nyash-python-compiler-plugin/src/lib.rs index 6bf1afd2..13cd8d73 100644 --- a/plugins/nyash-python-compiler-plugin/src/lib.rs +++ b/plugins/nyash-python-compiler-plugin/src/lib.rs @@ -123,9 +123,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -135,7 +135,9 @@ unsafe impl Sync for NyashTypeBoxFfi {} use std::ffi::CStr; extern "C" fn pycompiler_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "birth" => METHOD_BIRTH, @@ -156,52 +158,81 @@ extern "C" fn pycompiler_invoke_id( match method_id { METHOD_BIRTH => unsafe { let mut id_g = NEXT_ID.lock().unwrap(); - let id = *id_g; *id_g += 1; - if result_len.is_null() { return NYB_E_SHORT_BUFFER; } + let id = *id_g; + *id_g += 1; + if result_len.is_null() { + return NYB_E_SHORT_BUFFER; + } let need = 4usize; - if *result_len < need { *result_len = need; return NYB_E_SHORT_BUFFER; } + if *result_len < need { + *result_len = need; + return NYB_E_SHORT_BUFFER; + } let out = std::slice::from_raw_parts_mut(result, *result_len); out[0..4].copy_from_slice(&(id as u32).to_le_bytes()); *result_len = need; NYB_SUCCESS }, METHOD_COMPILE => unsafe { - let ir = if args.is_null() || args_len < 8 { None } else { + let ir = if args.is_null() || args_len < 8 { + None + } else { let buf = std::slice::from_raw_parts(args, args_len); let tag = u16::from_le_bytes([buf[4], buf[5]]); let len = u16::from_le_bytes([buf[6], buf[7]]) as usize; if tag == 6 && 8 + len <= buf.len() { - std::str::from_utf8(&buf[8..8+len]).ok().map(|s| s.to_string()) - } else { None } + std::str::from_utf8(&buf[8..8 + len]) + .ok() + .map(|s| s.to_string()) + } else { + None + } }; let nyash_source = if let Some(s) = ir.or_else(|| std::env::var("NYASH_PY_IR").ok()) { match serde_json::from_str::(&s).ok() { Some(Json::Object(map)) => { - if let Some(Json::String(src)) = map.get("nyash_source") { src.clone() } - else if let Some(module) = map.get("module") { + if let Some(Json::String(src)) = map.get("nyash_source") { + src.clone() + } else if let Some(module) = map.get("module") { let mut ret_expr = "0".to_string(); - if let Some(funcs) = module.get("functions").and_then(|v| v.as_array()) { + if let Some(funcs) = module.get("functions").and_then(|v| v.as_array()) + { if let Some(fun0) = funcs.get(0) { if let Some(retv) = fun0.get("return_value") { - if retv.is_number() { ret_expr = retv.to_string(); } - else if let Some(s) = retv.as_str() { ret_expr = s.to_string(); } + if retv.is_number() { + ret_expr = retv.to_string(); + } else if let Some(s) = retv.as_str() { + ret_expr = s.to_string(); + } } } } - format!("static box Generated {\n main() {\n return {}\n }\n}}", ret_expr) - } else { "static box Generated { main() { return 0 } }".to_string() } + format!( + "static box Generated {\n main() {\n return {}\n }\n}}", + ret_expr + ) + } else { + "static box Generated { main() { return 0 } }".to_string() + } } _ => "static box Generated { main() { return 0 } }".to_string(), } - } else { "static box Generated { main() { return 0 } }".to_string() }; + } else { + "static box Generated { main() { return 0 } }".to_string() + }; let bytes = nyash_source.as_bytes(); - if result_len.is_null() { return NYB_E_SHORT_BUFFER; } + if result_len.is_null() { + return NYB_E_SHORT_BUFFER; + } let need = 4 + bytes.len(); - if *result_len < need { *result_len = need; return NYB_E_SHORT_BUFFER; } + if *result_len < need { + *result_len = need; + return NYB_E_SHORT_BUFFER; + } let out = std::slice::from_raw_parts_mut(result, *result_len); out[0..2].copy_from_slice(&6u16.to_le_bytes()); out[2..4].copy_from_slice(&(bytes.len() as u16).to_le_bytes()); - out[4..4+bytes.len()].copy_from_slice(bytes); + out[4..4 + bytes.len()].copy_from_slice(bytes); *result_len = need; NYB_SUCCESS }, diff --git a/plugins/nyash-python-parser-plugin/src/lib.rs b/plugins/nyash-python-parser-plugin/src/lib.rs index b4280877..f1a8c11b 100644 --- a/plugins/nyash-python-parser-plugin/src/lib.rs +++ b/plugins/nyash-python-parser-plugin/src/lib.rs @@ -105,9 +105,9 @@ fn parse_python_code(py: Python, code: &str) -> ParseResult { // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -122,7 +122,9 @@ const METHOD_FINI: u32 = u32::MAX; use std::ffi::CStr; extern "C" fn pyparser_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "birth" => METHOD_BIRTH, @@ -143,7 +145,9 @@ extern "C" fn pyparser_invoke_id( match method_id { METHOD_BIRTH => unsafe { let instance_id = 1u32; // simple singleton - if result_len.is_null() { return -1; } + if result_len.is_null() { + return -1; + } if *result_len < 4 { *result_len = 4; return -1; @@ -157,26 +161,35 @@ extern "C" fn pyparser_invoke_id( // Decode TLV string from args if present, else env fallback let code = unsafe { if args.is_null() || args_len < 4 { - std::env::var("NYASH_PY_CODE").unwrap_or_else(|_| "def main():\n return 0".to_string()) + std::env::var("NYASH_PY_CODE") + .unwrap_or_else(|_| "def main():\n return 0".to_string()) } else { let buf = std::slice::from_raw_parts(args, args_len); if args_len >= 8 { let tag = u16::from_le_bytes([buf[0], buf[1]]); let len = u16::from_le_bytes([buf[2], buf[3]]) as usize; if tag == 6 && 4 + len <= args_len { - match std::str::from_utf8(&buf[4..4 + len]) { Ok(s) => s.to_string(), Err(_) => std::env::var("NYASH_PY_CODE").unwrap_or_else(|_| "def main():\n return 0".to_string()) } + match std::str::from_utf8(&buf[4..4 + len]) { + Ok(s) => s.to_string(), + Err(_) => std::env::var("NYASH_PY_CODE") + .unwrap_or_else(|_| "def main():\n return 0".to_string()), + } } else { - std::env::var("NYASH_PY_CODE").unwrap_or_else(|_| "def main():\n return 0".to_string()) + std::env::var("NYASH_PY_CODE") + .unwrap_or_else(|_| "def main():\n return 0".to_string()) } } else { - std::env::var("NYASH_PY_CODE").unwrap_or_else(|_| "def main():\n return 0".to_string()) + std::env::var("NYASH_PY_CODE") + .unwrap_or_else(|_| "def main():\n return 0".to_string()) } } }; let parse_result = Python::with_gil(|py| parse_python_code(py, &code)); match serde_json::to_string(&parse_result) { Ok(json) => unsafe { - if result_len.is_null() { return -1; } + if result_len.is_null() { + return -1; + } let bytes = json.as_bytes(); let need = 4 + bytes.len(); if *result_len < need { diff --git a/plugins/nyash-python-plugin/src/ffi.rs b/plugins/nyash-python-plugin/src/ffi.rs new file mode 100644 index 00000000..3888c62c --- /dev/null +++ b/plugins/nyash-python-plugin/src/ffi.rs @@ -0,0 +1,284 @@ +#![allow(non_snake_case, non_camel_case_types, dead_code)] +use libloading::Library; +use once_cell::sync::Lazy; +use std::os::raw::{c_char, c_int, c_long, c_void}; +use std::sync::Mutex; + +pub type PyObject = c_void; +pub type PyGILState_STATE = c_int; + +pub struct CPython { + pub(crate) _lib: Library, + pub(crate) Py_Initialize: unsafe extern "C" fn(), + pub(crate) Py_Finalize: unsafe extern "C" fn(), + pub(crate) Py_IsInitialized: unsafe extern "C" fn() -> c_int, + pub(crate) PyGILState_Ensure: unsafe extern "C" fn() -> PyGILState_STATE, + pub(crate) PyGILState_Release: unsafe extern "C" fn(PyGILState_STATE), + pub(crate) PyRun_StringFlags: unsafe extern "C" fn( + *const c_char, + c_int, + *mut PyObject, + *mut PyObject, + *mut c_void, + ) -> *mut PyObject, + pub(crate) PyImport_AddModule: unsafe extern "C" fn(*const c_char) -> *mut PyObject, + pub(crate) PyModule_GetDict: unsafe extern "C" fn(*mut PyObject) -> *mut PyObject, + pub(crate) PyImport_ImportModule: unsafe extern "C" fn(*const c_char) -> *mut PyObject, + pub(crate) PyObject_Str: unsafe extern "C" fn(*mut PyObject) -> *mut PyObject, + pub(crate) PyUnicode_AsUTF8: unsafe extern "C" fn(*mut PyObject) -> *const c_char, + pub(crate) Py_DecRef: unsafe extern "C" fn(*mut PyObject), + pub(crate) Py_IncRef: unsafe extern "C" fn(*mut PyObject), + pub(crate) PyObject_GetAttrString: + unsafe extern "C" fn(*mut PyObject, *const c_char) -> *mut PyObject, + pub(crate) PyObject_CallObject: + unsafe extern "C" fn(*mut PyObject, *mut PyObject) -> *mut PyObject, + pub(crate) PyObject_Call: + unsafe extern "C" fn(*mut PyObject, *mut PyObject, *mut PyObject) -> *mut PyObject, + pub(crate) PyTuple_New: unsafe extern "C" fn(isize) -> *mut PyObject, + pub(crate) PyTuple_SetItem: unsafe extern "C" fn(*mut PyObject, isize, *mut PyObject) -> c_int, + pub(crate) PyLong_FromLongLong: unsafe extern "C" fn(i64) -> *mut PyObject, + pub(crate) PyUnicode_FromString: unsafe extern "C" fn(*const c_char) -> *mut PyObject, + pub(crate) PyBool_FromLong: unsafe extern "C" fn(c_long: c_long) -> *mut PyObject, + pub(crate) PyFloat_FromDouble: unsafe extern "C" fn(f64) -> *mut PyObject, + pub(crate) PyFloat_AsDouble: unsafe extern "C" fn(*mut PyObject) -> f64, + pub(crate) PyLong_AsLongLong: unsafe extern "C" fn(*mut PyObject) -> i64, + pub(crate) PyBytes_FromStringAndSize: + unsafe extern "C" fn(*const c_char, isize) -> *mut PyObject, + pub(crate) PyBytes_AsStringAndSize: + unsafe extern "C" fn(*mut PyObject, *mut *mut c_char, *mut isize) -> c_int, + pub(crate) PyDict_New: unsafe extern "C" fn() -> *mut PyObject, + pub(crate) PyDict_SetItemString: + unsafe extern "C" fn(*mut PyObject, *const c_char, *mut PyObject) -> c_int, + pub(crate) PyErr_Occurred: unsafe extern "C" fn() -> *mut PyObject, + pub(crate) PyErr_Fetch: + unsafe extern "C" fn(*mut *mut PyObject, *mut *mut PyObject, *mut *mut PyObject), + pub(crate) PyErr_Clear: unsafe extern "C" fn(), +} + +pub static CPY: Lazy>> = Lazy::new(|| Mutex::new(None)); + +pub fn try_load_cpython() -> Result<(), ()> { + let mut candidates: Vec = vec![ + // Linux + "libpython3.12.so".into(), + "libpython3.12.so.1.0".into(), + "libpython3.11.so".into(), + "libpython3.11.so.1.0".into(), + "libpython3.10.so".into(), + "libpython3.10.so.1.0".into(), + "libpython3.9.so".into(), + "libpython3.9.so.1.0".into(), + // macOS + "libpython3.12.dylib".into(), + "libpython3.11.dylib".into(), + "libpython3.10.dylib".into(), + "libpython3.9.dylib".into(), + ]; + if cfg!(target_os = "windows") { + let dlls = [ + "python312.dll", + "python311.dll", + "python310.dll", + "python39.dll", + ]; + for d in dlls { + candidates.push(d.into()); + } + if let Ok(pyhome) = std::env::var("PYTHONHOME") { + for d in [ + "python312.dll", + "python311.dll", + "python310.dll", + "python39.dll", + ] + .iter() + { + let p = std::path::Path::new(&pyhome).join(d); + if p.exists() { + candidates.push(p.to_string_lossy().to_string()); + } + } + } + } + for name in candidates.into_iter() { + if let Ok(lib) = unsafe { Library::new(&name) } { + unsafe { + let Py_Initialize = *lib + .get::(b"Py_Initialize\0") + .map_err(|_| ())?; + let Py_Finalize = *lib + .get::(b"Py_Finalize\0") + .map_err(|_| ())?; + let Py_IsInitialized = *lib + .get:: c_int>(b"Py_IsInitialized\0") + .map_err(|_| ())?; + let PyGILState_Ensure = *lib + .get:: PyGILState_STATE>(b"PyGILState_Ensure\0") + .map_err(|_| ())?; + let PyGILState_Release = *lib + .get::(b"PyGILState_Release\0") + .map_err(|_| ())?; + let PyRun_StringFlags = *lib + .get:: *mut PyObject>(b"PyRun_StringFlags\0") + .map_err(|_| ())?; + let PyImport_AddModule = *lib + .get:: *mut PyObject>( + b"PyImport_AddModule\0", + ) + .map_err(|_| ())?; + let PyModule_GetDict = *lib + .get:: *mut PyObject>( + b"PyModule_GetDict\0", + ) + .map_err(|_| ())?; + let PyImport_ImportModule = *lib + .get:: *mut PyObject>( + b"PyImport_ImportModule\0", + ) + .map_err(|_| ())?; + let PyObject_Str = *lib + .get:: *mut PyObject>(b"PyObject_Str\0") + .map_err(|_| ())?; + let PyUnicode_AsUTF8 = *lib + .get:: *const c_char>( + b"PyUnicode_AsUTF8\0", + ) + .map_err(|_| ())?; + let Py_DecRef = *lib + .get::(b"Py_DecRef\0") + .map_err(|_| ())?; + let Py_IncRef = *lib + .get::(b"Py_IncRef\0") + .map_err(|_| ())?; + let PyObject_GetAttrString = *lib + .get:: *mut PyObject>( + b"PyObject_GetAttrString\0", + ) + .map_err(|_| ())?; + let PyObject_CallObject = *lib + .get:: *mut PyObject>( + b"PyObject_CallObject\0", + ) + .map_err(|_| ())?; + let PyObject_Call = *lib + .get:: *mut PyObject>(b"PyObject_Call\0") + .map_err(|_| ())?; + let PyTuple_New = *lib + .get:: *mut PyObject>(b"PyTuple_New\0") + .map_err(|_| ())?; + let PyTuple_SetItem = *lib + .get:: c_int>( + b"PyTuple_SetItem\0", + ) + .map_err(|_| ())?; + let PyLong_FromLongLong = *lib + .get:: *mut PyObject>(b"PyLong_FromLongLong\0") + .map_err(|_| ())?; + let PyUnicode_FromString = *lib + .get:: *mut PyObject>( + b"PyUnicode_FromString\0", + ) + .map_err(|_| ())?; + let PyBool_FromLong = *lib + .get:: *mut PyObject>( + b"PyBool_FromLong\0", + ) + .map_err(|_| ())?; + let PyFloat_FromDouble = *lib + .get:: *mut PyObject>(b"PyFloat_FromDouble\0") + .map_err(|_| ())?; + let PyFloat_AsDouble = *lib + .get:: f64>(b"PyFloat_AsDouble\0") + .map_err(|_| ())?; + let PyLong_AsLongLong = *lib + .get:: i64>(b"PyLong_AsLongLong\0") + .map_err(|_| ())?; + let PyBytes_FromStringAndSize = *lib + .get:: *mut PyObject>( + b"PyBytes_FromStringAndSize\0", + ) + .map_err(|_| ())?; + let PyBytes_AsStringAndSize = *lib.get:: c_int>(b"PyBytes_AsStringAndSize\0").map_err(|_| ())?; + let PyDict_New = *lib + .get:: *mut PyObject>(b"PyDict_New\0") + .map_err(|_| ())?; + let PyDict_SetItemString = *lib.get:: c_int>(b"PyDict_SetItemString\0").map_err(|_| ())?; + let PyErr_Occurred = *lib + .get:: *mut PyObject>(b"PyErr_Occurred\0") + .map_err(|_| ())?; + let PyErr_Fetch = + *lib.get::(b"PyErr_Fetch\0") + .map_err(|_| ())?; + let PyErr_Clear = *lib + .get::(b"PyErr_Clear\0") + .map_err(|_| ())?; + + let cpy = CPython { + _lib: lib, + Py_Initialize, + Py_Finalize, + Py_IsInitialized, + PyGILState_Ensure, + PyGILState_Release, + PyRun_StringFlags, + PyImport_AddModule, + PyModule_GetDict, + PyImport_ImportModule, + PyObject_Str, + PyUnicode_AsUTF8, + Py_DecRef, + Py_IncRef, + PyObject_GetAttrString, + PyObject_CallObject, + PyObject_Call, + PyTuple_New, + PyTuple_SetItem, + PyLong_FromLongLong, + PyUnicode_FromString, + PyBool_FromLong, + PyFloat_FromDouble, + PyFloat_AsDouble, + PyLong_AsLongLong, + PyBytes_FromStringAndSize, + PyBytes_AsStringAndSize, + PyDict_New, + PyDict_SetItemString, + PyErr_Occurred, + PyErr_Fetch, + PyErr_Clear, + }; + *CPY.lock().unwrap() = Some(cpy); + return Ok(()); + } + } + } + Err(()) +} + +pub fn ensure_cpython() -> Result<(), ()> { + if CPY.lock().unwrap().is_none() { + try_load_cpython()?; + unsafe { + if let Some(cpy) = &*CPY.lock().unwrap() { + if (cpy.Py_IsInitialized)() == 0 { + (cpy.Py_Initialize)(); + } + } + } + } + Ok(()) +} diff --git a/plugins/nyash-python-plugin/src/gil.rs b/plugins/nyash-python-plugin/src/gil.rs new file mode 100644 index 00000000..3a30cacc --- /dev/null +++ b/plugins/nyash-python-plugin/src/gil.rs @@ -0,0 +1,19 @@ +use crate::ffi::{CPython, PyGILState_STATE}; + +pub struct GILGuard<'a> { + cpy: &'a CPython, + state: PyGILState_STATE, +} + +impl<'a> GILGuard<'a> { + pub fn acquire(cpy: &'a CPython) -> Self { + let state = unsafe { (cpy.PyGILState_Ensure)() }; + GILGuard { cpy, state } + } +} + +impl<'a> Drop for GILGuard<'a> { + fn drop(&mut self) { + unsafe { (self.cpy.PyGILState_Release)(self.state) }; + } +} diff --git a/plugins/nyash-python-plugin/src/lib.rs b/plugins/nyash-python-plugin/src/lib.rs index 2b13058c..355db1dc 100644 --- a/plugins/nyash-python-plugin/src/lib.rs +++ b/plugins/nyash-python-plugin/src/lib.rs @@ -2,11 +2,8 @@ //! - ABI v1 compatible entry points + ABI v2 TypeBox exports //! - Two Box types: PyRuntimeBox (TYPE_ID=40) and PyObjectBox (TYPE_ID=41) -use libloading::Library; use once_cell::sync::Lazy; use std::collections::HashMap; -use std::ffi::{CStr, CString}; -use std::os::raw::{c_char, c_int, c_long, c_void}; use std::sync::{ atomic::{AtomicU32, Ordering}, Mutex, @@ -15,14 +12,14 @@ use std::sync::{ // ===== Error Codes (aligned with other plugins) ===== const NYB_SUCCESS: i32 = 0; const NYB_E_SHORT_BUFFER: i32 = -1; -const NYB_E_INVALID_TYPE: i32 = -2; +const _NYB_E_INVALID_TYPE: i32 = -2; const NYB_E_INVALID_METHOD: i32 = -3; const NYB_E_INVALID_ARGS: i32 = -4; const NYB_E_PLUGIN_ERROR: i32 = -5; const NYB_E_INVALID_HANDLE: i32 = -8; // ===== Type IDs (must match nyash.toml) ===== -const TYPE_ID_PY_RUNTIME: u32 = 40; +const _TYPE_ID_PY_RUNTIME: u32 = 40; const TYPE_ID_PY_OBJECT: u32 = 41; // ===== Method IDs (initial draft) ===== @@ -70,278 +67,14 @@ static PYOBJS: Lazy>> = static RT_COUNTER: AtomicU32 = AtomicU32::new(1); static OBJ_COUNTER: AtomicU32 = AtomicU32::new(1); -// ====== Minimal dynamic CPython loader ====== -type PyObject = c_void; -type PyGILState_STATE = c_int; +// ====== CPython FFI and GIL guard ====== +mod ffi; +mod gil; +mod pytypes; +use ffi::{ensure_cpython, PyObject}; +use pytypes::{DecodedValue, PyOwned}; -struct CPython { - _lib: Library, - Py_Initialize: unsafe extern "C" fn(), - Py_Finalize: unsafe extern "C" fn(), - Py_IsInitialized: unsafe extern "C" fn() -> c_int, - PyGILState_Ensure: unsafe extern "C" fn() -> PyGILState_STATE, - PyGILState_Release: unsafe extern "C" fn(PyGILState_STATE), - PyRun_StringFlags: unsafe extern "C" fn( - *const c_char, - c_int, - *mut PyObject, - *mut PyObject, - *mut c_void, - ) -> *mut PyObject, - PyImport_AddModule: unsafe extern "C" fn(*const c_char) -> *mut PyObject, - PyModule_GetDict: unsafe extern "C" fn(*mut PyObject) -> *mut PyObject, - PyImport_ImportModule: unsafe extern "C" fn(*const c_char) -> *mut PyObject, - PyObject_Str: unsafe extern "C" fn(*mut PyObject) -> *mut PyObject, - PyUnicode_AsUTF8: unsafe extern "C" fn(*mut PyObject) -> *const c_char, - Py_DecRef: unsafe extern "C" fn(*mut PyObject), - Py_IncRef: unsafe extern "C" fn(*mut PyObject), - // Added for getattr/call - PyObject_GetAttrString: unsafe extern "C" fn(*mut PyObject, *const c_char) -> *mut PyObject, - PyObject_CallObject: unsafe extern "C" fn(*mut PyObject, *mut PyObject) -> *mut PyObject, - PyObject_Call: - unsafe extern "C" fn(*mut PyObject, *mut PyObject, *mut PyObject) -> *mut PyObject, - PyTuple_New: unsafe extern "C" fn(isize) -> *mut PyObject, - PyTuple_SetItem: unsafe extern "C" fn(*mut PyObject, isize, *mut PyObject) -> c_int, - PyLong_FromLongLong: unsafe extern "C" fn(i64) -> *mut PyObject, - PyUnicode_FromString: unsafe extern "C" fn(*const c_char) -> *mut PyObject, - PyBool_FromLong: unsafe extern "C" fn(c_long: c_long) -> *mut PyObject, - // Added for float/bytes and error handling - PyFloat_FromDouble: unsafe extern "C" fn(f64) -> *mut PyObject, - PyFloat_AsDouble: unsafe extern "C" fn(*mut PyObject) -> f64, - PyLong_AsLongLong: unsafe extern "C" fn(*mut PyObject) -> i64, - PyBytes_FromStringAndSize: unsafe extern "C" fn(*const c_char, isize) -> *mut PyObject, - PyBytes_AsStringAndSize: - unsafe extern "C" fn(*mut PyObject, *mut *mut c_char, *mut isize) -> c_int, - PyDict_New: unsafe extern "C" fn() -> *mut PyObject, - PyDict_SetItemString: - unsafe extern "C" fn(*mut PyObject, *const c_char, *mut PyObject) -> c_int, - PyErr_Occurred: unsafe extern "C" fn() -> *mut PyObject, - PyErr_Fetch: unsafe extern "C" fn(*mut *mut PyObject, *mut *mut PyObject, *mut *mut PyObject), - PyErr_Clear: unsafe extern "C" fn(), -} - -static CPY: Lazy>> = Lazy::new(|| Mutex::new(None)); - -fn try_load_cpython() -> Result<(), ()> { - let mut candidates: Vec = vec![ - // Linux/WSL common - "libpython3.12.so".into(), - "libpython3.12.so.1.0".into(), - "libpython3.11.so".into(), - "libpython3.11.so.1.0".into(), - "libpython3.10.so".into(), - "libpython3.10.so.1.0".into(), - "libpython3.9.so".into(), - "libpython3.9.so.1.0".into(), - // macOS - "libpython3.12.dylib".into(), - "libpython3.11.dylib".into(), - "libpython3.10.dylib".into(), - "libpython3.9.dylib".into(), - ]; - // Windows DLLs (search via PATH / System32) - if cfg!(target_os = "windows") { - let dlls = [ - "python312.dll", - "python311.dll", - "python310.dll", - "python39.dll", - ]; - for d in dlls.iter() { - candidates.push((*d).into()); - } - if let Ok(pyhome) = std::env::var("PYTHONHOME") { - for d in dlls.iter() { - let p = std::path::Path::new(&pyhome).join(d); - if p.exists() { - candidates.push(p.to_string_lossy().to_string()); - } - } - } - } - for name in candidates.into_iter() { - if let Ok(lib) = unsafe { Library::new(&name) } { - unsafe { - let Py_Initialize = *lib - .get::(b"Py_Initialize\0") - .map_err(|_| ())?; - let Py_Finalize = *lib - .get::(b"Py_Finalize\0") - .map_err(|_| ())?; - let Py_IsInitialized = *lib - .get:: c_int>(b"Py_IsInitialized\0") - .map_err(|_| ())?; - let PyGILState_Ensure = *lib - .get:: PyGILState_STATE>(b"PyGILState_Ensure\0") - .map_err(|_| ())?; - let PyGILState_Release = *lib - .get::(b"PyGILState_Release\0") - .map_err(|_| ())?; - let PyRun_StringFlags = *lib - .get:: *mut PyObject>(b"PyRun_StringFlags\0") - .map_err(|_| ())?; - let PyImport_AddModule = *lib - .get:: *mut PyObject>( - b"PyImport_AddModule\0", - ) - .map_err(|_| ())?; - let PyModule_GetDict = *lib - .get:: *mut PyObject>( - b"PyModule_GetDict\0", - ) - .map_err(|_| ())?; - let PyImport_ImportModule = *lib - .get:: *mut PyObject>( - b"PyImport_ImportModule\0", - ) - .map_err(|_| ())?; - let PyObject_Str = *lib - .get:: *mut PyObject>(b"PyObject_Str\0") - .map_err(|_| ())?; - let PyUnicode_AsUTF8 = *lib - .get:: *const c_char>( - b"PyUnicode_AsUTF8\0", - ) - .map_err(|_| ())?; - let Py_DecRef = *lib - .get::(b"Py_DecRef\0") - .map_err(|_| ())?; - let Py_IncRef = *lib - .get::(b"Py_IncRef\0") - .map_err(|_| ())?; - let PyObject_GetAttrString = *lib - .get:: *mut PyObject>( - b"PyObject_GetAttrString\0", - ) - .map_err(|_| ())?; - let PyObject_CallObject = *lib - .get:: *mut PyObject>( - b"PyObject_CallObject\0", - ) - .map_err(|_| ())?; - let PyObject_Call = *lib - .get:: *mut PyObject>(b"PyObject_Call\0") - .map_err(|_| ())?; - let PyTuple_New = *lib - .get:: *mut PyObject>(b"PyTuple_New\0") - .map_err(|_| ())?; - let PyTuple_SetItem = *lib - .get:: c_int>( - b"PyTuple_SetItem\0", - ) - .map_err(|_| ())?; - let PyLong_FromLongLong = *lib - .get:: *mut PyObject>(b"PyLong_FromLongLong\0") - .map_err(|_| ())?; - let PyUnicode_FromString = *lib - .get:: *mut PyObject>( - b"PyUnicode_FromString\0", - ) - .map_err(|_| ())?; - let PyBool_FromLong = *lib - .get:: *mut PyObject>( - b"PyBool_FromLong\0", - ) - .map_err(|_| ())?; - let PyFloat_FromDouble = *lib - .get:: *mut PyObject>(b"PyFloat_FromDouble\0") - .map_err(|_| ())?; - let PyFloat_AsDouble = *lib - .get:: f64>(b"PyFloat_AsDouble\0") - .map_err(|_| ())?; - let PyLong_AsLongLong = *lib - .get:: i64>(b"PyLong_AsLongLong\0") - .map_err(|_| ())?; - let PyBytes_FromStringAndSize = *lib - .get:: *mut PyObject>( - b"PyBytes_FromStringAndSize\0", - ) - .map_err(|_| ())?; - let PyBytes_AsStringAndSize = *lib.get:: c_int>(b"PyBytes_AsStringAndSize\0").map_err(|_| ())?; - let PyErr_Occurred = *lib - .get:: *mut PyObject>(b"PyErr_Occurred\0") - .map_err(|_| ())?; - let PyDict_New = *lib - .get:: *mut PyObject>(b"PyDict_New\0") - .map_err(|_| ())?; - let PyDict_SetItemString = *lib.get:: c_int>(b"PyDict_SetItemString\0").map_err(|_| ())?; - let PyErr_Fetch = - *lib.get::(b"PyErr_Fetch\0") - .map_err(|_| ())?; - let PyErr_Clear = *lib - .get::(b"PyErr_Clear\0") - .map_err(|_| ())?; - - let cpy = CPython { - _lib: lib, - Py_Initialize, - Py_Finalize, - Py_IsInitialized, - PyGILState_Ensure, - PyGILState_Release, - PyRun_StringFlags, - PyImport_AddModule, - PyModule_GetDict, - PyImport_ImportModule, - PyObject_Str, - PyUnicode_AsUTF8, - Py_DecRef, - Py_IncRef, - PyObject_GetAttrString, - PyObject_CallObject, - PyObject_Call, - PyTuple_New, - PyTuple_SetItem, - PyLong_FromLongLong, - PyUnicode_FromString, - PyBool_FromLong, - PyFloat_FromDouble, - PyFloat_AsDouble, - PyLong_AsLongLong, - PyBytes_FromStringAndSize, - PyBytes_AsStringAndSize, - PyErr_Occurred, - PyDict_New, - PyDict_SetItemString, - PyErr_Fetch, - PyErr_Clear, - }; - *CPY.lock().unwrap() = Some(cpy); - return Ok(()); - } - } - } - Err(()) -} - -fn ensure_cpython() -> Result<(), ()> { - if CPY.lock().unwrap().is_none() { - try_load_cpython()?; - // Initialize on first load - unsafe { - if let Some(cpy) = &*CPY.lock().unwrap() { - if (cpy.Py_IsInitialized)() == 0 { - (cpy.Py_Initialize)(); - } - } - } - } - Ok(()) -} +// loader moved to ffi.rs // legacy v1 abi/init removed @@ -390,9 +123,9 @@ fn handle_py_runtime( } let id = RT_COUNTER.fetch_add(1, Ordering::Relaxed); let mut inst = PyRuntimeInstance::default(); - if let Some(cpy) = &*CPY.lock().unwrap() { - let c_main = CString::new("__main__").unwrap(); - let state = (cpy.PyGILState_Ensure)(); + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let c_main = pytypes::cstring_from_str("__main__").expect("literal __main__"); + let _gil = gil::GILGuard::acquire(cpy); let module = (cpy.PyImport_AddModule)(c_main.as_ptr()); if !module.is_null() { let dict = (cpy.PyModule_GetDict)(module); @@ -400,7 +133,6 @@ fn handle_py_runtime( inst.globals = Some(dict); } } - (cpy.PyGILState_Release)(state); } if let Ok(mut map) = RUNTIMES.lock() { map.insert(id, inst); @@ -424,7 +156,7 @@ fn handle_py_runtime( return NYB_E_PLUGIN_ERROR; } // Allow zero-arg eval by reading from env var (NYASH_PY_EVAL_CODE) for bootstrap demos - let argc = tlv_count_args(_args, _args_len); + let argc = pytypes::count_tlv_args(_args, _args_len); let code = if argc == 0 { std::env::var("NYASH_PY_EVAL_CODE").unwrap_or_else(|_| "".to_string()) } else { @@ -434,12 +166,12 @@ fn handle_py_runtime( return NYB_E_INVALID_ARGS; } }; - let c_code = match CString::new(code) { + let c_code = match pytypes::cstring_from_str(&code) { Ok(s) => s, Err(_) => return NYB_E_INVALID_ARGS, }; - if let Some(cpy) = &*CPY.lock().unwrap() { - let state = (cpy.PyGILState_Ensure)(); + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let _gil = gil::GILGuard::acquire(cpy); // use per-runtime globals if available let mut dict: *mut PyObject = std::ptr::null_mut(); if let Ok(map) = RUNTIMES.lock() { @@ -450,10 +182,10 @@ fn handle_py_runtime( } } if dict.is_null() { - let c_main = CString::new("__main__").unwrap(); + let c_main = + pytypes::cstring_from_str("__main__").expect("literal __main__"); let module = (cpy.PyImport_AddModule)(c_main.as_ptr()); if module.is_null() { - (cpy.PyGILState_Release)(state); return NYB_E_PLUGIN_ERROR; } dict = (cpy.PyModule_GetDict)(module); @@ -467,8 +199,7 @@ fn handle_py_runtime( std::ptr::null_mut(), ); if obj.is_null() { - let msg = take_py_error_string(cpy); - (cpy.PyGILState_Release)(state); + let msg = pytypes::take_py_error_string(cpy); if method_id == PY_METHOD_EVAL_R { return NYB_E_PLUGIN_ERROR; } @@ -479,11 +210,13 @@ fn handle_py_runtime( } if (method_id == PY_METHOD_EVAL || method_id == PY_METHOD_EVAL_R) && should_autodecode() - && try_write_autodecode(cpy, obj, result, result_len) { - (cpy.Py_DecRef)(obj); - (cpy.PyGILState_Release)(state); - return NYB_SUCCESS; + if let Some(decoded) = pytypes::autodecode(cpy, obj) { + if write_autodecode_result(&decoded, result, result_len) { + (cpy.Py_DecRef)(obj); + return NYB_SUCCESS; + } + } } // Store as PyObjectBox handle let id = OBJ_COUNTER.fetch_add(1, Ordering::Relaxed); @@ -491,13 +224,12 @@ fn handle_py_runtime( map.insert(id, PyObjectInstance::default()); } else { (cpy.Py_DecRef)(obj); - (cpy.PyGILState_Release)(state); return NYB_E_PLUGIN_ERROR; } // Keep reference (obj is new ref). We model store via separate map and hold pointer via raw address table. // To actually manage pointer per id, we extend PyObjectInstance in 10.5b with a field. For now, attach through side-table. - OBJ_PTRS.lock().unwrap().insert(id, PyPtr(obj)); - (cpy.PyGILState_Release)(state); + let owned = PyOwned::from_new(obj).expect("non-null PyObject"); + PY_HANDLES.lock().unwrap().insert(id, owned); return write_tlv_handle(TYPE_ID_PY_OBJECT, id, result, result_len); } NYB_E_PLUGIN_ERROR @@ -509,16 +241,15 @@ fn handle_py_runtime( let Some(name) = read_arg_string(_args, _args_len, 0) else { return NYB_E_INVALID_ARGS; }; - let c_name = match CString::new(name) { + let c_name = match pytypes::cstring_from_str(&name) { Ok(s) => s, Err(_) => return NYB_E_INVALID_ARGS, }; - if let Some(cpy) = &*CPY.lock().unwrap() { - let state = (cpy.PyGILState_Ensure)(); + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let _gil = gil::GILGuard::acquire(cpy); let obj = (cpy.PyImport_ImportModule)(c_name.as_ptr()); if obj.is_null() { - let msg = take_py_error_string(cpy); - (cpy.PyGILState_Release)(state); + let msg = pytypes::take_py_error_string(cpy); if method_id == PY_METHOD_IMPORT_R { return NYB_E_PLUGIN_ERROR; } @@ -540,11 +271,10 @@ fn handle_py_runtime( map.insert(id, PyObjectInstance::default()); } else { (cpy.Py_DecRef)(obj); - (cpy.PyGILState_Release)(state); return NYB_E_PLUGIN_ERROR; } - OBJ_PTRS.lock().unwrap().insert(id, PyPtr(obj)); - (cpy.PyGILState_Release)(state); + let owned = PyOwned::from_new(obj).expect("non-null PyObject"); + PY_HANDLES.lock().unwrap().insert(id, owned); return write_tlv_handle(TYPE_ID_PY_OBJECT, id, result, result_len); } NYB_E_PLUGIN_ERROR @@ -565,13 +295,7 @@ fn handle_py_object( match method_id { PYO_METHOD_BIRTH => NYB_E_INVALID_METHOD, // should be created via runtime PYO_METHOD_FINI => { - if let Some(cpy) = &*CPY.lock().unwrap() { - if let Some(ptr) = OBJ_PTRS.lock().unwrap().remove(&instance_id) { - unsafe { - (cpy.Py_DecRef)(ptr.0); - } - } - } + PY_HANDLES.lock().unwrap().remove(&instance_id); if let Ok(mut map) = PYOBJS.lock() { map.remove(&instance_id); NYB_SUCCESS @@ -586,21 +310,22 @@ fn handle_py_object( let Some(name) = read_arg_string(_args, _args_len, 0) else { return NYB_E_INVALID_ARGS; }; - if let Some(cpy) = &*CPY.lock().unwrap() { - let Some(obj) = OBJ_PTRS.lock().unwrap().get(&instance_id).copied() else { - return NYB_E_INVALID_HANDLE; + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let obj_ptr = { + let guard = PY_HANDLES.lock().unwrap(); + let Some(handle) = guard.get(&instance_id) else { + return NYB_E_INVALID_HANDLE; + }; + handle.as_ptr() }; - let c_name = match CString::new(name) { + let c_name = match pytypes::cstring_from_str(&name) { Ok(s) => s, Err(_) => return NYB_E_INVALID_ARGS, }; - let state = unsafe { (cpy.PyGILState_Ensure)() }; - let attr = unsafe { (cpy.PyObject_GetAttrString)(obj.0, c_name.as_ptr()) }; + let _gil = gil::GILGuard::acquire(cpy); + let attr = unsafe { (cpy.PyObject_GetAttrString)(obj_ptr, c_name.as_ptr()) }; if attr.is_null() { - let msg = take_py_error_string(cpy); - unsafe { - (cpy.PyGILState_Release)(state); - } + let msg = pytypes::take_py_error_string(cpy); if method_id == PYO_METHOD_GETATTR_R { return NYB_E_PLUGIN_ERROR; } @@ -609,18 +334,19 @@ fn handle_py_object( } return NYB_E_PLUGIN_ERROR; } - if should_autodecode() && try_write_autodecode(cpy, attr, result, result_len) { - unsafe { - (cpy.Py_DecRef)(attr); - (cpy.PyGILState_Release)(state); + if should_autodecode() { + if let Some(decoded) = pytypes::autodecode(cpy, attr) { + if write_autodecode_result(&decoded, result, result_len) { + unsafe { + (cpy.Py_DecRef)(attr); + } + return NYB_SUCCESS; + } } - return NYB_SUCCESS; } let id = OBJ_COUNTER.fetch_add(1, Ordering::Relaxed); - OBJ_PTRS.lock().unwrap().insert(id, PyPtr(attr)); - unsafe { - (cpy.PyGILState_Release)(state); - } + let owned = unsafe { PyOwned::from_new(attr).expect("non-null PyObject") }; + PY_HANDLES.lock().unwrap().insert(id, owned); return write_tlv_handle(TYPE_ID_PY_OBJECT, id, result, result_len); } NYB_E_PLUGIN_ERROR @@ -629,36 +355,26 @@ fn handle_py_object( if ensure_cpython().is_err() { return NYB_E_PLUGIN_ERROR; } - if let Some(cpy) = &*CPY.lock().unwrap() { - let Some(func) = OBJ_PTRS.lock().unwrap().get(&instance_id).copied() else { - return NYB_E_INVALID_HANDLE; + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let func_ptr = { + let guard = PY_HANDLES.lock().unwrap(); + let Some(handle) = guard.get(&instance_id) else { + return NYB_E_INVALID_HANDLE; + }; + handle.as_ptr() }; - let state = unsafe { (cpy.PyGILState_Ensure)() }; - // Build tuple from TLV args - let argc = tlv_count_args(_args, _args_len); - let tuple = unsafe { (cpy.PyTuple_New)(argc as isize) }; - if tuple.is_null() { - unsafe { - (cpy.PyGILState_Release)(state); - } - return NYB_E_PLUGIN_ERROR; - } - if !fill_tuple_from_tlv(cpy, tuple, _args, _args_len) { - unsafe { - (cpy.Py_DecRef)(tuple); - (cpy.PyGILState_Release)(state); - } - return NYB_E_INVALID_ARGS; - } - let ret = unsafe { (cpy.PyObject_CallObject)(func.0, tuple) }; + let _gil = gil::GILGuard::acquire(cpy); + // Build tuple from TLV args via pytypes + let tuple = match pytypes::tuple_from_tlv(cpy, _args, _args_len) { + Ok(t) => t, + Err(_) => return NYB_E_INVALID_ARGS, + }; + let ret = unsafe { (cpy.PyObject_CallObject)(func_ptr, tuple) }; unsafe { (cpy.Py_DecRef)(tuple); } if ret.is_null() { - let msg = take_py_error_string(cpy); - unsafe { - (cpy.PyGILState_Release)(state); - } + let msg = pytypes::take_py_error_string(cpy); if method_id == PYO_METHOD_CALL_R { return NYB_E_PLUGIN_ERROR; } @@ -667,18 +383,19 @@ fn handle_py_object( } return NYB_E_PLUGIN_ERROR; } - if should_autodecode() && try_write_autodecode(cpy, ret, result, result_len) { - unsafe { - (cpy.Py_DecRef)(ret); - (cpy.PyGILState_Release)(state); + if should_autodecode() { + if let Some(decoded) = pytypes::autodecode(cpy, ret) { + if write_autodecode_result(&decoded, result, result_len) { + unsafe { + (cpy.Py_DecRef)(ret); + } + return NYB_SUCCESS; + } } - return NYB_SUCCESS; } let id = OBJ_COUNTER.fetch_add(1, Ordering::Relaxed); - OBJ_PTRS.lock().unwrap().insert(id, PyPtr(ret)); - unsafe { - (cpy.PyGILState_Release)(state); - } + let owned = unsafe { PyOwned::from_new(ret).expect("non-null PyObject") }; + PY_HANDLES.lock().unwrap().insert(id, owned); return write_tlv_handle(TYPE_ID_PY_OBJECT, id, result, result_len); } NYB_E_PLUGIN_ERROR @@ -687,46 +404,37 @@ fn handle_py_object( if ensure_cpython().is_err() { return NYB_E_PLUGIN_ERROR; } - if let Some(cpy) = &*CPY.lock().unwrap() { - let Some(func) = OBJ_PTRS.lock().unwrap().get(&instance_id).copied() else { - return NYB_E_INVALID_HANDLE; + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let func_ptr = { + let guard = PY_HANDLES.lock().unwrap(); + let Some(handle) = guard.get(&instance_id) else { + return NYB_E_INVALID_HANDLE; + }; + handle.as_ptr() }; - let state = unsafe { (cpy.PyGILState_Ensure)() }; + let _gil = gil::GILGuard::acquire(cpy); // Empty args tuple for kwargs-only call let args_tup = unsafe { (cpy.PyTuple_New)(0) }; if args_tup.is_null() { - unsafe { - (cpy.PyGILState_Release)(state); - } return NYB_E_PLUGIN_ERROR; } - // Build kwargs dict from TLV pairs - let kwargs = unsafe { (cpy.PyDict_New)() }; - if kwargs.is_null() { - unsafe { - (cpy.Py_DecRef)(args_tup); - (cpy.PyGILState_Release)(state); + // Build kwargs dict from TLV pairs via pytypes + let kwargs = match pytypes::kwargs_from_tlv(cpy, _args, _args_len) { + Ok(d) => d, + Err(_) => { + unsafe { + (cpy.Py_DecRef)(args_tup); + } + return NYB_E_INVALID_ARGS; } - return NYB_E_PLUGIN_ERROR; - } - if !fill_kwargs_from_tlv(cpy, kwargs, _args, _args_len) { - unsafe { - (cpy.Py_DecRef)(kwargs); - (cpy.Py_DecRef)(args_tup); - (cpy.PyGILState_Release)(state); - } - return NYB_E_INVALID_ARGS; - } - let ret = unsafe { (cpy.PyObject_Call)(func.0, args_tup, kwargs) }; + }; + let ret = unsafe { (cpy.PyObject_Call)(func_ptr, args_tup, kwargs) }; unsafe { (cpy.Py_DecRef)(kwargs); (cpy.Py_DecRef)(args_tup); } if ret.is_null() { - let msg = take_py_error_string(cpy); - unsafe { - (cpy.PyGILState_Release)(state); - } + let msg = pytypes::take_py_error_string(cpy); if method_id == PYO_METHOD_CALL_KW_R { return NYB_E_PLUGIN_ERROR; } @@ -737,19 +445,19 @@ fn handle_py_object( } if (method_id == PYO_METHOD_CALL_KW || method_id == PYO_METHOD_CALL_KW_R) && should_autodecode() - && try_write_autodecode(cpy, ret, result, result_len) { - unsafe { - (cpy.Py_DecRef)(ret); - (cpy.PyGILState_Release)(state); + if let Some(decoded) = pytypes::autodecode(cpy, ret) { + if write_autodecode_result(&decoded, result, result_len) { + unsafe { + (cpy.Py_DecRef)(ret); + } + return NYB_SUCCESS; + } } - return NYB_SUCCESS; } let id = OBJ_COUNTER.fetch_add(1, Ordering::Relaxed); - OBJ_PTRS.lock().unwrap().insert(id, PyPtr(ret)); - unsafe { - (cpy.PyGILState_Release)(state); - } + let owned = unsafe { PyOwned::from_new(ret).expect("non-null PyObject") }; + PY_HANDLES.lock().unwrap().insert(id, owned); return write_tlv_handle(TYPE_ID_PY_OBJECT, id, result, result_len); } NYB_E_PLUGIN_ERROR @@ -758,32 +466,31 @@ fn handle_py_object( if ensure_cpython().is_err() { return NYB_E_PLUGIN_ERROR; } - if let Some(cpy) = &*CPY.lock().unwrap() { - let Some(obj) = OBJ_PTRS.lock().unwrap().get(&instance_id).copied() else { - return NYB_E_INVALID_HANDLE; + if let Some(cpy) = &*ffi::CPY.lock().unwrap() { + let obj_ptr = { + let guard = PY_HANDLES.lock().unwrap(); + let Some(handle) = guard.get(&instance_id) else { + return NYB_E_INVALID_HANDLE; + }; + handle.as_ptr() }; - let state = unsafe { (cpy.PyGILState_Ensure)() }; - let s_obj = unsafe { (cpy.PyObject_Str)(obj.0) }; + let _gil = gil::GILGuard::acquire(cpy); + let s_obj = unsafe { (cpy.PyObject_Str)(obj_ptr) }; if s_obj.is_null() { - unsafe { - (cpy.PyGILState_Release)(state); - } return NYB_E_PLUGIN_ERROR; } - let cstr = unsafe { (cpy.PyUnicode_AsUTF8)(s_obj) }; - if cstr.is_null() { - unsafe { - (cpy.Py_DecRef)(s_obj); - (cpy.PyGILState_Release)(state); + let rust_str = unsafe { + let cstr = (cpy.PyUnicode_AsUTF8)(s_obj); + match pytypes::cstr_to_string(cstr) { + Some(s) => s, + None => { + (cpy.Py_DecRef)(s_obj); + return NYB_E_PLUGIN_ERROR; + } } - return NYB_E_PLUGIN_ERROR; - } - let rust_str = unsafe { CStr::from_ptr(cstr) } - .to_string_lossy() - .to_string(); + }; unsafe { (cpy.Py_DecRef)(s_obj); - (cpy.PyGILState_Release)(state); } return write_tlv_string(&rust_str, result, result_len); } @@ -798,9 +505,9 @@ fn handle_py_object( // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -809,12 +516,28 @@ pub struct NyashTypeBoxFfi { unsafe impl Sync for NyashTypeBoxFfi {} extern "C" fn pyruntime_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); - match s.as_ref() { + if name.is_null() { + return 0; + } + let Some(s) = (unsafe { pytypes::cstr_to_string(name) }) else { + return 0; + }; + match s.as_str() { "birth" => PY_METHOD_BIRTH, - "eval" | "evalR" => { if s.as_ref() == "evalR" { PY_METHOD_EVAL_R } else { PY_METHOD_EVAL } } - "import" | "importR" => { if s.as_ref() == "importR" { PY_METHOD_IMPORT_R } else { PY_METHOD_IMPORT } } + "eval" | "evalR" => { + if s == "evalR" { + PY_METHOD_EVAL_R + } else { + PY_METHOD_EVAL + } + } + "import" | "importR" => { + if s == "importR" { + PY_METHOD_IMPORT_R + } else { + PY_METHOD_IMPORT + } + } "fini" => PY_METHOD_FINI, _ => 0, } @@ -832,15 +555,33 @@ extern "C" fn pyruntime_invoke_id( } extern "C" fn pyobject_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } - let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); - match s.as_ref() { + if name.is_null() { + return 0; + } + let Some(s) = (unsafe { pytypes::cstr_to_string(name) }) else { + return 0; + }; + match s.as_str() { "getattr" | "getAttr" | "getattrR" | "getAttrR" => { - if s.ends_with('R') { PYO_METHOD_GETATTR_R } else { PYO_METHOD_GETATTR } + if s.ends_with('R') { + PYO_METHOD_GETATTR_R + } else { + PYO_METHOD_GETATTR + } + } + "call" | "callR" => { + if s.ends_with('R') { + PYO_METHOD_CALL_R + } else { + PYO_METHOD_CALL + } } - "call" | "callR" => { if s.ends_with('R') { PYO_METHOD_CALL_R } else { PYO_METHOD_CALL } } "callKw" | "callKW" | "call_kw" | "callKwR" | "callKWR" => { - if s.to_lowercase().ends_with('r') { PYO_METHOD_CALL_KW_R } else { PYO_METHOD_CALL_KW } + if s.to_lowercase().ends_with('r') { + PYO_METHOD_CALL_KW_R + } else { + PYO_METHOD_CALL_KW + } } "str" | "toString" => PYO_METHOD_STR, "birth" => PYO_METHOD_BIRTH, @@ -941,11 +682,7 @@ fn read_arg_string(args: *const u8, args_len: usize, n: usize) -> Option } // Side-table for PyObject* storage (instance_id -> pointer) -#[derive(Copy, Clone)] -struct PyPtr(*mut PyObject); -unsafe impl Send for PyPtr {} -unsafe impl Sync for PyPtr {} -static OBJ_PTRS: Lazy>> = Lazy::new(|| Mutex::new(HashMap::new())); +static PY_HANDLES: Lazy>> = Lazy::new(|| Mutex::new(HashMap::new())); // Base TLV writer used by helpers fn write_tlv_result(payloads: &[(u8, &[u8])], result: *mut u8, result_len: *mut usize) -> i32 { @@ -974,336 +711,54 @@ fn write_tlv_result(payloads: &[(u8, &[u8])], result: *mut u8, result_len: *mut NYB_SUCCESS } -fn tlv_count_args(args: *const u8, args_len: usize) -> usize { - if args.is_null() || args_len < 4 { - return 0; - } - let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; - if buf.len() < 4 { - return 0; - } - let argc = u16::from_le_bytes([buf[2], buf[3]]) as usize; - argc -} - -fn fill_tuple_from_tlv( - cpy: &CPython, - tuple: *mut PyObject, - args: *const u8, - args_len: usize, -) -> bool { - if args.is_null() || args_len < 4 { - return true; - } - let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; - let mut off = 4usize; - let mut idx: isize = 0; - while off + 4 <= buf.len() { - let tag = buf[off]; - let _rsv = buf[off + 1]; - let size = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; - if off + 4 + size > buf.len() { - return false; - } - let payload = &buf[off + 4..off + 4 + size]; - let mut obj: *mut PyObject = std::ptr::null_mut(); - unsafe { - match tag { - 1 => { - // Bool (1 byte) - let v = if size >= 1 && payload[0] != 0 { 1 } else { 0 }; - obj = (cpy.PyBool_FromLong)(v); - } - 2 => { - // I32 - if size != 4 { - return false; - } - let mut b = [0u8; 4]; - b.copy_from_slice(payload); - let v = i32::from_le_bytes(b) as i64; - obj = (cpy.PyLong_FromLongLong)(v); - } - 3 => { - // I64 - if size != 8 { - return false; - } - let mut b = [0u8; 8]; - b.copy_from_slice(payload); - let v = i64::from_le_bytes(b); - obj = (cpy.PyLong_FromLongLong)(v); - } - 5 => { - // F64 - if size != 8 { - return false; - } - let mut b = [0u8; 8]; - b.copy_from_slice(payload); - let v = f64::from_le_bytes(b); - obj = (cpy.PyFloat_FromDouble)(v); - } - 6 => { - // String - let c = match CString::new(payload) { - Ok(c) => c, - Err(_) => return false, - }; - obj = (cpy.PyUnicode_FromString)(c.as_ptr()); - } - 7 => { - // Bytes - if size > 0 { - let ptr = payload.as_ptr() as *const c_char; - obj = (cpy.PyBytes_FromStringAndSize)(ptr, size as isize); - } else { - obj = (cpy.PyBytes_FromStringAndSize)(std::ptr::null(), 0); - } - } - 8 => { - // Handle - if size != 8 { - return false; - } - let mut t = [0u8; 4]; - t.copy_from_slice(&payload[0..4]); - let mut i = [0u8; 4]; - i.copy_from_slice(&payload[4..8]); - let _type_id = u32::from_le_bytes(t); - let inst_id = u32::from_le_bytes(i); - if let Some(ptr) = OBJ_PTRS.lock().unwrap().get(&inst_id).copied() { - // PyTuple_SetItem steals a reference; INCREF to keep original alive - (cpy.Py_IncRef)(ptr.0); - obj = ptr.0; - } else { - return false; - } - } - _ => { - return false; - } - } - if obj.is_null() { - return false; - } - let rc = (cpy.PyTuple_SetItem)(tuple, idx, obj); - if rc != 0 { - return false; - } - idx += 1; - } - off += 4 + size; - } - true -} - -fn take_py_error_string(cpy: &CPython) -> Option { - unsafe { - if (cpy.PyErr_Occurred)().is_null() { - return None; - } - let mut ptype: *mut PyObject = std::ptr::null_mut(); - let mut pvalue: *mut PyObject = std::ptr::null_mut(); - let mut ptrace: *mut PyObject = std::ptr::null_mut(); - (cpy.PyErr_Fetch)(&mut ptype, &mut pvalue, &mut ptrace); - let s = if !pvalue.is_null() { - let sobj = (cpy.PyObject_Str)(pvalue); - if sobj.is_null() { - (cpy.PyErr_Clear)(); - return Some("Python error".to_string()); - } - let cstr = (cpy.PyUnicode_AsUTF8)(sobj); - let msg = if cstr.is_null() { - "Python error".to_string() - } else { - CStr::from_ptr(cstr).to_string_lossy().to_string() - }; - (cpy.Py_DecRef)(sobj); - msg - } else { - "Python error".to_string() - }; - (cpy.PyErr_Clear)(); - Some(s) - } -} - -fn fill_kwargs_from_tlv( - cpy: &CPython, - dict: *mut PyObject, - args: *const u8, - args_len: usize, -) -> bool { - if args.is_null() || args_len < 4 { - return true; - } - let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; - let mut off = 4usize; - loop { - if off + 4 > buf.len() { - break; - } - let tag_k = buf[off]; - let _rsv = buf[off + 1]; - let size_k = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; - if tag_k != 6 || off + 4 + size_k > buf.len() { - return false; - } - let key_bytes = &buf[off + 4..off + 4 + size_k]; - off += 4 + size_k; - if off + 4 > buf.len() { - return false; - } - let tag_v = buf[off]; - let _rsv2 = buf[off + 1]; - let size_v = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; - if off + 4 + size_v > buf.len() { - return false; - } - let val_payload = &buf[off + 4..off + 4 + size_v]; - off += 4 + size_v; - unsafe { - let key_c = match CString::new(key_bytes) { - Ok(c) => c, - Err(_) => return false, - }; - let mut obj: *mut PyObject = std::ptr::null_mut(); - match tag_v { - 1 => { - let v = if size_v >= 1 && val_payload[0] != 0 { - 1 - } else { - 0 - }; - obj = (cpy.PyBool_FromLong)(v); - } - 2 => { - if size_v != 4 { - return false; - } - let mut b = [0u8; 4]; - b.copy_from_slice(val_payload); - obj = (cpy.PyLong_FromLongLong)(i32::from_le_bytes(b) as i64); - } - 3 => { - if size_v != 8 { - return false; - } - let mut b = [0u8; 8]; - b.copy_from_slice(val_payload); - obj = (cpy.PyLong_FromLongLong)(i64::from_le_bytes(b)); - } - 5 => { - if size_v != 8 { - return false; - } - let mut b = [0u8; 8]; - b.copy_from_slice(val_payload); - obj = (cpy.PyFloat_FromDouble)(f64::from_le_bytes(b)); - } - 6 => { - let c = match CString::new(val_payload) { - Ok(c) => c, - Err(_) => return false, - }; - obj = (cpy.PyUnicode_FromString)(c.as_ptr()); - } - 7 => { - let ptr = if size_v > 0 { - val_payload.as_ptr() as *const c_char - } else { - std::ptr::null() - }; - obj = (cpy.PyBytes_FromStringAndSize)(ptr, size_v as isize); - } - 8 => { - if size_v != 8 { - return false; - } - let mut i = [0u8; 4]; - i.copy_from_slice(&val_payload[4..8]); - let inst_id = u32::from_le_bytes(i); - if let Some(p) = OBJ_PTRS.lock().unwrap().get(&inst_id).copied() { - (cpy.Py_IncRef)(p.0); - obj = p.0; - } else { - return false; - } - } - _ => return false, - } - if obj.is_null() { - return false; - } - let rc = (cpy.PyDict_SetItemString)(dict, key_c.as_ptr(), obj); - (cpy.Py_DecRef)(obj); - if rc != 0 { - return false; - } - } - } - true -} fn should_autodecode() -> bool { std::env::var("NYASH_PY_AUTODECODE") .map(|v| v != "0") .unwrap_or(false) } -fn try_write_autodecode( - cpy: &CPython, - obj: *mut PyObject, +fn autodecode_logging_enabled() -> bool { + std::env::var("NYASH_PY_LOG") + .map(|v| v != "0") + .unwrap_or(false) +} + +fn write_autodecode_result( + decoded: &DecodedValue, result: *mut u8, result_len: *mut usize, ) -> bool { - unsafe { - // Float -> tag=5 - let mut had_err = false; - let f = (cpy.PyFloat_AsDouble)(obj); - if !(cpy.PyErr_Occurred)().is_null() { - had_err = true; - (cpy.PyErr_Clear)(); + let rc = match decoded { + DecodedValue::Float(value) => { + if autodecode_logging_enabled() { + eprintln!("[PyPlugin] autodecode: Float {}", value); + } + let payload = value.to_le_bytes(); + write_tlv_result(&[(5u8, payload.as_slice())], result, result_len) } - if !had_err { - eprintln!("[PyPlugin] autodecode: Float {}", f); - let mut payload = [0u8; 8]; - payload.copy_from_slice(&f.to_le_bytes()); - let rc = write_tlv_result(&[(5u8, &payload)], result, result_len); - return rc == NYB_SUCCESS || rc == NYB_E_SHORT_BUFFER; + DecodedValue::Int(value) => { + if autodecode_logging_enabled() { + eprintln!("[PyPlugin] autodecode: I64 {}", value); + } + let payload = value.to_le_bytes(); + write_tlv_result(&[(3u8, payload.as_slice())], result, result_len) } - // Integer (PyLong) -> tag=3 (i64) - let i = (cpy.PyLong_AsLongLong)(obj); - if !(cpy.PyErr_Occurred)().is_null() { - (cpy.PyErr_Clear)(); - } else { - eprintln!("[PyPlugin] autodecode: I64 {}", i); - let mut payload = [0u8; 8]; - payload.copy_from_slice(&i.to_le_bytes()); - let rc = write_tlv_result(&[(3u8, &payload)], result, result_len); - return rc == NYB_SUCCESS || rc == NYB_E_SHORT_BUFFER; + DecodedValue::Str(text) => { + if autodecode_logging_enabled() { + eprintln!( + "[PyPlugin] autodecode: String '{}', len={} ", + text, + text.len() + ); + } + write_tlv_result(&[(6u8, text.as_bytes())], result, result_len) } - // Unicode -> tag=6 - let u = (cpy.PyUnicode_AsUTF8)(obj); - if !(cpy.PyErr_Occurred)().is_null() { - (cpy.PyErr_Clear)(); - } else if !u.is_null() { - let s = CStr::from_ptr(u).to_string_lossy(); - eprintln!("[PyPlugin] autodecode: String '{}', len={} ", s, s.len()); - let rc = write_tlv_result(&[(6u8, s.as_bytes())], result, result_len); - return rc == NYB_SUCCESS || rc == NYB_E_SHORT_BUFFER; + DecodedValue::Bytes(data) => { + if autodecode_logging_enabled() { + eprintln!("[PyPlugin] autodecode: Bytes {} bytes", data.len()); + } + write_tlv_result(&[(7u8, data.as_slice())], result, result_len) } - // Bytes -> tag=7 - let mut ptr: *mut c_char = std::ptr::null_mut(); - let mut sz: isize = 0; - if (cpy.PyBytes_AsStringAndSize)(obj, &mut ptr, &mut sz) == 0 { - let slice = std::slice::from_raw_parts(ptr as *const u8, sz as usize); - eprintln!("[PyPlugin] autodecode: Bytes {} bytes", sz); - let rc = write_tlv_result(&[(7u8, slice)], result, result_len); - return rc == NYB_SUCCESS || rc == NYB_E_SHORT_BUFFER; - } else if !(cpy.PyErr_Occurred)().is_null() { - (cpy.PyErr_Clear)(); - } - false - } + }; + rc == NYB_SUCCESS || rc == NYB_E_SHORT_BUFFER } diff --git a/plugins/nyash-python-plugin/src/pytypes.rs b/plugins/nyash-python-plugin/src/pytypes.rs new file mode 100644 index 00000000..c9fb76e0 --- /dev/null +++ b/plugins/nyash-python-plugin/src/pytypes.rs @@ -0,0 +1,461 @@ +use crate::ffi::{self, CPython, PyObject}; +use std::ffi::{CStr, CString}; +use std::marker::PhantomData; +use std::os::raw::c_char; +use std::ptr::NonNull; + +pub enum DecodedValue { + Float(f64), + Int(i64), + Str(String), + Bytes(Vec), +} + +pub struct PyOwned { + ptr: NonNull, +} + +#[allow(dead_code)] +pub struct PyBorrowed<'a> { + ptr: NonNull, + _marker: PhantomData<&'a PyObject>, +} + +impl PyOwned { + pub unsafe fn from_new(ptr: *mut PyObject) -> Option { + NonNull::new(ptr).map(|ptr| PyOwned { ptr }) + } + + pub unsafe fn from_raw(ptr: *mut PyObject) -> Option { + NonNull::new(ptr).map(|ptr| PyOwned { ptr }) + } + + #[allow(dead_code)] + pub unsafe fn from_borrowed(cpy: &CPython, borrowed: PyBorrowed<'_>) -> Self { + (cpy.Py_IncRef)(borrowed.ptr.as_ptr()); + PyOwned { ptr: borrowed.ptr } + } + + pub fn as_ptr(&self) -> *mut PyObject { + self.ptr.as_ptr() + } + + #[allow(dead_code)] + pub fn borrow(&self) -> PyBorrowed<'_> { + PyBorrowed { + ptr: self.ptr, + _marker: PhantomData, + } + } + + #[allow(dead_code)] + pub fn clone_ref(&self, cpy: &CPython) -> Self { + unsafe { + (cpy.Py_IncRef)(self.ptr.as_ptr()); + } + PyOwned { ptr: self.ptr } + } + + pub fn into_raw(self) -> *mut PyObject { + let ptr = self.ptr.as_ptr(); + std::mem::forget(self); + ptr + } +} + +impl Drop for PyOwned { + fn drop(&mut self) { + if let Ok(guard) = ffi::CPY.lock() { + if let Some(cpy) = guard.as_ref() { + unsafe { + (cpy.Py_DecRef)(self.ptr.as_ptr()); + } + } + } + } +} + +impl Clone for PyOwned { + fn clone(&self) -> Self { + let guard = ffi::CPY.lock().expect("CPython state poisoned"); + let cpy = guard.as_ref().expect("CPython not initialized"); + unsafe { + (cpy.Py_IncRef)(self.ptr.as_ptr()); + } + PyOwned { ptr: self.ptr } + } +} + +unsafe impl Send for PyOwned {} +unsafe impl Sync for PyOwned {} + +#[allow(dead_code)] +impl<'a> PyBorrowed<'a> { + pub unsafe fn new(ptr: *mut PyObject) -> Option { + NonNull::new(ptr).map(|ptr| PyBorrowed { + ptr, + _marker: PhantomData, + }) + } + + pub fn as_ptr(&self) -> *mut PyObject { + self.ptr.as_ptr() + } +} + +impl<'a> Copy for PyBorrowed<'a> {} + +impl<'a> Clone for PyBorrowed<'a> { + fn clone(&self) -> Self { + *self + } +} + +pub fn autodecode(cpy: &CPython, obj: *mut PyObject) -> Option { + unsafe { + let f = (cpy.PyFloat_AsDouble)(obj); + if (cpy.PyErr_Occurred)().is_null() { + return Some(DecodedValue::Float(f)); + } + (cpy.PyErr_Clear)(); + + let i = (cpy.PyLong_AsLongLong)(obj); + if (cpy.PyErr_Occurred)().is_null() { + return Some(DecodedValue::Int(i)); + } + (cpy.PyErr_Clear)(); + + let u = (cpy.PyUnicode_AsUTF8)(obj); + if (cpy.PyErr_Occurred)().is_null() && !u.is_null() { + let s = CStr::from_ptr(u).to_string_lossy().to_string(); + return Some(DecodedValue::Str(s)); + } + (cpy.PyErr_Clear)(); + + let mut ptr: *mut c_char = std::ptr::null_mut(); + let mut sz: isize = 0; + if (cpy.PyBytes_AsStringAndSize)(obj, &mut ptr, &mut sz) == 0 { + let slice = std::slice::from_raw_parts(ptr as *const u8, sz as usize); + return Some(DecodedValue::Bytes(slice.to_vec())); + } + if !(cpy.PyErr_Occurred)().is_null() { + (cpy.PyErr_Clear)(); + } + } + None +} + +pub fn cstring_from_str(s: &str) -> Result { + CString::new(s).map_err(|_| ()) +} + +pub unsafe fn cstr_to_string(ptr: *const c_char) -> Option { + if ptr.is_null() { + return None; + } + Some(CStr::from_ptr(ptr).to_string_lossy().to_string()) +} + +#[allow(dead_code)] +pub unsafe fn incref(cpy: &CPython, obj: *mut PyObject) { + (cpy.Py_IncRef)(obj); +} + +#[allow(dead_code)] +pub unsafe fn decref(cpy: &CPython, obj: *mut PyObject) { + (cpy.Py_DecRef)(obj); +} + +pub fn take_py_error_string(cpy: &CPython) -> Option { + unsafe { + if (cpy.PyErr_Occurred)().is_null() { + return None; + } + let mut ptype: *mut PyObject = std::ptr::null_mut(); + let mut pvalue: *mut PyObject = std::ptr::null_mut(); + let mut ptrace: *mut PyObject = std::ptr::null_mut(); + (cpy.PyErr_Fetch)(&mut ptype, &mut pvalue, &mut ptrace); + let s = if !pvalue.is_null() { + let sobj = (cpy.PyObject_Str)(pvalue); + if sobj.is_null() { + (cpy.PyErr_Clear)(); + return Some("Python error".to_string()); + } + let cstr = (cpy.PyUnicode_AsUTF8)(sobj); + let msg = if cstr.is_null() { + "Python error".to_string() + } else { + CStr::from_ptr(cstr).to_string_lossy().to_string() + }; + (cpy.Py_DecRef)(sobj); + msg + } else { + "Python error".to_string() + }; + (cpy.PyErr_Clear)(); + Some(s) + } +} + +pub fn count_tlv_args(args: *const u8, args_len: usize) -> usize { + if args.is_null() || args_len < 4 { + return 0; + } + let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; + if buf.len() < 4 { + return 0; + } + u16::from_le_bytes([buf[2], buf[3]]) as usize +} + +pub fn tuple_from_tlv( + cpy: &CPython, + args: *const u8, + args_len: usize, +) -> Result<*mut PyObject, ()> { + let argc = count_tlv_args(args, args_len) as isize; + let tuple = unsafe { (cpy.PyTuple_New)(argc) }; + if tuple.is_null() { + return Err(()); + } + if argc == 0 { + return Ok(tuple); + } + if !fill_tuple_from_tlv(cpy, tuple, args, args_len) { + unsafe { + (cpy.Py_DecRef)(tuple); + } + return Err(()); + } + Ok(tuple) +} + +pub fn kwargs_from_tlv( + cpy: &CPython, + args: *const u8, + args_len: usize, +) -> Result<*mut PyObject, ()> { + let dict = unsafe { (cpy.PyDict_New)() }; + if dict.is_null() { + return Err(()); + } + if !fill_kwargs_from_tlv(cpy, dict, args, args_len) { + unsafe { + (cpy.Py_DecRef)(dict); + } + return Err(()); + } + Ok(dict) +} + +pub fn fill_tuple_from_tlv( + cpy: &CPython, + tuple: *mut PyObject, + args: *const u8, + args_len: usize, +) -> bool { + if args.is_null() || args_len < 4 { + return true; + } + let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; + let mut off = 4usize; + let mut idx: isize = 0; + while off + 4 <= buf.len() { + let tag = buf[off]; + let _rsv = buf[off + 1]; + let size = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; + if off + 4 + size > buf.len() { + return false; + } + let payload = &buf[off + 4..off + 4 + size]; + let mut obj: *mut PyObject = std::ptr::null_mut(); + unsafe { + let mut owned_transfer: Option = None; + match tag { + 1 => { + let v = if size >= 1 && payload[0] != 0 { 1 } else { 0 }; + obj = (cpy.PyBool_FromLong)(v); + } + 2 => { + if size != 4 { + return false; + } + let mut b = [0u8; 4]; + b.copy_from_slice(payload); + obj = (cpy.PyLong_FromLongLong)(i32::from_le_bytes(b) as i64); + } + 3 => { + if size != 8 { + return false; + } + let mut b = [0u8; 8]; + b.copy_from_slice(payload); + obj = (cpy.PyLong_FromLongLong)(i64::from_le_bytes(b)); + } + 5 => { + if size != 8 { + return false; + } + let mut b = [0u8; 8]; + b.copy_from_slice(payload); + obj = (cpy.PyFloat_FromDouble)(f64::from_le_bytes(b)); + } + 6 => { + let c = match CString::new(payload) { + Ok(c) => c, + Err(_) => return false, + }; + obj = (cpy.PyUnicode_FromString)(c.as_ptr()); + } + 7 => { + let ptr = if size > 0 { + payload.as_ptr() as *const c_char + } else { + std::ptr::null() + }; + obj = (cpy.PyBytes_FromStringAndSize)(ptr, size as isize); + } + 8 => { + if size != 8 { + return false; + } + let mut i = [0u8; 4]; + i.copy_from_slice(&payload[4..8]); + let inst_id = u32::from_le_bytes(i); + let Some(handle) = super::PY_HANDLES.lock().unwrap().get(&inst_id).cloned() + else { + return false; + }; + owned_transfer = Some(handle); + } + _ => return false, + } + let raw = if let Some(handle) = owned_transfer.take() { + handle.into_raw() + } else { + obj + }; + if (cpy.PyTuple_SetItem)(tuple, idx, raw) != 0 { + if let Some(rewrap) = PyOwned::from_raw(raw) { + drop(rewrap); + } + return false; + } + idx += 1; + } + off += 4 + size; + } + true +} + +pub fn fill_kwargs_from_tlv( + cpy: &CPython, + dict: *mut PyObject, + args: *const u8, + args_len: usize, +) -> bool { + if args.is_null() || args_len < 4 { + return true; + } + let buf = unsafe { std::slice::from_raw_parts(args, args_len) }; + let mut off = 4usize; + while off + 4 <= buf.len() { + // key (string) + if buf[off] != 6 { + return false; + } + let key_size = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; + if off + 4 + key_size > buf.len() { + return false; + } + let key_slice = &buf[off + 4..off + 4 + key_size]; + let key_c = match CString::new(key_slice) { + Ok(c) => c, + Err(_) => return false, + }; + off += 4 + key_size; + if off + 4 > buf.len() { + return false; + } + let tag_v = buf[off]; + let _rsv = buf[off + 1]; + let size_v = u16::from_le_bytes([buf[off + 2], buf[off + 3]]) as usize; + if off + 4 + size_v > buf.len() { + return false; + } + let val_payload = &buf[off + 4..off + 4 + size_v]; + let obj: *mut PyObject; + unsafe { + match tag_v { + 1 => { + let v = if size_v >= 1 && val_payload[0] != 0 { + 1 + } else { + 0 + }; + obj = (cpy.PyBool_FromLong)(v); + } + 2 => { + if size_v != 4 { + return false; + } + let mut b = [0u8; 4]; + b.copy_from_slice(val_payload); + obj = (cpy.PyLong_FromLongLong)(i32::from_le_bytes(b) as i64); + } + 3 => { + if size_v != 8 { + return false; + } + let mut b = [0u8; 8]; + b.copy_from_slice(val_payload); + obj = (cpy.PyLong_FromLongLong)(i64::from_le_bytes(b)); + } + 5 => { + if size_v != 8 { + return false; + } + let mut b = [0u8; 8]; + b.copy_from_slice(val_payload); + obj = (cpy.PyFloat_FromDouble)(f64::from_le_bytes(b)); + } + 6 => { + let c = match CString::new(val_payload) { + Ok(c) => c, + Err(_) => return false, + }; + obj = (cpy.PyUnicode_FromString)(c.as_ptr()); + } + 7 => { + let ptr = if size_v > 0 { + val_payload.as_ptr() as *const c_char + } else { + std::ptr::null() + }; + obj = (cpy.PyBytes_FromStringAndSize)(ptr, size_v as isize); + } + 8 => { + if size_v != 8 { + return false; + } + let mut i = [0u8; 4]; + i.copy_from_slice(&val_payload[4..8]); + let inst_id = u32::from_le_bytes(i); + let Some(handle) = super::PY_HANDLES.lock().unwrap().get(&inst_id).cloned() + else { + return false; + }; + obj = handle.into_raw(); + } + _ => return false, + } + let rc = (cpy.PyDict_SetItemString)(dict, key_c.as_ptr(), obj); + (cpy.Py_DecRef)(obj); + if rc != 0 { + return false; + } + } + off += 4 + size_v; + } + true +} diff --git a/plugins/nyash-regex-plugin/src/lib.rs b/plugins/nyash-regex-plugin/src/lib.rs index 62e7c719..de77a3f9 100644 --- a/plugins/nyash-regex-plugin/src/lib.rs +++ b/plugins/nyash-regex-plugin/src/lib.rs @@ -208,9 +208,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -220,7 +220,9 @@ unsafe impl Sync for NyashTypeBoxFfi {} use std::ffi::CStr; extern "C" fn regex_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "compile" => M_COMPILE, @@ -246,66 +248,148 @@ extern "C" fn regex_invoke_id( match method_id { M_BIRTH => { // mirror v1: birth may take optional pattern - if result_len.is_null() { return E_ARGS; } - if preflight(result, result_len, 4) { return E_SHORT; } + if result_len.is_null() { + return E_ARGS; + } + if preflight(result, result_len, 4) { + return E_SHORT; + } let id = NEXT_ID.fetch_add(1, Ordering::Relaxed); let inst = if let Some(pat) = read_arg_string(args, args_len, 0) { - match Regex::new(&pat) { Ok(re) => RegexInstance { re: Some(re) }, Err(_) => RegexInstance { re: None } } - } else { RegexInstance { re: None } }; - if let Ok(mut m) = INST.lock() { m.insert(id, inst); } else { return E_PLUGIN; } + match Regex::new(&pat) { + Ok(re) => RegexInstance { re: Some(re) }, + Err(_) => RegexInstance { re: None }, + } + } else { + RegexInstance { re: None } + }; + if let Ok(mut m) = INST.lock() { + m.insert(id, inst); + } else { + return E_PLUGIN; + } let b = id.to_le_bytes(); std::ptr::copy_nonoverlapping(b.as_ptr(), result, 4); - *result_len = 4; OK + *result_len = 4; + OK } M_FINI => { - if let Ok(mut m) = INST.lock() { m.remove(&instance_id); OK } else { E_PLUGIN } + if let Ok(mut m) = INST.lock() { + m.remove(&instance_id); + OK + } else { + E_PLUGIN + } } M_COMPILE => { - let pat = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; + let pat = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; if let Ok(mut m) = INST.lock() { - if let Some(inst) = m.get_mut(&instance_id) { inst.re = Regex::new(&pat).ok(); OK } else { E_HANDLE } - } else { E_PLUGIN } + if let Some(inst) = m.get_mut(&instance_id) { + inst.re = Regex::new(&pat).ok(); + OK + } else { + E_HANDLE + } + } else { + E_PLUGIN + } } M_IS_MATCH => { - let text = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; - if let Ok(m) = INST.lock() { - if let Some(inst) = m.get(&instance_id) { - if let Some(re) = &inst.re { return write_tlv_bool(re.is_match(&text), result, result_len); } else { return write_tlv_bool(false, result, result_len); } - } else { return E_HANDLE; } - } else { return E_PLUGIN; } - } - M_FIND => { - let text = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; + let text = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; if let Ok(m) = INST.lock() { if let Some(inst) = m.get(&instance_id) { if let Some(re) = &inst.re { - let s = re.find(&text).map(|m| m.as_str().to_string()).unwrap_or_else(|| "".to_string()); - return write_tlv_string(&s, result, result_len); - } else { return write_tlv_string("", result, result_len); } - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + return write_tlv_bool(re.is_match(&text), result, result_len); + } else { + return write_tlv_bool(false, result, result_len); + } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } - M_REPLACE_ALL => { - let text = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; - let repl = match read_arg_string(args, args_len, 1) { Some(s) => s, None => return E_ARGS }; + M_FIND => { + let text = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; if let Ok(m) = INST.lock() { if let Some(inst) = m.get(&instance_id) { - if let Some(re) = &inst.re { let out = re.replace_all(&text, repl.as_str()).to_string(); return write_tlv_string(&out, result, result_len); } else { return write_tlv_string(&text, result, result_len); } - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + if let Some(re) = &inst.re { + let s = re + .find(&text) + .map(|m| m.as_str().to_string()) + .unwrap_or_else(|| "".to_string()); + return write_tlv_string(&s, result, result_len); + } else { + return write_tlv_string("", result, result_len); + } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } + } + M_REPLACE_ALL => { + let text = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; + let repl = match read_arg_string(args, args_len, 1) { + Some(s) => s, + None => return E_ARGS, + }; + if let Ok(m) = INST.lock() { + if let Some(inst) = m.get(&instance_id) { + if let Some(re) = &inst.re { + let out = re.replace_all(&text, repl.as_str()).to_string(); + return write_tlv_string(&out, result, result_len); + } else { + return write_tlv_string(&text, result, result_len); + } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } M_SPLIT => { - let text = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; + let text = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; let limit = read_arg_i64(args, args_len, 1).unwrap_or(0); if let Ok(m) = INST.lock() { if let Some(inst) = m.get(&instance_id) { if let Some(re) = &inst.re { - let parts: Vec = if limit > 0 { re.splitn(&text, limit as usize).map(|s| s.to_string()).collect() } else { re.split(&text).map(|s| s.to_string()).collect() }; + let parts: Vec = if limit > 0 { + re.splitn(&text, limit as usize) + .map(|s| s.to_string()) + .collect() + } else { + re.split(&text).map(|s| s.to_string()).collect() + }; let out = parts.join("\n"); return write_tlv_string(&out, result, result_len); - } else { return write_tlv_string(&text, result, result_len); } - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + } else { + return write_tlv_string(&text, result, result_len); + } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } _ => E_METHOD, } diff --git a/plugins/nyash-toml-plugin/src/lib.rs b/plugins/nyash-toml-plugin/src/lib.rs index 7f588b0d..59d7b0fe 100644 --- a/plugins/nyash-toml-plugin/src/lib.rs +++ b/plugins/nyash-toml-plugin/src/lib.rs @@ -151,9 +151,9 @@ pub extern "C" fn nyash_plugin_invoke( // ===== TypeBox ABI v2 (resolve/invoke_id) ===== #[repr(C)] pub struct NyashTypeBoxFfi { - pub abi_tag: u32, // 'TYBX' - pub version: u16, // 1 - pub struct_size: u16, // sizeof(NyashTypeBoxFfi) + pub abi_tag: u32, // 'TYBX' + pub version: u16, // 1 + pub struct_size: u16, // sizeof(NyashTypeBoxFfi) pub name: *const std::os::raw::c_char, pub resolve: Option u32>, pub invoke_id: Option i32>, @@ -163,7 +163,9 @@ unsafe impl Sync for NyashTypeBoxFfi {} use std::ffi::CStr; extern "C" fn toml_resolve(name: *const std::os::raw::c_char) -> u32 { - if name.is_null() { return 0; } + if name.is_null() { + return 0; + } let s = unsafe { CStr::from_ptr(name) }.to_string_lossy(); match s.as_ref() { "parse" => M_PARSE, @@ -186,47 +188,93 @@ extern "C" fn toml_invoke_id( unsafe { match method_id { M_BIRTH => { - if result_len.is_null() { return E_ARGS; } - if preflight(result, result_len, 4) { return E_SHORT; } + if result_len.is_null() { + return E_ARGS; + } + if preflight(result, result_len, 4) { + return E_SHORT; + } let id = NEXT_ID.fetch_add(1, Ordering::Relaxed); - if let Ok(mut m) = INST.lock() { m.insert(id, TomlInstance { value: None }); } else { return E_PLUGIN; } + if let Ok(mut m) = INST.lock() { + m.insert(id, TomlInstance { value: None }); + } else { + return E_PLUGIN; + } let b = id.to_le_bytes(); std::ptr::copy_nonoverlapping(b.as_ptr(), result, 4); - *result_len = 4; OK + *result_len = 4; + OK + } + M_FINI => { + if let Ok(mut m) = INST.lock() { + m.remove(&instance_id); + OK + } else { + E_PLUGIN + } } - M_FINI => { if let Ok(mut m) = INST.lock() { m.remove(&instance_id); OK } else { E_PLUGIN } } M_PARSE => { - let text = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; + let text = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; if let Ok(mut m) = INST.lock() { if let Some(inst) = m.get_mut(&instance_id) { inst.value = toml::from_str::(&text).ok(); return write_tlv_bool(inst.value.is_some(), result, result_len); - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } M_GET => { - let path = match read_arg_string(args, args_len, 0) { Some(s) => s, None => return E_ARGS }; + let path = match read_arg_string(args, args_len, 0) { + Some(s) => s, + None => return E_ARGS, + }; if let Ok(m) = INST.lock() { if let Some(inst) = m.get(&instance_id) { - let mut cur = match &inst.value { Some(v) => v, None => { return write_tlv_string("", result, result_len); } }; + let mut cur = match &inst.value { + Some(v) => v, + None => { + return write_tlv_string("", result, result_len); + } + }; if !path.is_empty() { for seg in path.split('.') { - match cur.get(seg) { Some(v) => cur = v, None => { return write_tlv_string("", result, result_len); } } + match cur.get(seg) { + Some(v) => cur = v, + None => { + return write_tlv_string("", result, result_len); + } + } } } return write_tlv_string(&cur.to_string(), result, result_len); - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } M_TO_JSON => { if let Ok(m) = INST.lock() { if let Some(inst) = m.get(&instance_id) { if let Some(v) = &inst.value { - if let Ok(s) = serde_json::to_string(v) { return write_tlv_string(&s, result, result_len); } + if let Ok(s) = serde_json::to_string(v) { + return write_tlv_string(&s, result, result_len); + } } return write_tlv_string("{}", result, result_len); - } else { return E_HANDLE; } - } else { return E_PLUGIN; } + } else { + return E_HANDLE; + } + } else { + return E_PLUGIN; + } } _ => E_METHOD, } diff --git a/src/box_operators.rs b/src/box_operators.rs index 45f09577..bbc9403a 100644 --- a/src/box_operators.rs +++ b/src/box_operators.rs @@ -16,48 +16,58 @@ use crate::operator_traits::{ OperatorError, }; -// ===== IntegerBox Operator Implementations ===== - -/// IntegerBox + IntegerBox -> IntegerBox -impl NyashAdd for IntegerBox { - type Output = IntegerBox; - - fn add(self, rhs: IntegerBox) -> Self::Output { - IntegerBox::new(self.value + rhs.value) - } +// Small helpers to reduce duplication in dynamic operators +#[inline] +fn concat_result(left: &dyn NyashBox, right: &dyn NyashBox) -> Box { + let l = left.to_string_box(); + let r = right.to_string_box(); + Box::new(StringBox::new(format!("{}{}", l.value, r.value))) } -/// IntegerBox - IntegerBox -> IntegerBox -impl NyashSub for IntegerBox { - type Output = IntegerBox; - - fn sub(self, rhs: IntegerBox) -> Self::Output { - IntegerBox::new(self.value - rhs.value) - } +#[inline] +fn can_repeat(times: i64) -> bool { + (0..=10_000).contains(×) } -/// IntegerBox * IntegerBox -> IntegerBox -impl NyashMul for IntegerBox { - type Output = IntegerBox; - - fn mul(self, rhs: IntegerBox) -> Self::Output { - IntegerBox::new(self.value * rhs.value) - } -} - -/// IntegerBox / IntegerBox -> IntegerBox (with zero check) -impl NyashDiv for IntegerBox { - type Output = Result; - - fn div(self, rhs: IntegerBox) -> Self::Output { - if rhs.value == 0 { - Err(OperatorError::DivisionByZero) - } else { - Ok(IntegerBox::new(self.value / rhs.value)) +// ===== Static numeric operators (macro-generated) ===== +macro_rules! impl_static_numeric_ops { + ($ty:ty, $zero:expr) => { + impl NyashAdd<$ty> for $ty { + type Output = $ty; + fn add(self, rhs: $ty) -> Self::Output { + < $ty >::new(self.value + rhs.value) + } } - } + + impl NyashSub<$ty> for $ty { + type Output = $ty; + fn sub(self, rhs: $ty) -> Self::Output { + < $ty >::new(self.value - rhs.value) + } + } + + impl NyashMul<$ty> for $ty { + type Output = $ty; + fn mul(self, rhs: $ty) -> Self::Output { + < $ty >::new(self.value * rhs.value) + } + } + + impl NyashDiv<$ty> for $ty { + type Output = Result<$ty, OperatorError>; + fn div(self, rhs: $ty) -> Self::Output { + if rhs.value == $zero { + Err(OperatorError::DivisionByZero) + } else { + Ok(< $ty >::new(self.value / rhs.value)) + } + } + } + }; } +impl_static_numeric_ops!(IntegerBox, 0); + /// Dynamic dispatch implementation for IntegerBox impl DynamicAdd for IntegerBox { fn try_add(&self, other: &dyn NyashBox) -> Option> { @@ -73,14 +83,8 @@ impl DynamicAdd for IntegerBox { ))); } - // Fallback: Convert both to strings and concatenate - // This preserves the existing AddBox behavior - let left_str = self.to_string_box(); - let right_str = other.to_string_box(); - Some(Box::new(StringBox::new(format!( - "{}{}", - left_str.value, right_str.value - )))) + // Fallback: Convert both to strings and concatenate (existing AddBox behavior) + Some(concat_result(self, other)) } fn can_add_with(&self, other_type: &str) -> bool { @@ -126,8 +130,7 @@ impl DynamicMul for IntegerBox { // IntegerBox * StringBox -> Repeated string if let Some(other_str) = other.as_any().downcast_ref::() { - if self.value >= 0 && self.value <= 10000 { - // Safety limit + if can_repeat(self.value) { let repeated = other_str.value.repeat(self.value as usize); return Some(Box::new(StringBox::new(repeated))); } @@ -171,47 +174,7 @@ impl DynamicDiv for IntegerBox { } } -// ===== FloatBox Operator Implementations ===== - -/// FloatBox + FloatBox -> FloatBox -impl NyashAdd for FloatBox { - type Output = FloatBox; - - fn add(self, rhs: FloatBox) -> Self::Output { - FloatBox::new(self.value + rhs.value) - } -} - -/// FloatBox - FloatBox -> FloatBox -impl NyashSub for FloatBox { - type Output = FloatBox; - - fn sub(self, rhs: FloatBox) -> Self::Output { - FloatBox::new(self.value - rhs.value) - } -} - -/// FloatBox * FloatBox -> FloatBox -impl NyashMul for FloatBox { - type Output = FloatBox; - - fn mul(self, rhs: FloatBox) -> Self::Output { - FloatBox::new(self.value * rhs.value) - } -} - -/// FloatBox / FloatBox -> FloatBox (with zero check) -impl NyashDiv for FloatBox { - type Output = Result; - - fn div(self, rhs: FloatBox) -> Self::Output { - if rhs.value == 0.0 { - Err(OperatorError::DivisionByZero) - } else { - Ok(FloatBox::new(self.value / rhs.value)) - } - } -} +impl_static_numeric_ops!(FloatBox, 0.0); // ===== FloatBox Dynamic Operator Implementations ===== @@ -228,12 +191,7 @@ impl DynamicAdd for FloatBox { } // Fallback: Convert both to strings and concatenate - let left_str = self.to_string_box(); - let right_str = other.to_string_box(); - Some(Box::new(StringBox::new(format!( - "{}{}", - left_str.value, right_str.value - )))) + Some(concat_result(self, other)) } fn can_add_with(&self, other_type: &str) -> bool { @@ -344,11 +302,7 @@ impl DynamicAdd for StringBox { } // StringBox + any other type -> Convert to string and concatenate - let other_str = other.to_string_box(); - Some(Box::new(StringBox::new(format!( - "{}{}", - self.value, other_str.value - )))) + Some(concat_result(self, other)) } fn can_add_with(&self, _other_type: &str) -> bool { @@ -370,8 +324,7 @@ impl DynamicMul for StringBox { fn try_mul(&self, other: &dyn NyashBox) -> Option> { // StringBox * IntegerBox -> Repeated string if let Some(other_int) = other.as_any().downcast_ref::() { - if other_int.value >= 0 && other_int.value <= 10000 { - // Safety limit + if can_repeat(other_int.value) { let repeated = self.value.repeat(other_int.value as usize); return Some(Box::new(StringBox::new(repeated))); } @@ -505,6 +458,67 @@ impl DynamicDiv for BoolBox { pub struct OperatorResolver; impl OperatorResolver { + #[inline] + fn try_dyn_left_add(left: &dyn NyashBox, right: &dyn NyashBox) -> Option> { + if let Some(int_box) = left.as_any().downcast_ref::() { + if let Some(result) = int_box.try_add(right) { return Some(result); } + } + if let Some(str_box) = left.as_any().downcast_ref::() { + if let Some(result) = str_box.try_add(right) { return Some(result); } + } + if let Some(float_box) = left.as_any().downcast_ref::() { + if let Some(result) = float_box.try_add(right) { return Some(result); } + } + if let Some(bool_box) = left.as_any().downcast_ref::() { + if let Some(result) = bool_box.try_add(right) { return Some(result); } + } + None + } + + #[inline] + fn try_dyn_left_sub(left: &dyn NyashBox, right: &dyn NyashBox) -> Option> { + if let Some(int_box) = left.as_any().downcast_ref::() { + if let Some(result) = int_box.try_sub(right) { return Some(result); } + } + if let Some(float_box) = left.as_any().downcast_ref::() { + if let Some(result) = float_box.try_sub(right) { return Some(result); } + } + if let Some(bool_box) = left.as_any().downcast_ref::() { + if let Some(result) = bool_box.try_sub(right) { return Some(result); } + } + None + } + + #[inline] + fn try_dyn_left_mul(left: &dyn NyashBox, right: &dyn NyashBox) -> Option> { + if let Some(int_box) = left.as_any().downcast_ref::() { + if let Some(result) = int_box.try_mul(right) { return Some(result); } + } + if let Some(str_box) = left.as_any().downcast_ref::() { + if let Some(result) = str_box.try_mul(right) { return Some(result); } + } + if let Some(float_box) = left.as_any().downcast_ref::() { + if let Some(result) = float_box.try_mul(right) { return Some(result); } + } + if let Some(bool_box) = left.as_any().downcast_ref::() { + if let Some(result) = bool_box.try_mul(right) { return Some(result); } + } + None + } + + #[inline] + fn try_dyn_left_div(left: &dyn NyashBox, right: &dyn NyashBox) -> Option> { + if let Some(int_box) = left.as_any().downcast_ref::() { + return int_box.try_div(right); + } + if let Some(float_box) = left.as_any().downcast_ref::() { + return float_box.try_div(right); + } + if let Some(bool_box) = left.as_any().downcast_ref::() { + return bool_box.try_div(right); + } + None + } /// Resolve addition operation with hybrid dispatch pub fn resolve_add( left: &dyn NyashBox, @@ -513,33 +527,7 @@ impl OperatorResolver { // Try to cast to concrete types first and use their DynamicAdd implementation // This approach uses the concrete types rather than trait objects - // Check if left implements DynamicAdd by trying common types - if let Some(int_box) = left.as_any().downcast_ref::() { - if let Some(result) = int_box.try_add(right) { - return Ok(result); - } - } - - if let Some(str_box) = left.as_any().downcast_ref::() { - if let Some(result) = str_box.try_add(right) { - return Ok(result); - } - } - - if let Some(float_box) = right - .as_any() - .downcast_ref::() - { - if let Some(result) = float_box.try_add(right) { - return Ok(result); - } - } - - if let Some(bool_box) = left.as_any().downcast_ref::() { - if let Some(result) = bool_box.try_add(right) { - return Ok(result); - } - } + if let Some(result) = Self::try_dyn_left_add(left, right) { return Ok(result); } Err(OperatorError::UnsupportedOperation { operator: "+".to_string(), @@ -553,21 +541,7 @@ impl OperatorResolver { left: &dyn NyashBox, right: &dyn NyashBox, ) -> Result, OperatorError> { - // Try concrete types for DynamicSub - if let Some(int_box) = left.as_any().downcast_ref::() { - if let Some(result) = int_box.try_sub(right) { - return Ok(result); - } - } - - if let Some(float_box) = left - .as_any() - .downcast_ref::() - { - if let Some(result) = float_box.try_sub(right) { - return Ok(result); - } - } + if let Some(result) = Self::try_dyn_left_sub(left, right) { return Ok(result); } Err(OperatorError::UnsupportedOperation { operator: "-".to_string(), @@ -581,33 +555,7 @@ impl OperatorResolver { left: &dyn NyashBox, right: &dyn NyashBox, ) -> Result, OperatorError> { - // Try concrete types for DynamicMul - if let Some(int_box) = left.as_any().downcast_ref::() { - if let Some(result) = int_box.try_mul(right) { - return Ok(result); - } - } - - if let Some(str_box) = left.as_any().downcast_ref::() { - if let Some(result) = str_box.try_mul(right) { - return Ok(result); - } - } - - if let Some(float_box) = left - .as_any() - .downcast_ref::() - { - if let Some(result) = float_box.try_mul(right) { - return Ok(result); - } - } - - if let Some(bool_box) = left.as_any().downcast_ref::() { - if let Some(result) = bool_box.try_mul(right) { - return Ok(result); - } - } + if let Some(result) = Self::try_dyn_left_mul(left, right) { return Ok(result); } Err(OperatorError::UnsupportedOperation { operator: "*".to_string(), @@ -621,35 +569,7 @@ impl OperatorResolver { left: &dyn NyashBox, right: &dyn NyashBox, ) -> Result, OperatorError> { - // Try concrete types for DynamicDiv - if let Some(int_box) = left.as_any().downcast_ref::() { - if let Some(result) = int_box.try_div(right) { - return Ok(result); - } else { - // If try_div returns None, it might be division by zero - return Err(OperatorError::DivisionByZero); - } - } - - if let Some(float_box) = left - .as_any() - .downcast_ref::() - { - if let Some(result) = float_box.try_div(right) { - return Ok(result); - } else { - // If try_div returns None, it might be division by zero - return Err(OperatorError::DivisionByZero); - } - } - - if let Some(bool_box) = left.as_any().downcast_ref::() { - if let Some(result) = bool_box.try_div(right) { - return Ok(result); - } else { - return Err(OperatorError::DivisionByZero); - } - } + if let Some(result) = Self::try_dyn_left_div(left, right) { return Ok(result); } Err(OperatorError::UnsupportedOperation { operator: "/".to_string(), diff --git a/src/mir/instruction_kinds/mod.rs b/src/mir/instruction_kinds/mod.rs index af5a31f8..14c9b40f 100644 --- a/src/mir/instruction_kinds/mod.rs +++ b/src/mir/instruction_kinds/mod.rs @@ -11,6 +11,37 @@ use crate::mir::instruction::{ TypeOpKind as MirTypeOpKind, WeakRefOp as MirWeakRefOp, }; +// Local macro utilities for generating InstructionMeta boilerplate. +// This macro is intentionally scoped to this module to avoid polluting the crate namespace. +macro_rules! inst_meta { + ( + $( + pub struct $name:ident { $($field:ident : $fty:ty),* $(,)? } + => { + from_mir = |$i:ident| $from_expr:expr; + effects = $effects:expr; + dst = $dst:expr; + used = $used:expr; + } + )+ + ) => { + $( + #[derive(Debug, Clone)] + pub struct $name { $(pub $field: $fty),* } + + impl $name { + pub fn from_mir($i: &MirInstruction) -> Option { $from_expr } + } + + impl InstructionMeta for $name { + fn effects(&self) -> EffectMask { ($effects)(self) } + fn dst(&self) -> Option { ($dst)(self) } + fn used(&self) -> Vec { ($used)(self) } + } + )+ + }; +} + pub trait InstructionMeta { fn effects(&self) -> EffectMask; fn dst(&self) -> Option; @@ -130,499 +161,289 @@ pub fn used_via_meta(i: &MirInstruction) -> Option> { None } -// ---- BarrierRead ---- -#[derive(Debug, Clone, Copy)] -pub struct BarrierReadInst { pub ptr: ValueId } - -impl BarrierReadInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::BarrierRead { ptr } => Some(BarrierReadInst { ptr: *ptr }), _ => None } +// ---- BarrierRead ---- (macro-generated) +inst_meta! { + pub struct BarrierReadInst { ptr: ValueId } + => { + from_mir = |i| match i { MirInstruction::BarrierRead { ptr } => Some(BarrierReadInst { ptr: *ptr }), _ => None }; + effects = |_: &Self| EffectMask::READ.add(Effect::Barrier); + dst = |_: &Self| None; + used = |s: &Self| vec![s.ptr]; } } -impl InstructionMeta for BarrierReadInst { - fn effects(&self) -> EffectMask { EffectMask::READ.add(Effect::Barrier) } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.ptr] } -} - -// ---- BarrierWrite ---- -#[derive(Debug, Clone, Copy)] -pub struct BarrierWriteInst { pub ptr: ValueId } - -impl BarrierWriteInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::BarrierWrite { ptr } => Some(BarrierWriteInst { ptr: *ptr }), _ => None } +// ---- BarrierWrite ---- (macro-generated) +inst_meta! { + pub struct BarrierWriteInst { ptr: ValueId } + => { + from_mir = |i| match i { MirInstruction::BarrierWrite { ptr } => Some(BarrierWriteInst { ptr: *ptr }), _ => None }; + effects = |_: &Self| EffectMask::WRITE.add(Effect::Barrier); + dst = |_: &Self| None; + used = |s: &Self| vec![s.ptr]; } } -impl InstructionMeta for BarrierWriteInst { - fn effects(&self) -> EffectMask { EffectMask::WRITE.add(Effect::Barrier) } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.ptr] } -} - -// ---- Barrier (unified) ---- -#[derive(Debug, Clone, Copy)] -pub struct BarrierInst { pub op: MirBarrierOp, pub ptr: ValueId } - -impl BarrierInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::Barrier { op, ptr } => Some(BarrierInst { op: *op, ptr: *ptr }), _ => None } +// ---- Barrier (unified) ---- (macro-generated) +inst_meta! { + pub struct BarrierInst { op: MirBarrierOp, ptr: ValueId } + => { + from_mir = |i| match i { MirInstruction::Barrier { op, ptr } => Some(BarrierInst { op: *op, ptr: *ptr }), _ => None }; + effects = |s: &Self| match s.op { MirBarrierOp::Read => EffectMask::READ.add(Effect::Barrier), MirBarrierOp::Write => EffectMask::WRITE.add(Effect::Barrier) }; + dst = |_: &Self| None; + used = |s: &Self| vec![s.ptr]; } } -impl InstructionMeta for BarrierInst { - fn effects(&self) -> EffectMask { - match self.op { - MirBarrierOp::Read => EffectMask::READ.add(Effect::Barrier), - MirBarrierOp::Write => EffectMask::WRITE.add(Effect::Barrier), - } - } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.ptr] } -} - -// ---- Ref ops ---- -#[derive(Debug, Clone, Copy)] -pub struct RefNewInst { pub dst: ValueId, pub box_val: ValueId } -impl RefNewInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::RefNew { dst, box_val } => Some(RefNewInst { dst: *dst, box_val: *box_val }), _ => None } +// ---- Ref ops ---- (macro-generated) +inst_meta! { + pub struct RefNewInst { dst: ValueId, box_val: ValueId } + => { + from_mir = |i| match i { MirInstruction::RefNew { dst, box_val } => Some(RefNewInst { dst: *dst, box_val: *box_val }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.box_val]; } } -impl InstructionMeta for RefNewInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.box_val] } -} - -#[derive(Debug, Clone, Copy)] -pub struct RefGetInst { pub dst: ValueId, pub reference: ValueId } -impl RefGetInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::RefGet { dst, reference, .. } => Some(RefGetInst { dst: *dst, reference: *reference }), _ => None } +inst_meta! { + pub struct RefGetInst { dst: ValueId, reference: ValueId } + => { + from_mir = |i| match i { MirInstruction::RefGet { dst, reference, .. } => Some(RefGetInst { dst: *dst, reference: *reference }), _ => None }; + effects = |_: &Self| EffectMask::READ; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.reference]; } } -impl InstructionMeta for RefGetInst { - fn effects(&self) -> EffectMask { EffectMask::READ } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.reference] } -} - -#[derive(Debug, Clone, Copy)] -pub struct RefSetInst { pub reference: ValueId, pub value: ValueId } -impl RefSetInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::RefSet { reference, value, .. } => Some(RefSetInst { reference: *reference, value: *value }), _ => None } - } -} -impl InstructionMeta for RefSetInst { - fn effects(&self) -> EffectMask { EffectMask::WRITE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.reference, self.value] } -} - -// ---- Weak ops ---- -#[derive(Debug, Clone, Copy)] -pub struct WeakNewInst { pub dst: ValueId, pub box_val: ValueId } -impl WeakNewInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::WeakNew { dst, box_val } => Some(WeakNewInst { dst: *dst, box_val: *box_val }), _ => None } - } -} -impl InstructionMeta for WeakNewInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.box_val] } -} - -#[derive(Debug, Clone, Copy)] -pub struct WeakLoadInst { pub dst: ValueId, pub weak_ref: ValueId } -impl WeakLoadInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::WeakLoad { dst, weak_ref } => Some(WeakLoadInst { dst: *dst, weak_ref: *weak_ref }), _ => None } - } -} -impl InstructionMeta for WeakLoadInst { - fn effects(&self) -> EffectMask { EffectMask::READ } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.weak_ref] } -} - -#[derive(Debug, Clone, Copy)] -pub struct WeakRefInst { pub dst: ValueId, pub op: MirWeakRefOp, pub value: ValueId } -impl WeakRefInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::WeakRef { dst, op, value } => Some(WeakRefInst { dst: *dst, op: *op, value: *value }), _ => None } - } -} -impl InstructionMeta for WeakRefInst { - fn effects(&self) -> EffectMask { - match self.op { - MirWeakRefOp::New => EffectMask::PURE, - MirWeakRefOp::Load => EffectMask::READ, - } - } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.value] } -} - -// ---- Future ops ---- -#[derive(Debug, Clone, Copy)] -pub struct FutureNewInst { pub dst: ValueId, pub value: ValueId } -impl FutureNewInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::FutureNew { dst, value } => Some(FutureNewInst { dst: *dst, value: *value }), _ => None } - } -} -impl InstructionMeta for FutureNewInst { - fn effects(&self) -> EffectMask { EffectMask::PURE.add(Effect::Alloc) } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.value] } -} - -#[derive(Debug, Clone, Copy)] -pub struct FutureSetInst { pub future: ValueId, pub value: ValueId } -impl FutureSetInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::FutureSet { future, value } => Some(FutureSetInst { future: *future, value: *value }), _ => None } - } -} -impl InstructionMeta for FutureSetInst { - fn effects(&self) -> EffectMask { EffectMask::WRITE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.future, self.value] } -} - -#[derive(Debug, Clone, Copy)] -pub struct AwaitInst { pub dst: ValueId, pub future: ValueId } -impl AwaitInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::Await { dst, future } => Some(AwaitInst { dst: *dst, future: *future }), _ => None } - } -} -impl InstructionMeta for AwaitInst { - fn effects(&self) -> EffectMask { EffectMask::READ.add(Effect::Async) } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.future] } -} - -// ---- UnaryOp ---- -#[derive(Debug, Clone, Copy)] -pub struct UnaryOpInst { - pub dst: ValueId, - pub operand: ValueId, -} - -impl UnaryOpInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::UnaryOp { dst, operand, .. } => Some(UnaryOpInst { dst: *dst, operand: *operand }), - _ => None, - } +inst_meta! { + pub struct RefSetInst { reference: ValueId, value: ValueId } + => { + from_mir = |i| match i { MirInstruction::RefSet { reference, value, .. } => Some(RefSetInst { reference: *reference, value: *value }), _ => None }; + effects = |_: &Self| EffectMask::WRITE; + dst = |_: &Self| None; + used = |s: &Self| vec![s.reference, s.value]; } } -impl InstructionMeta for UnaryOpInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.operand] } +// ---- Weak ops ---- (macro-generated) +inst_meta! { + pub struct WeakNewInst { dst: ValueId, box_val: ValueId } + => { + from_mir = |i| match i { MirInstruction::WeakNew { dst, box_val } => Some(WeakNewInst { dst: *dst, box_val: *box_val }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.box_val]; + } } - -// ---- Compare ---- -#[derive(Debug, Clone, Copy)] -pub struct CompareInst { - pub dst: ValueId, - pub lhs: ValueId, - pub rhs: ValueId, +inst_meta! { + pub struct WeakLoadInst { dst: ValueId, weak_ref: ValueId } + => { + from_mir = |i| match i { MirInstruction::WeakLoad { dst, weak_ref } => Some(WeakLoadInst { dst: *dst, weak_ref: *weak_ref }), _ => None }; + effects = |_: &Self| EffectMask::READ; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.weak_ref]; + } } - -impl CompareInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Compare { dst, lhs, rhs, .. } => Some(CompareInst { dst: *dst, lhs: *lhs, rhs: *rhs }), - _ => None, - } +inst_meta! { + pub struct WeakRefInst { dst: ValueId, op: MirWeakRefOp, value: ValueId } + => { + from_mir = |i| match i { MirInstruction::WeakRef { dst, op, value } => Some(WeakRefInst { dst: *dst, op: *op, value: *value }), _ => None }; + effects = |s: &Self| match s.op { MirWeakRefOp::New => EffectMask::PURE, MirWeakRefOp::Load => EffectMask::READ }; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.value]; } } -impl InstructionMeta for CompareInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.lhs, self.rhs] } +// ---- Future ops ---- (macro-generated) +inst_meta! { + pub struct FutureNewInst { dst: ValueId, value: ValueId } + => { + from_mir = |i| match i { MirInstruction::FutureNew { dst, value } => Some(FutureNewInst { dst: *dst, value: *value }), _ => None }; + effects = |_: &Self| EffectMask::PURE.add(Effect::Alloc); + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.value]; + } } - -// ---- Load ---- -#[derive(Debug, Clone, Copy)] -pub struct LoadInst { - pub dst: ValueId, - pub ptr: ValueId, +inst_meta! { + pub struct FutureSetInst { future: ValueId, value: ValueId } + => { + from_mir = |i| match i { MirInstruction::FutureSet { future, value } => Some(FutureSetInst { future: *future, value: *value }), _ => None }; + effects = |_: &Self| EffectMask::WRITE; + dst = |_: &Self| None; + used = |s: &Self| vec![s.future, s.value]; + } } - -impl LoadInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Load { dst, ptr } => Some(LoadInst { dst: *dst, ptr: *ptr }), - _ => None, - } +inst_meta! { + pub struct AwaitInst { dst: ValueId, future: ValueId } + => { + from_mir = |i| match i { MirInstruction::Await { dst, future } => Some(AwaitInst { dst: *dst, future: *future }), _ => None }; + effects = |_: &Self| EffectMask::READ.add(Effect::Async); + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.future]; } } -impl InstructionMeta for LoadInst { - fn effects(&self) -> EffectMask { EffectMask::READ } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.ptr] } -} - -// ---- Cast ---- -#[derive(Debug, Clone)] -pub struct CastInst { - pub dst: ValueId, - pub value: ValueId, - pub target_type: MirType, -} - -impl CastInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Cast { dst, value, target_type } => - Some(CastInst { dst: *dst, value: *value, target_type: target_type.clone() }), - _ => None, - } +// ---- UnaryOp ---- (macro-generated) +inst_meta! { + pub struct UnaryOpInst { dst: ValueId, operand: ValueId } + => { + from_mir = |i| match i { MirInstruction::UnaryOp { dst, operand, .. } => Some(UnaryOpInst { dst: *dst, operand: *operand }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.operand]; } } -impl InstructionMeta for CastInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.value] } -} - -// ---- TypeOp ---- -#[derive(Debug, Clone)] -pub struct TypeOpInst { - pub dst: ValueId, - pub op: MirTypeOpKind, - pub value: ValueId, - pub ty: MirType, -} - -impl TypeOpInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::TypeOp { dst, op, value, ty } => - Some(TypeOpInst { dst: *dst, op: *op, value: *value, ty: ty.clone() }), - _ => None, - } +// ---- Compare ---- (macro-generated) +inst_meta! { + pub struct CompareInst { dst: ValueId, lhs: ValueId, rhs: ValueId } + => { + from_mir = |i| match i { MirInstruction::Compare { dst, lhs, rhs, .. } => Some(CompareInst { dst: *dst, lhs: *lhs, rhs: *rhs }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.lhs, s.rhs]; } } -impl InstructionMeta for TypeOpInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.value] } -} - -// ---- ArrayGet ---- -#[derive(Debug, Clone, Copy)] -pub struct ArrayGetInst { - pub dst: ValueId, - pub array: ValueId, - pub index: ValueId, -} - -impl ArrayGetInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::ArrayGet { dst, array, index } => - Some(ArrayGetInst { dst: *dst, array: *array, index: *index }), - _ => None, - } +// ---- Load ---- (macro-generated) +inst_meta! { + pub struct LoadInst { dst: ValueId, ptr: ValueId } + => { + from_mir = |i| match i { MirInstruction::Load { dst, ptr } => Some(LoadInst { dst: *dst, ptr: *ptr }), _ => None }; + effects = |_: &Self| EffectMask::READ; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.ptr]; } } -impl InstructionMeta for ArrayGetInst { - fn effects(&self) -> EffectMask { EffectMask::READ } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { vec![self.array, self.index] } -} - -// ---- Phi ---- -#[derive(Debug, Clone)] -pub struct PhiInst { pub dst: ValueId, pub inputs: Vec<(BasicBlockId, ValueId)> } - -impl PhiInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Phi { dst, inputs } => Some(PhiInst { dst: *dst, inputs: inputs.clone() }), - _ => None, - } +// ---- Cast ---- (macro-generated) +inst_meta! { + pub struct CastInst { dst: ValueId, value: ValueId, target_type: MirType } + => { + from_mir = |i| match i { MirInstruction::Cast { dst, value, target_type } => Some(CastInst { dst: *dst, value: *value, target_type: target_type.clone() }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.value]; } } -impl InstructionMeta for PhiInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { self.inputs.iter().map(|(_, v)| *v).collect() } -} - -// ---- NewBox ---- -#[derive(Debug, Clone)] -pub struct NewBoxInst { - pub dst: ValueId, - pub args: Vec, -} - -impl NewBoxInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::NewBox { dst, args, .. } => - Some(NewBoxInst { dst: *dst, args: args.clone() }), - _ => None, - } +// ---- TypeOp ---- (macro-generated) +inst_meta! { + pub struct TypeOpInst { dst: ValueId, op: MirTypeOpKind, value: ValueId, ty: MirType } + => { + from_mir = |i| match i { MirInstruction::TypeOp { dst, op, value, ty } => Some(TypeOpInst { dst: *dst, op: *op, value: *value, ty: ty.clone() }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.value]; } } -impl InstructionMeta for NewBoxInst { - fn effects(&self) -> EffectMask { EffectMask::PURE.add(Effect::Alloc) } - fn dst(&self) -> Option { Some(self.dst) } - fn used(&self) -> Vec { self.args.clone() } -} - -// ---- Store ---- -#[derive(Debug, Clone, Copy)] -pub struct StoreInst { - pub value: ValueId, - pub ptr: ValueId, -} - -impl StoreInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Store { value, ptr } => Some(StoreInst { value: *value, ptr: *ptr }), - _ => None, - } +// ---- ArrayGet ---- (macro-generated) +inst_meta! { + pub struct ArrayGetInst { dst: ValueId, array: ValueId, index: ValueId } + => { + from_mir = |i| match i { MirInstruction::ArrayGet { dst, array, index } => Some(ArrayGetInst { dst: *dst, array: *array, index: *index }), _ => None }; + effects = |_: &Self| EffectMask::READ; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| vec![s.array, s.index]; } } -impl InstructionMeta for StoreInst { - fn effects(&self) -> EffectMask { EffectMask::WRITE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.value, self.ptr] } -} - -// ---- ArraySet ---- -#[derive(Debug, Clone, Copy)] -pub struct ArraySetInst { - pub array: ValueId, - pub index: ValueId, - pub value: ValueId, -} - -impl ArraySetInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::ArraySet { array, index, value } => - Some(ArraySetInst { array: *array, index: *index, value: *value }), - _ => None, - } +// ---- Phi ---- (macro-generated) +inst_meta! { + pub struct PhiInst { dst: ValueId, inputs: Vec<(BasicBlockId, ValueId)> } + => { + from_mir = |i| match i { MirInstruction::Phi { dst, inputs } => Some(PhiInst { dst: *dst, inputs: inputs.clone() }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |s: &Self| Some(s.dst); + used = |s: &Self| s.inputs.iter().map(|(_, v)| *v).collect(); } } -impl InstructionMeta for ArraySetInst { - fn effects(&self) -> EffectMask { EffectMask::WRITE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.array, self.index, self.value] } -} - -// ---- Return ---- -#[derive(Debug, Clone, Copy)] -pub struct ReturnInst { pub value: Option } - -impl ReturnInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Return { value } => Some(ReturnInst { value: *value }), - _ => None, - } +// ---- NewBox ---- (macro-generated) +inst_meta! { + pub struct NewBoxInst { dst: ValueId, args: Vec } + => { + from_mir = |i| match i { MirInstruction::NewBox { dst, args, .. } => Some(NewBoxInst { dst: *dst, args: args.clone() }), _ => None }; + effects = |_: &Self| EffectMask::PURE.add(Effect::Alloc); + dst = |s: &Self| Some(s.dst); + used = |s: &Self| s.args.clone(); } } -impl InstructionMeta for ReturnInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { self.value.map(|v| vec![v]).unwrap_or_default() } -} - -// ---- Branch ---- -#[derive(Debug, Clone, Copy)] -pub struct BranchInst { pub condition: ValueId } - -impl BranchInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Branch { condition, .. } => Some(BranchInst { condition: *condition }), - _ => None, - } +// ---- Store ---- (macro-generated) +inst_meta! { + pub struct StoreInst { value: ValueId, ptr: ValueId } + => { + from_mir = |i| match i { MirInstruction::Store { value, ptr } => Some(StoreInst { value: *value, ptr: *ptr }), _ => None }; + effects = |_: &Self| EffectMask::WRITE; + dst = |_: &Self| None; + used = |s: &Self| vec![s.value, s.ptr]; } } -impl InstructionMeta for BranchInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.condition] } -} - -// ---- Jump ---- -#[derive(Debug, Clone, Copy)] -pub struct JumpInst; - -impl JumpInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::Jump { .. } => Some(JumpInst), _ => None } +// ---- ArraySet ---- (macro-generated) +inst_meta! { + pub struct ArraySetInst { array: ValueId, index: ValueId, value: ValueId } + => { + from_mir = |i| match i { MirInstruction::ArraySet { array, index, value } => Some(ArraySetInst { array: *array, index: *index, value: *value }), _ => None }; + effects = |_: &Self| EffectMask::WRITE; + dst = |_: &Self| None; + used = |s: &Self| vec![s.array, s.index, s.value]; } } -impl InstructionMeta for JumpInst { - fn effects(&self) -> EffectMask { EffectMask::PURE } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { Vec::new() } -} - -// ---- Print ---- -#[derive(Debug, Clone, Copy)] -pub struct PrintInst { pub value: ValueId, pub effects_mask: EffectMask } - -impl PrintInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { - MirInstruction::Print { value, effects } => Some(PrintInst { value: *value, effects_mask: *effects }), - _ => None, - } +// ---- Return ---- (macro-generated) +inst_meta! { + pub struct ReturnInst { value: Option } + => { + from_mir = |i| match i { MirInstruction::Return { value } => Some(ReturnInst { value: *value }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |_: &Self| None; + used = |s: &Self| s.value.map(|v| vec![v]).unwrap_or_default(); } } -impl InstructionMeta for PrintInst { - fn effects(&self) -> EffectMask { self.effects_mask } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.value] } -} - -// ---- Debug ---- -#[derive(Debug, Clone, Copy)] -pub struct DebugInst { pub value: ValueId } - -impl DebugInst { - pub fn from_mir(i: &MirInstruction) -> Option { - match i { MirInstruction::Debug { value, .. } => Some(DebugInst { value: *value }), _ => None } +// ---- Branch ---- (macro-generated) +inst_meta! { + pub struct BranchInst { condition: ValueId } + => { + from_mir = |i| match i { MirInstruction::Branch { condition, .. } => Some(BranchInst { condition: *condition }), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |_: &Self| None; + used = |s: &Self| vec![s.condition]; } } -impl InstructionMeta for DebugInst { - fn effects(&self) -> EffectMask { EffectMask::PURE.add(Effect::Debug) } - fn dst(&self) -> Option { None } - fn used(&self) -> Vec { vec![self.value] } +// ---- Jump ---- (macro-generated) +inst_meta! { + pub struct JumpInst { } + => { + from_mir = |i| match i { MirInstruction::Jump { .. } => Some(JumpInst {}), _ => None }; + effects = |_: &Self| EffectMask::PURE; + dst = |_: &Self| None; + used = |_: &Self| Vec::new(); + } +} + +// ---- Print ---- (macro-generated) +inst_meta! { + pub struct PrintInst { value: ValueId, effects_mask: EffectMask } + => { + from_mir = |i| match i { MirInstruction::Print { value, effects } => Some(PrintInst { value: *value, effects_mask: *effects }), _ => None }; + effects = |s: &Self| s.effects_mask; + dst = |_: &Self| None; + used = |s: &Self| vec![s.value]; + } +} + +// ---- Debug ---- (macro-generated) +inst_meta! { + pub struct DebugInst { value: ValueId } + => { + from_mir = |i| match i { MirInstruction::Debug { value, .. } => Some(DebugInst { value: *value }), _ => None }; + effects = |_: &Self| EffectMask::PURE.add(Effect::Debug); + dst = |_: &Self| None; + used = |s: &Self| vec![s.value]; + } } // ---- Call-like (dst/used only; effects fallback in MirInstruction) ---- diff --git a/src/tests/typebox_tlv_diff.rs b/src/tests/typebox_tlv_diff.rs index e2edae1d..ae149785 100644 --- a/src/tests/typebox_tlv_diff.rs +++ b/src/tests/typebox_tlv_diff.rs @@ -3,18 +3,88 @@ mod tests { use crate::box_trait::{IntegerBox, NyashBox, StringBox}; use crate::boxes::array::ArrayBox; use crate::boxes::math_box::FloatBox; + use crate::runtime::plugin_loader_unified::PluginHost; use std::env; use std::fs; use std::path::PathBuf; + // RAII: environment variable guard (restores on drop) + struct EnvGuard { + key: &'static str, + prev: Option, + } + + impl EnvGuard { + fn set(key: &'static str, val: &str) -> Self { + let prev = env::var(key).ok(); + env::set_var(key, val); + EnvGuard { key, prev } + } + + fn remove(key: &'static str) -> Self { + let prev = env::var(key).ok(); + env::remove_var(key); + EnvGuard { key, prev } + } + } + + impl Drop for EnvGuard { + fn drop(&mut self) { + match &self.prev { + Some(v) => env::set_var(self.key, v), + None => env::remove_var(self.key), + } + } + } + + // Helper: read-lock the global plugin host and pass immutable ref to closure + fn with_host(f: impl FnOnce(&PluginHost) -> R) -> R { + let host = crate::runtime::get_global_plugin_host(); + let guard = host.read().expect("plugin host RwLock poisoned"); + f(&*guard) + } + + // ---- Test helpers (invoke wrappers) ---- + fn inv_ok( + h: &PluginHost, + box_ty: &str, + method: &str, + id: u32, + args: &[Box], + ) -> Option> { + h.invoke_instance_method(box_ty, method, id, args) + .expect(&format!("invoke {}::{}", box_ty, method)) + } + + fn inv_some( + h: &PluginHost, + box_ty: &str, + method: &str, + id: u32, + args: &[Box], + ) -> Box { + inv_ok(h, box_ty, method, id, args) + .unwrap_or_else(|| panic!("{}::{} returned None", box_ty, method)) + } + + fn inv_void( + h: &PluginHost, + box_ty: &str, + method: &str, + id: u32, + args: &[Box], + ) { + let _ = h + .invoke_instance_method(box_ty, method, id, args) + .expect(&format!("invoke {}::{}", box_ty, method)); + } + fn ensure_host() { let _ = crate::runtime::init_global_plugin_host("nyash.toml"); } fn create_plugin_instance(box_type: &str) -> (String, u32, Box) { - let host = crate::runtime::get_global_plugin_host(); - let host = host.read().unwrap(); - let bx = host.create_box(box_type, &[]).expect("create_box"); + let bx = with_host(|h| h.create_box(box_type, &[]).expect("create_box")); // Downcast to PluginBoxV2 to get instance_id if let Some(p) = bx .as_any() @@ -30,58 +100,25 @@ mod tests { #[ignore = "MIR13 parity: MapBox TLV vs TypeBox under unified BoxCall/TypeOp pending"] fn mapbox_get_set_size_tlv_vs_typebox() { ensure_host(); - let host = crate::runtime::get_global_plugin_host(); - - // TLV path: disable typebox - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + // TLV path: disable typebox (restored automatically) + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("MapBox"); - let out_tlv = { - let h = host.read().unwrap(); - // set("k", 42) - let _ = h - .invoke_instance_method( - &bt1, - "set", - id1, - &[Box::new(StringBox::new("k")), Box::new(IntegerBox::new(42))], - ) - .expect("set tlv"); - // size() - let sz = h - .invoke_instance_method(&bt1, "size", id1, &[]) - .expect("size tlv") - .unwrap(); - // get("k") - let gv = h - .invoke_instance_method(&bt1, "get", id1, &[Box::new(StringBox::new("k"))]) - .expect("get tlv") - .unwrap(); + let out_tlv = with_host(|h| { + inv_void(h, &bt1, "set", id1, &[Box::new(StringBox::new("k")), Box::new(IntegerBox::new(42))]); + let sz = inv_some(h, &bt1, "size", id1, &[]); + let gv = inv_some(h, &bt1, "get", id1, &[Box::new(StringBox::new("k"))]); (sz.to_string_box().value, gv.to_string_box().value) - }; + }); // TypeBox path: enable typebox - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("MapBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method( - &bt2, - "set", - id2, - &[Box::new(StringBox::new("k")), Box::new(IntegerBox::new(42))], - ) - .expect("set tb"); - let sz = h - .invoke_instance_method(&bt2, "size", id2, &[]) - .expect("size tb") - .unwrap(); - let gv = h - .invoke_instance_method(&bt2, "get", id2, &[Box::new(StringBox::new("k"))]) - .expect("get tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void(h, &bt2, "set", id2, &[Box::new(StringBox::new("k")), Box::new(IntegerBox::new(42))]); + let sz = inv_some(h, &bt2, "size", id2, &[]); + let gv = inv_some(h, &bt2, "get", id2, &[Box::new(StringBox::new("k"))]); (sz.to_string_box().value, gv.to_string_box().value) - }; + }); assert_eq!(out_tlv, out_tb, "TLV vs TypeBox results should match"); } @@ -90,53 +127,24 @@ mod tests { #[ignore = "MIR13 parity: ArrayBox len/get under unified ops pending"] fn arraybox_set_get_len_tlv_vs_typebox() { ensure_host(); - let host = crate::runtime::get_global_plugin_host(); - // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + // TLV path (guarded) + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("ArrayBox"); - let out_tlv = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method( - &bt1, - "set", - id1, - &[Box::new(IntegerBox::new(0)), Box::new(IntegerBox::new(7))], - ) - .expect("set tlv"); - let ln = h - .invoke_instance_method(&bt1, "len", id1, &[]) - .expect("len tlv") - .unwrap(); - let gv = h - .invoke_instance_method(&bt1, "get", id1, &[Box::new(IntegerBox::new(0))]) - .expect("get tlv") - .unwrap(); + let out_tlv = with_host(|h| { + inv_void(h, &bt1, "set", id1, &[Box::new(IntegerBox::new(0)), Box::new(IntegerBox::new(7))]); + let ln = inv_some(h, &bt1, "len", id1, &[]); + let gv = inv_some(h, &bt1, "get", id1, &[Box::new(IntegerBox::new(0))]); (ln.to_string_box().value, gv.to_string_box().value) - }; - // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + }); + // TypeBox path (guarded) + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("ArrayBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method( - &bt2, - "set", - id2, - &[Box::new(IntegerBox::new(0)), Box::new(IntegerBox::new(7))], - ) - .expect("set tb"); - let ln = h - .invoke_instance_method(&bt2, "length", id2, &[]) - .expect("len tb") - .unwrap(); - let gv = h - .invoke_instance_method(&bt2, "get", id2, &[Box::new(IntegerBox::new(0))]) - .expect("get tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void(h, &bt2, "set", id2, &[Box::new(IntegerBox::new(0)), Box::new(IntegerBox::new(7))]); + let ln = inv_some(h, &bt2, "length", id2, &[]); + let gv = inv_some(h, &bt2, "get", id2, &[Box::new(IntegerBox::new(0))]); (ln.to_string_box().value, gv.to_string_box().value) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (ArrayBox)" @@ -149,36 +157,22 @@ mod tests { ensure_host(); let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("StringBox"); - let out_tlv = { - let h = host.read().unwrap(); + let out_tlv = with_host(|h| { // birth with init string: use fromUtf8 via set of arg in create? Current loader birth() no-arg, so concat - let _ = h - .invoke_instance_method(&bt1, "concat", id1, &[Box::new(StringBox::new("ab"))]) - .expect("concat tlv") - .unwrap(); - let ln = h - .invoke_instance_method(&bt1, "length", id1, &[]) - .expect("len tlv") - .unwrap(); + inv_void(h, &bt1, "concat", id1, &[Box::new(StringBox::new("ab"))]); + let ln = inv_some(h, &bt1, "length", id1, &[]); (ln.to_string_box().value) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("StringBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt2, "concat", id2, &[Box::new(StringBox::new("ab"))]) - .expect("concat tb") - .unwrap(); - let ln = h - .invoke_instance_method(&bt2, "length", id2, &[]) - .expect("len tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void(h, &bt2, "concat", id2, &[Box::new(StringBox::new("ab"))]); + let ln = inv_some(h, &bt2, "length", id2, &[]); (ln.to_string_box().value) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (StringBox)" @@ -191,35 +185,21 @@ mod tests { ensure_host(); let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("IntegerBox"); - let out_tlv = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt1, "set", id1, &[Box::new(IntegerBox::new(123))]) - .expect("set tlv") - .unwrap(); - let gv = h - .invoke_instance_method(&bt1, "get", id1, &[]) - .expect("get tlv") - .unwrap(); + let out_tlv = with_host(|h| { + inv_void(h, &bt1, "set", id1, &[Box::new(IntegerBox::new(123))]); + let gv = inv_some(h, &bt1, "get", id1, &[]); gv.to_string_box().value - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("IntegerBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt2, "set", id2, &[Box::new(IntegerBox::new(123))]) - .expect("set tb") - .unwrap(); - let gv = h - .invoke_instance_method(&bt2, "get", id2, &[]) - .expect("get tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void(h, &bt2, "set", id2, &[Box::new(IntegerBox::new(123))]); + let gv = inv_some(h, &bt2, "get", id2, &[]); gv.to_string_box().value - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (IntegerBox)" @@ -232,25 +212,23 @@ mod tests { ensure_host(); let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("ConsoleBox"); - let out_tlv_is_none = { - let h = host.read().unwrap(); + let out_tlv_is_none = with_host(|h| { let rv = h .invoke_instance_method(&bt1, "println", id1, &[Box::new(StringBox::new("hello"))]) .expect("println tlv"); rv.is_none() - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("ConsoleBox"); - let out_tb_is_none = { - let h = host.read().unwrap(); + let out_tb_is_none = with_host(|h| { let rv = h .invoke_instance_method(&bt2, "println", id2, &[Box::new(StringBox::new("hello"))]) .expect("println tb"); rv.is_none() - }; + }); assert!( out_tlv_is_none && out_tb_is_none, "println should return void/None in both modes" @@ -264,62 +242,36 @@ mod tests { let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("MathBox"); - let out_tlv = { - let h = host.read().unwrap(); - let s1 = h - .invoke_instance_method(&bt1, "sqrt", id1, &[Box::new(IntegerBox::new(9))]) - .expect("sqrt tlv") - .unwrap(); - let s2 = h - .invoke_instance_method(&bt1, "sin", id1, &[Box::new(IntegerBox::new(0))]) - .expect("sin tlv") - .unwrap(); - let s3 = h - .invoke_instance_method(&bt1, "cos", id1, &[Box::new(IntegerBox::new(0))]) - .expect("cos tlv") - .unwrap(); - let s4 = h - .invoke_instance_method(&bt1, "round", id1, &[Box::new(IntegerBox::new(26))]) - .expect("round tlv") - .unwrap(); + let out_tlv = with_host(|h| { + let s1 = inv_some(h, &bt1, "sqrt", id1, &[Box::new(IntegerBox::new(9))]); + let s2 = inv_some(h, &bt1, "sin", id1, &[Box::new(IntegerBox::new(0))]); + let s3 = inv_some(h, &bt1, "cos", id1, &[Box::new(IntegerBox::new(0))]); + let s4 = inv_some(h, &bt1, "round", id1, &[Box::new(IntegerBox::new(26))]); ( s1.to_string_box().value, s2.to_string_box().value, s3.to_string_box().value, s4.to_string_box().value, ) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("MathBox"); - let out_tb = { - let h = host.read().unwrap(); - let s1 = h - .invoke_instance_method(&bt2, "sqrt", id2, &[Box::new(IntegerBox::new(9))]) - .expect("sqrt tb") - .unwrap(); - let s2 = h - .invoke_instance_method(&bt2, "sin", id2, &[Box::new(IntegerBox::new(0))]) - .expect("sin tb") - .unwrap(); - let s3 = h - .invoke_instance_method(&bt2, "cos", id2, &[Box::new(IntegerBox::new(0))]) - .expect("cos tb") - .unwrap(); - let s4 = h - .invoke_instance_method(&bt2, "round", id2, &[Box::new(IntegerBox::new(26))]) - .expect("round tb") - .unwrap(); + let out_tb = with_host(|h| { + let s1 = inv_some(h, &bt2, "sqrt", id2, &[Box::new(IntegerBox::new(9))]); + let s2 = inv_some(h, &bt2, "sin", id2, &[Box::new(IntegerBox::new(0))]); + let s3 = inv_some(h, &bt2, "cos", id2, &[Box::new(IntegerBox::new(0))]); + let s4 = inv_some(h, &bt2, "round", id2, &[Box::new(IntegerBox::new(26))]); ( s1.to_string_box().value, s2.to_string_box().value, s3.to_string_box().value, s4.to_string_box().value, ) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (MathBox)" @@ -341,46 +293,34 @@ mod tests { }; // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("EncodingBox"); - let out_tlv = { - let h = host.read().unwrap(); - let b64 = h - .invoke_instance_method( - &bt1, - "base64Encode", - id1, - &[Box::new(StringBox::new("hi"))], - ) - .expect("b64 tlv") - .unwrap(); - let hex = h - .invoke_instance_method(&bt1, "hexEncode", id1, &[Box::new(StringBox::new("hi"))]) - .expect("hex tlv") - .unwrap(); + let out_tlv = with_host(|h| { + let b64 = inv_some( + h, + &bt1, + "base64Encode", + id1, + &[Box::new(StringBox::new("hi"))], + ); + let hex = inv_some(h, &bt1, "hexEncode", id1, &[Box::new(StringBox::new("hi"))]); (b64.to_string_box().value, hex.to_string_box().value) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("EncodingBox"); - let out_tb = { - let h = host.read().unwrap(); - let b64 = h - .invoke_instance_method( - &bt2, - "base64Encode", - id2, - &[Box::new(StringBox::new("hi"))], - ) - .expect("b64 tb") - .unwrap(); - let hex = h - .invoke_instance_method(&bt2, "hexEncode", id2, &[Box::new(StringBox::new("hi"))]) - .expect("hex tb") - .unwrap(); + let out_tb = with_host(|h| { + let b64 = inv_some( + h, + &bt2, + "base64Encode", + id2, + &[Box::new(StringBox::new("hi"))], + ); + let hex = inv_some(h, &bt2, "hexEncode", id2, &[Box::new(StringBox::new("hi"))]); (b64.to_string_box().value, hex.to_string_box().value) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (EncodingBox)" @@ -394,42 +334,24 @@ mod tests { let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("RegexBox"); - let out_tlv = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt1, "compile", id1, &[Box::new(StringBox::new("h.+o"))]) - .expect("compile tlv"); - let m = h - .invoke_instance_method(&bt1, "isMatch", id1, &[Box::new(StringBox::new("hello"))]) - .expect("isMatch tlv") - .unwrap(); - let f = h - .invoke_instance_method(&bt1, "find", id1, &[Box::new(StringBox::new("hello"))]) - .expect("find tlv") - .unwrap(); + let out_tlv = with_host(|h| { + inv_void(h, &bt1, "compile", id1, &[Box::new(StringBox::new("h.+o"))]); + let m = inv_some(h, &bt1, "isMatch", id1, &[Box::new(StringBox::new("hello"))]); + let f = inv_some(h, &bt1, "find", id1, &[Box::new(StringBox::new("hello"))]); (m.to_string_box().value, f.to_string_box().value) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("RegexBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt2, "compile", id2, &[Box::new(StringBox::new("h.+o"))]) - .expect("compile tb"); - let m = h - .invoke_instance_method(&bt2, "isMatch", id2, &[Box::new(StringBox::new("hello"))]) - .expect("isMatch tb") - .unwrap(); - let f = h - .invoke_instance_method(&bt2, "find", id2, &[Box::new(StringBox::new("hello"))]) - .expect("find tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void(h, &bt2, "compile", id2, &[Box::new(StringBox::new("h.+o"))]); + let m = inv_some(h, &bt2, "isMatch", id2, &[Box::new(StringBox::new("hello"))]); + let f = inv_some(h, &bt2, "find", id2, &[Box::new(StringBox::new("hello"))]); (m.to_string_box().value, f.to_string_box().value) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (RegexBox)" @@ -443,108 +365,66 @@ mod tests { let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("PathBox"); - let out_tlv = { - let h = host.read().unwrap(); - let j = h - .invoke_instance_method( - &bt1, - "join", - id1, - &[ - Box::new(StringBox::new("/a/b")), - Box::new(StringBox::new("c.txt")), - ], - ) - .expect("join tlv") - .unwrap(); - let d = h - .invoke_instance_method( - &bt1, - "dirname", - id1, - &[Box::new(StringBox::new("/a/b/c.txt"))], - ) - .expect("dirname tlv") - .unwrap(); - let b = h - .invoke_instance_method( - &bt1, - "basename", - id1, - &[Box::new(StringBox::new("/a/b/c.txt"))], - ) - .expect("basename tlv") - .unwrap(); - let n = h - .invoke_instance_method( - &bt1, - "normalize", - id1, - &[Box::new(StringBox::new("/a/./b/../b/c"))], - ) - .expect("normalize tlv") - .unwrap(); + let out_tlv = with_host(|h| { + let j = inv_some( + h, + &bt1, + "join", + id1, + &[ + Box::new(StringBox::new("/a/b")), + Box::new(StringBox::new("c.txt")), + ], + ); + let d = inv_some(h, &bt1, "dirname", id1, &[Box::new(StringBox::new("/a/b/c.txt"))]); + let b = inv_some(h, &bt1, "basename", id1, &[Box::new(StringBox::new("/a/b/c.txt"))]); + let n = inv_some( + h, + &bt1, + "normalize", + id1, + &[Box::new(StringBox::new("/a/./b/../b/c"))], + ); ( j.to_string_box().value, d.to_string_box().value, b.to_string_box().value, n.to_string_box().value, ) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("PathBox"); - let out_tb = { - let h = host.read().unwrap(); - let j = h - .invoke_instance_method( - &bt2, - "join", - id2, - &[ - Box::new(StringBox::new("/a/b")), - Box::new(StringBox::new("c.txt")), - ], - ) - .expect("join tb") - .unwrap(); - let d = h - .invoke_instance_method( - &bt2, - "dirname", - id2, - &[Box::new(StringBox::new("/a/b/c.txt"))], - ) - .expect("dirname tb") - .unwrap(); - let b = h - .invoke_instance_method( - &bt2, - "basename", - id2, - &[Box::new(StringBox::new("/a/b/c.txt"))], - ) - .expect("basename tb") - .unwrap(); - let n = h - .invoke_instance_method( - &bt2, - "normalize", - id2, - &[Box::new(StringBox::new("/a/./b/../b/c"))], - ) - .expect("normalize tb") - .unwrap(); + let out_tb = with_host(|h| { + let j = inv_some( + h, + &bt2, + "join", + id2, + &[ + Box::new(StringBox::new("/a/b")), + Box::new(StringBox::new("c.txt")), + ], + ); + let d = inv_some(h, &bt2, "dirname", id2, &[Box::new(StringBox::new("/a/b/c.txt"))]); + let b = inv_some(h, &bt2, "basename", id2, &[Box::new(StringBox::new("/a/b/c.txt"))]); + let n = inv_some( + h, + &bt2, + "normalize", + id2, + &[Box::new(StringBox::new("/a/./b/../b/c"))], + ); ( j.to_string_box().value, d.to_string_box().value, b.to_string_box().value, n.to_string_box().value, ) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (PathBox)" @@ -559,54 +439,48 @@ mod tests { let toml_text = "[package]\nname=\"nyash\"\n[deps]\nregex=\"1\"\n"; // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("TOMLBox"); - let out_tlv = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt1, "parse", id1, &[Box::new(StringBox::new(toml_text))]) - .expect("parse tlv") - .unwrap(); - let name = h - .invoke_instance_method( - &bt1, - "get", - id1, - &[Box::new(StringBox::new("package.name"))], - ) - .expect("get tlv") - .unwrap(); - let json = h - .invoke_instance_method(&bt1, "toJson", id1, &[]) - .expect("toJson tlv") - .unwrap(); + let out_tlv = with_host(|h| { + inv_void( + h, + &bt1, + "parse", + id1, + &[Box::new(StringBox::new(toml_text))], + ); + let name = inv_some( + h, + &bt1, + "get", + id1, + &[Box::new(StringBox::new("package.name"))], + ); + let json = inv_some(h, &bt1, "toJson", id1, &[]); (name.to_string_box().value, json.to_string_box().value) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("TOMLBox"); - let out_tb = { - let h = host.read().unwrap(); - let _ = h - .invoke_instance_method(&bt2, "parse", id2, &[Box::new(StringBox::new(toml_text))]) - .expect("parse tb") - .unwrap(); - let name = h - .invoke_instance_method( - &bt2, - "get", - id2, - &[Box::new(StringBox::new("package.name"))], - ) - .expect("get tb") - .unwrap(); - let json = h - .invoke_instance_method(&bt2, "toJson", id2, &[]) - .expect("toJson tb") - .unwrap(); + let out_tb = with_host(|h| { + inv_void( + h, + &bt2, + "parse", + id2, + &[Box::new(StringBox::new(toml_text))], + ); + let name = inv_some( + h, + &bt2, + "get", + id2, + &[Box::new(StringBox::new("package.name"))], + ); + let json = inv_some(h, &bt2, "toJson", id2, &[]); (name.to_string_box().value, json.to_string_box().value) - }; + }); assert_eq!( out_tlv, out_tb, "TLV vs TypeBox results should match (TOMLBox)" @@ -620,28 +494,20 @@ mod tests { let host = crate::runtime::get_global_plugin_host(); // TLV path - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("TimeBox"); - let t_tlv = { - let h = host.read().unwrap(); - let v = h - .invoke_instance_method(&bt1, "now", id1, &[]) - .expect("now tlv") - .unwrap(); + let t_tlv = with_host(|h| { + let v = inv_some(h, &bt1, "now", id1, &[]); v.to_string_box().value.parse::().unwrap_or(0) - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("TimeBox"); - let t_tb = { - let h = host.read().unwrap(); - let v = h - .invoke_instance_method(&bt2, "now", id2, &[]) - .expect("now tb") - .unwrap(); + let t_tb = with_host(|h| { + let v = inv_some(h, &bt2, "now", id2, &[]); v.to_string_box().value.parse::().unwrap_or(0) - }; + }); let diff = (t_tb - t_tlv).abs(); assert!(diff < 5_000, "TimeBox.now difference too large: {}ms", diff); @@ -654,49 +520,31 @@ mod tests { let host = crate::runtime::get_global_plugin_host(); // TLV path: verify get->inc->get increases by 1 - env::set_var("NYASH_DISABLE_TYPEBOX", "1"); + let _g = EnvGuard::set("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("CounterBox"); - let (a1, b1) = { - let h = host.read().unwrap(); - let a = h - .invoke_instance_method(&bt1, "get", id1, &[]) - .expect("get tlv") - .unwrap(); - let _ = h - .invoke_instance_method(&bt1, "inc", id1, &[]) - .expect("inc tlv"); - let b = h - .invoke_instance_method(&bt1, "get", id1, &[]) - .expect("get2 tlv") - .unwrap(); + let (a1, b1) = with_host(|h| { + let a = inv_some(h, &bt1, "get", id1, &[]); + inv_void(h, &bt1, "inc", id1, &[]); + let b = inv_some(h, &bt1, "get", id1, &[]); ( a.to_string_box().value.parse::().unwrap_or(0), b.to_string_box().value.parse::().unwrap_or(0), ) - }; + }); assert_eq!(b1 - a1, 1, "CounterBox TLV should increment by 1"); // TypeBox path: verify same delta behavior (not comparing absolute values due to singleton) - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("CounterBox"); - let (a2, b2) = { - let h = host.read().unwrap(); - let a = h - .invoke_instance_method(&bt2, "get", id2, &[]) - .expect("get tb") - .unwrap(); - let _ = h - .invoke_instance_method(&bt2, "inc", id2, &[]) - .expect("inc tb"); - let b = h - .invoke_instance_method(&bt2, "get", id2, &[]) - .expect("get2 tb") - .unwrap(); + let (a2, b2) = with_host(|h| { + let a = inv_some(h, &bt2, "get", id2, &[]); + inv_void(h, &bt2, "inc", id2, &[]); + let b = inv_some(h, &bt2, "get", id2, &[]); ( a.to_string_box().value.parse::().unwrap_or(0), b.to_string_box().value.parse::().unwrap_or(0), ) - }; + }); assert_eq!(b2 - a2, 1, "CounterBox TypeBox should increment by 1"); } @@ -718,8 +566,7 @@ mod tests { // TLV path env::set_var("NYASH_DISABLE_TYPEBOX", "1"); let (bt1, id1, _hold1) = create_plugin_instance("FileBox"); - let out_tlv = { - let h = host.read().unwrap(); + let out_tlv = with_host(|h| { let _ = h .invoke_instance_method( &bt1, @@ -749,21 +596,17 @@ mod tests { ], ) .expect("open2 tlv"); - let rd = h - .invoke_instance_method(&bt1, "read", id1, &[]) - .expect("read tlv") - .unwrap(); + let rd = inv_some(h, &bt1, "read", id1, &[]); let _ = h .invoke_instance_method(&bt1, "close", id1, &[]) .expect("close2 tlv"); rd.to_string_box().value - }; + }); // TypeBox path - env::remove_var("NYASH_DISABLE_TYPEBOX"); + let _g2 = EnvGuard::remove("NYASH_DISABLE_TYPEBOX"); let (bt2, id2, _hold2) = create_plugin_instance("FileBox"); - let out_tb = { - let h = host.read().unwrap(); + let out_tb = with_host(|h| { let _ = h .invoke_instance_method( &bt2, @@ -792,15 +635,12 @@ mod tests { ], ) .expect("open2 tb"); - let rd = h - .invoke_instance_method(&bt2, "read", id2, &[]) - .expect("read tb") - .unwrap(); + let rd = inv_some(h, &bt2, "read", id2, &[]); let _ = h .invoke_instance_method(&bt2, "close", id2, &[]) .expect("close2 tb"); rd.to_string_box().value - }; + }); // Cleanup best-effort let _ = fs::remove_file(&path_str); @@ -813,7 +653,9 @@ mod tests { fn rand_id() -> u64 { use std::time::{SystemTime, UNIX_EPOCH}; - let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap(); + let now = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("SystemTime before UNIX_EPOCH"); now.as_micros() as u64 } } diff --git a/src/tokenizer.rs b/src/tokenizer.rs index 90f4268b..f2028f31 100644 --- a/src/tokenizer.rs +++ b/src/tokenizer.rs @@ -359,86 +359,11 @@ impl NyashTokenizer { self.advance(); return Ok(Token::new(TokenType::PipeForward, start_line, start_column)); } - Some('<') => { + // 単文字トークンはテーブル駆動で処理 + Some(c) if self.single_char_token(c).is_some() => { + let tt = self.single_char_token(c).unwrap(); self.advance(); - Ok(Token::new(TokenType::LESS, start_line, start_column)) - } - Some('>') => { - self.advance(); - Ok(Token::new(TokenType::GREATER, start_line, start_column)) - } - Some('&') => { - self.advance(); - Ok(Token::new(TokenType::BitAnd, start_line, start_column)) - } - Some('|') => { - self.advance(); - Ok(Token::new(TokenType::BitOr, start_line, start_column)) - } - Some('^') => { - self.advance(); - Ok(Token::new(TokenType::BitXor, start_line, start_column)) - } - Some('=') => { - self.advance(); - Ok(Token::new(TokenType::ASSIGN, start_line, start_column)) - } - Some('+') => { - self.advance(); - Ok(Token::new(TokenType::PLUS, start_line, start_column)) - } - Some('-') => { - self.advance(); - Ok(Token::new(TokenType::MINUS, start_line, start_column)) - } - Some('*') => { - self.advance(); - Ok(Token::new(TokenType::MULTIPLY, start_line, start_column)) - } - Some('/') => { - self.advance(); - Ok(Token::new(TokenType::DIVIDE, start_line, start_column)) - } - Some('%') => { - self.advance(); - Ok(Token::new(TokenType::MODULO, start_line, start_column)) - } - Some('.') => { - self.advance(); - Ok(Token::new(TokenType::DOT, start_line, start_column)) - } - Some('(') => { - self.advance(); - Ok(Token::new(TokenType::LPAREN, start_line, start_column)) - } - Some(')') => { - self.advance(); - Ok(Token::new(TokenType::RPAREN, start_line, start_column)) - } - Some('[') => { - self.advance(); - Ok(Token::new(TokenType::LBRACK, start_line, start_column)) - } - Some(']') => { - self.advance(); - Ok(Token::new(TokenType::RBRACK, start_line, start_column)) - } - Some('{') => { - self.advance(); - Ok(Token::new(TokenType::LBRACE, start_line, start_column)) - } - Some('}') => { - self.advance(); - Ok(Token::new(TokenType::RBRACE, start_line, start_column)) - } - Some(',') => { - self.advance(); - Ok(Token::new(TokenType::COMMA, start_line, start_column)) - } - // '?' and ':' are handled earlier (including variants); avoid duplicate arms - Some('\n') => { - self.advance(); - Ok(Token::new(TokenType::NEWLINE, start_line, start_column)) + Ok(Token::new(tt, start_line, start_column)) } Some(c) => Err(TokenizeError::UnexpectedCharacter { char: c, @@ -449,6 +374,34 @@ impl NyashTokenizer { } } + // 単文字トークンのマップ（最長一致系は呼び出し元で処理済み） + fn single_char_token(&self, c: char) -> Option { + // '?' は上位で分岐済み、':' も同様。ここでは純粋な1文字を扱う。 + match c { + '<' => Some(TokenType::LESS), + '>' => Some(TokenType::GREATER), + '&' => Some(TokenType::BitAnd), + '|' => Some(TokenType::BitOr), + '^' => Some(TokenType::BitXor), + '=' => Some(TokenType::ASSIGN), + '+' => Some(TokenType::PLUS), + '-' => Some(TokenType::MINUS), + '*' => Some(TokenType::MULTIPLY), + '/' => Some(TokenType::DIVIDE), + '%' => Some(TokenType::MODULO), + '.' => Some(TokenType::DOT), + '(' => Some(TokenType::LPAREN), + ')' => Some(TokenType::RPAREN), + '[' => Some(TokenType::LBRACK), + ']' => Some(TokenType::RBRACK), + '{' => Some(TokenType::LBRACE), + '}' => Some(TokenType::RBRACE), + ',' => Some(TokenType::COMMA), + '\n' => Some(TokenType::NEWLINE), + _ => None, + } + } + /// 文字列リテラルを読み取り fn read_string(&mut self) -> Result { let start_line = self.line; diff --git a/tools/build_llvm.sh b/tools/build_llvm.sh index 8ee80878..61210962 100644 --- a/tools/build_llvm.sh +++ b/tools/build_llvm.sh @@ -84,7 +84,18 @@ if [[ "${NYASH_LLVM_SKIP_EMIT:-0}" != "1" ]]; then fi fi fi - ./target/release/ny-llvmc --in "$NYASH_LLVM_MIR_JSON" --out "$OBJ" + if [[ "${NYASH_LLVM_EMIT:-obj}" == "exe" ]]; then + echo " emitting EXE via ny-llvmc (crate) ..." >&2 + # Ensure NyRT is built (for libnyrt.a) + if [[ ! -f crates/nyrt/target/release/libnyrt.a && "${NYASH_LLVM_SKIP_NYRT_BUILD:-0}" != "1" ]]; then + ( cd crates/nyrt && cargo build --release -j 24 >/dev/null ) + fi + NYRT_DIR_HINT="${NYASH_LLVM_NYRT:-crates/nyrt/target/release}" + ./target/release/ny-llvmc --in "$NYASH_LLVM_MIR_JSON" --out "$OUT" --emit exe --nyrt "$NYRT_DIR_HINT" ${NYASH_LLVM_LIBS:+--libs "$NYASH_LLVM_LIBS"} + echo "✅ Done: $OUT"; echo " (runtime may require nyash.toml and plugins depending on app)"; exit 0 + else + ./target/release/ny-llvmc --in "$NYASH_LLVM_MIR_JSON" --out "$OBJ" + fi ;; esac if [[ "$COMPILER_MODE" == "harness" ]]; then diff --git a/tools/crate_exe_smoke.sh b/tools/crate_exe_smoke.sh new file mode 100644 index 00000000..9d51d394 --- /dev/null +++ b/tools/crate_exe_smoke.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ "${NYASH_CLI_VERBOSE:-0}" == "1" ]]; then + set -x +fi + +ROOT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")"/.. && pwd) +cd "$ROOT_DIR" + +BIN=./target/release/nyash +NYLL=./target/release/ny-llvmc + +if [[ ! -x "$BIN" ]]; then + echo "[build] nyash ..." >&2 + cargo build --release >/dev/null +fi +if [[ ! -x "$NYLL" ]]; then + echo "[build] ny-llvmc ..." >&2 + cargo build --release -p nyash-llvm-compiler >/dev/null +fi + +APP="${1:-apps/tests/ternary_basic.nyash}" +OUT="${2:-tmp/crate_exe_smoke}" + +mkdir -p tmp +JSON=tmp/crate_exe_smoke.json + +echo "[1/3] Emitting MIR JSON via CLI ..." >&2 +"$BIN" --emit-mir-json "$JSON" --backend mir "$APP" >/dev/null + +echo "[2/3] Building EXE via ny-llvmc (crate) ..." >&2 +NYASH_LLVM_NYRT_DIR=${NYASH_LLVM_NYRT:-target/release} +if [[ ! -f "$NYASH_LLVM_NYRT_DIR/libnyrt.a" ]]; then + ( cd crates/nyrt && cargo build --release >/dev/null ) +fi +"$NYLL" --in "$JSON" --out "$OUT" --emit exe --nyrt "$NYASH_LLVM_NYRT_DIR" ${NYASH_LLVM_LIBS:+--libs "$NYASH_LLVM_LIBS"} + +echo "[3/3] Running EXE ..." >&2 +set +e +"$OUT" >/dev/null 2>&1 +CODE=$? +set -e +echo "exit=$CODE" + +echo "✅ crate_exe_smoke OK: $OUT (exit=$CODE)" +exit 0