- Feasible and attractive: treating the Nyash ABI itself as a TypeBox (C ABI) is a clean way to de‑Rust the runtime boundary while aligning with "Everything is Box".
- Key to success: keep the ABI surface minimal, versioned, allocator/context‑centric, and future‑proofed; treat reflection, invocation, and async as Boxes too.
- Migration: start as a C shim over current Rust NyRT, run conformance tests in parallel, then swap implementations behind the same ABI, and finally reimplement in Nyash.
### Technical Feasibility
- C implementation: realistic if the ABI is small, pure C (no exceptions), with explicit lifetimes, error/status returns, and allocator control. Similar precedents exist (CPython C-API, Lua C API, COM‑style vtables).
- TypeBox integration: natural fit if the ABI object is "just another Box" exposing a stable vtable plus discovery via `nyash.toml`/plugin loader.
- Self‑hosting path: clear when the compiler and runtime only depend on the C ABI Box; the compiler written in Nyash can build itself by linking against that Box.
### ABI Shape: Recommended Adjustments
- Versioning and size: include `api_version` and `struct_size` in the TypeBox header so older loaders can detect compatibility.
- Context first: most functions accept a `nyash_ctx*` holding allocator, scheduler, TLS, and feature flags; this avoids global state.
- Status + out‑params: adopt `nyash_status` enum return, with outputs via out‑params; do not return ownership in a raw return position.
- Object model: make every heap object implement retain/release via a common header/layout and expose a `nyash_obj_vtable` for method lookup.
- Method invocation: prefer selector/handle lookup over raw string calls. Pattern:
-`nyash_status lookup_selector(ctx, type_id, name, out nyash_selector*)`
-`nyash_status call(ctx, obj, selector, argv, argc, out result)`
- Allows caching and JIT fast paths; string lookup happens at most once per call‑site.
### Minimal core surface:
-`retain(obj)`, `release(obj)`
-`type_of(value) -> type_id`
-`is_null`, `eq`, `hash`
-`lookup_selector`, `call`
-`create_value(type_id, data, flags)`
-`alloc`, `free`, `mem_stats` via `nyash_allocator`
-`error_new`, `error_code`, `error_message` (errors are Boxes)
-`nyash_value`: 16‑byte payload recommended for portability and JIT friendliness:
-`u64 type_id; u64 payload; u64 meta;` or `u64[2] payload` if you want 128‑bit immediates later. Ensure 16‑byte alignment for vector ops and stable lowering in LLVM.
- Inline vs heap:
- Use low meta bits for immediate tags (small int, bool, null, small enum).
- Heap payloads are opaque pointers to refcounted objects with a common header.
-`type_id`: stable 64‑bit value with namespacing: top 16 bits vendor, 8 bits kind (struct, enum, trait/interface, array, func), remaining bits a stable hash of the fully‑qualified name + version.
- Thread safety: use atomic increments/decrements; defer destruction on the owning scheduler thread if the vtable marks "affine".
- Cycle handling: three options; pick one early and document:
- No cross‑cycle guarantee, plus "weakref" Boxes and explicit `close()` for graph owners (lowest complexity).
- Trial‑deferred cycle detection (Bacon–Rajan) at safepoints.
- Arena/region Boxes for compiler/IR lifetimes (good for self‑hosting).
- Allocation: `nyash_allocator` in `nyash_ctx` allows embedding, custom arenas, and leak accounting. Every allocation path must route through it.
### Type Safety Guarantees
- Validate `type_id` on every API entry that consumes a `nyash_value`. Return `NYASH_E_TYPE`.
- Provide `nyash_cast(ctx, value, target_type_id, out value2)` that performs runtime checks (and potentially conversion) in one place.
- For generics/parametric types, define a "type constructor" `type_id` plus encoded parameters hashed into a derived `type_id`.
### Performance Considerations
- Selector caching: make `nyash_selector` a stable handle (contains resolved vtable slot + inline cache cookie). JIT can inline calls directly via the slot.
- Avoid string dispatch in hot paths: strings resolve only once per call‑site; ABI preserves handles to avoid repeated lookups.
- Small immediates: keep i31/i61, bool, small enum as immediates; `meta` carries tag; JIT emits zero‑branch unbox.
- Error handling: status codes + separate error Box avoids exceptions; zero‑overhead fast path.
- Marshaling: keep `argv` as a flat `nyash_value*` slice; for varargs heavy sites, support `nyash_tuple` Box to amortize allocations.
- Layout stability: document alignment and endianness; avoid bitfields in the ABI; use masks and shifts.
### Async and Errors
- Async as Box: define `nyash_future` TypeBox with
-`poll(ctx, self, out ready, out result_or_err)`
-`awaitable` integration at the language level; scheduler belongs to `nyash_ctx`.
- Errors as Box: `nyash_error` carries `code`, `message`, `data` (Box). Every API returning non‑OK can also optionally fill an out error Box.
### Versioning and Compatibility
- Feature negotiation: `supports(ctx, feature_id)` and a `capabilities` bitmask in the provider header.
- Semantic version in header, strict size check: reject if `struct_size < required`.
- Reserved fields: pad both the provider and object vtables with reserved pointers for forward extension.
- Cross‑platform: define `NYASH_ABI_API` macro for visibility/calling convention; test x86_64 Linux, macOS, Windows (MSVC) early.
### Coexistence and Migration
- Phase 1 (C shim over Rust):
- Implement the C TypeBox provider whose functions forward to existing Rust NyRT via `extern "C"` glue. This validates the ABI without rewriting runtime logic.
- Place under `plugins/nyash_abi_c/` and add mapping in `nyash.toml`.
- Phase 2 (feature parity in C):
- Incrementally reimplement hot paths: small immediates, retain/release, allocator, string, array. Keep complex features (async, reflection) temporarily forwarded to Rust until replaced.
- Phase 3 (Nyash reimplementation):
- Recode the C provider in Nyash, but keep the exact same C ABI surface via an AOT target producing the same symbols.
- Coexistence:
- Loader picks provider by name and version from `nyash.toml`. Keep Rust and C providers shippable concurrently; choose via env (`NYASH_ABI_IMPL=c,rust`).
- Compatibility:
- Add a conformance suite that exercises only the ABI; run it against both providers until results match.
- Fuzz/property tests for `nyash_value` encode/decode and `type_id` canonicalization.
- ABI stability:
- Generate a C header from a single source of truth; forbid breaking changes unless version bumps.
- Integration smokes:
- Use `tools/llvm_smoke.sh` with `NYASH_C_ABI=1` to validate JIT/AOT end‑to‑end with the C provider loaded.
### Philosophical Fit
- ABI‑as‑Box completes the idea that "runtime powers" are themselves values. It turns reflection, invocation, and scheduling into first‑class participants rather than privileged side channels.
- Beauty comes from uniform dispatch: the compiler, runtime, and plugins all talk through the same Box vocabulary with selectors and capabilities.
### Alternatives and Trade‑offs
- Keep Rust dependency:
- Pro: faster to evolve; borrow‑checker catches many bugs.
- Con: self‑hosting depends on Rust toolchain; ABI surface must still be C for plugins; harder for other ecosystems to embed.
- WASM/WASI ABI:
- Pro: portable sandbox, good for plugins.
- Con: host interop and low‑latency GC/RC semantics are harder; JIT integration overhead.
- Minimal micro‑kernel runtime in C + high‑level services in Rust:
- Pro: balance; critical ABI in C, complex features in Rust behind handles.
- Con: more moving parts, boundary crossings remain.
- Provide a stub provider that returns `NYASH_E_NOT_IMPL` but passes header/version checks; wire it in `nyash.toml`.
- Add a conformance test crate in `tests/abi/` that loads the provider by name and validates the header, alloc, retain/release, and immediate encodes.
- Implement small immediates + retain/release + strings:
- Backed by a simple thread‑safe RC and arena allocator in C.
- Introduce selector caching in the VM/JIT:
- Plumb `nyash_selector` handles in the call path; deopt to string lookup on cache miss.
If you want, I can draft the minimal v0 header (with versioning/capabilities and 16‑byte value layout) and a stub `plugins/nyash_abi_c` skeleton that loads in the current repo, plus a small conformance test to exercise header negotiation.
