hakorune/docs/development/roadmap/phases/phase-21.8

Phase 21.8 — Numeric Core Integration & Builder Support

Status: proposal (to hand off to Claude Code)

Goal

Integrate the new numeric core boxes (IntArrayCore + MatI64) into the Hakorune selfhost chain so that:

• Stage‑B → MirBuilder → ny‑llvmc(crate) can emit MIR(JSON) and EXE for code that uses:
  • using nyash.core.numeric.intarray as IntArrayCore
  • using nyash.core.numeric.matrix_i64 as MatI64
• The matmul_core microbench (MatI64 + IntArrayCore) runs end‑to‑end in EXE mode and can be compared fairly against a matching C implementation.

21.6 provides the core boxes; 21.8 focuses on wiring them into the builder/runtime chain without changing default behaviour for other code.

Scope (21.8, this host)

• Stage‑B / MirBuilder:

  • Ensure MatI64 and IntArrayCore are recognized as valid boxes when referenced via:
    • using nyash.core.numeric.matrix_i64 as MatI64
    • using nyash.core.numeric.intarray as IntArrayCore
  • Fix the current provider‑emit failure:
    • Error today: [mirbuilder/parse/error] undefined variable: MatI64 during env.mirbuilder.emit.
    • Diagnose and adjust Stage‑B / MirBuilder so that static box references (MatI64.new, A.mul_naive) compile in the same way as other boxes.

• AotPrep / emit pipeline:

  • Keep AotPrep unchanged for now; the goal is to make tools/hakorune_emit_mir.sh succeed on matmul_core sources without special‑casing.
  • Ensure tools/hakorune_emit_mir.sh with:
    • HAKO_APPLY_AOT_PREP=1 NYASH_AOT_COLLECTIONS_HOT=1 NYASH_LLVM_FAST=1 NYASH_MIR_LOOP_HOIST=1
    • can emit valid MIR(JSON) for MatI64/IntArrayCore code.

• Microbench integration:

  • Finish wiring matmul_core in tools/perf/microbench.sh:
    • Hako side: MatI64/IntArrayCore based O(n³) matmul (MatI64.mul_naive).
    • C side: MatI64Core { int64_t *ptr; rows; cols; stride; } with identical algorithm.
  • Accept that performance may still be far from the 80% target; 21.8 focuses on structural integration and parity, not tuning.

Out of scope:

• New optimizations inside AotPrep / CollectionsHot.
• SIMD/blocked matmul kernels (to be handled in a later optimization phase).
• f64/complex matrix variants.

Tasks for implementation (Claude Code)

1. Fix MatI64 visibility in Stage‑B / MirBuilder

   • Reproduce the current failure:
     • Use a small .hako like:
       • using nyash.core.numeric.matrix_i64 as MatI64
       • static box Main { method main(args) { local n = 4; local A = MatI64.new(n,n); return A.at(0,0); } }
     • Confirm env.mirbuilder.emit reports undefined variable: MatI64.
   • Investigate how modules from nyash.toml ("nyash.core.numeric.matrix_i64" = "lang/src/runtime/numeric/mat_i64_box.hako") are made visible to Stage‑B and MirBuilder.
   • Adjust the resolver / module prelude so that MatI64 (and IntArrayCore) are treated like other core boxes:
     • Either via explicit prelude inclusion,
     • Or via module registry entries consumed by the builder.

2. Ensure tools/hakorune_emit_mir.sh can emit MIR(JSON) for matmul_core

   • Once MatI64 is visible, run:
     • HAKO_APPLY_AOT_PREP=1 NYASH_AOT_COLLECTIONS_HOT=1 NYASH_LLVM_FAST=1 NYASH_MIR_LOOP_HOIST=1 NYASH_JSON_ONLY=1 tools/hakorune_emit_mir.sh <matmul_core.hako> tmp/matmul_core.json
   • Acceptance:
     • No undefined variable: MatI64 / IntArrayCore errors.
     • tmp/matmul_core.json is valid MIR(JSON) (same schema as existing matmul case).

3. Finish matmul_core microbench

   • Use the existing skeleton in tools/perf/microbench.sh (case matmul_core):
     • Confirm Hako side compiles and runs under --backend vm.
     • Confirm EXE path works:
       • NYASH_SKIP_TOML_ENV=1 NYASH_LLVM_SKIP_BUILD=1 tools/perf/microbench.sh --case matmul_core --backend llvm --exe --runs 1 --n 64
   • Update benchmarks/README.md:
     • Add matmul_core row with a short description:
       • “MatI64/IntArrayCore vs MatI64Core C struct (ptr+rows+cols+stride)”
     • Record initial ratios (even if far from 80%).

4. Keep existing behaviour stable

   • No changes to default user behaviour, env toggles, or existing benches beyond adding matmul_core.
   • Ensure quick/profile smokes (where applicable) remain green with numeric core present.

Notes

• 21.6 already introduced:
  • NyRT IntArrayCore (Vec + RwLock) and handle‑based externs (nyash.intarray.*).
  • Hako wrappers IntArrayCore and MatI64 in lang/src/runtime/numeric/.
  • nyash.toml module aliases for nyash.core.numeric.intarray and nyash.core.numeric.matrix_i64.
• 21.8 is about wiring these into the builder/emit chain so that Hakorune can compile and benchmark numeric core code end‑to‑end.