macroctx: pass ctx JSON (caps) to user macros; add demo macro; docs: MacroCtx section and sandbox Box API; golden for for/foreach normalized AST

CURRENT_TASK.md

@@ -3,6 +3,14 @@
 Updated: 2025‑09‑20
 
 ## Today (Done)
+- Polishing Sprint（非破壊・仕様不変）
+  - main の薄型化（bin→lib 委譲）とテスト import 整流
+  - LLVM Python ビルダ: builders/* に一本化（fallback 除去）
+  - 重要区間の例外ログ化（`NYASH_CLI_VERBOSE=1` 連動）
+  - 生成物の既定出力を `tmp/` に統一（tools/build_llvm.sh, tools/build_aot.sh, llvm_builder.py CLI）
+  - README 冒頭に Execution Status を追記（Active/Inactive の明示）
+  - DEV_QUICKSTART に Acceptance Checklist を追記
+  - lib 内コメントのトーン整流（実装は要点のみ）
 - PeekExpr → If 連鎖の正変換を安定化
   - マクロ側（IfMatchNormalize）での検出を has_kind("PeekExpr") に統一。
   - Local/Assignment/Return/Print の4経路で PeekExpr を If に置換できるよう整備。
@@ -55,6 +63,14 @@ Updated: 2025‑09‑20
 3) 自己ホスト前展開の観測強化（ログ/スモーク）と安定運用
 4) ランタイムcapabilities（io/net/env）のPyVM側実効化は必要になった時点で最小修正
 
+## Polishing Sprint (non‑breaking, minimal)
+- [x] Thin bin entry (src/main.rs): remove duplicate `pub mod` list; use `nyash_rust::runner::NyashRunner` and friends.
+- [x] Adjust main test imports to refer to `nyash_rust::box_trait::*`.
+- [x] Add debug logs in Python LLVM builder for previously silent exceptions (gated by `NYASH_CLI_VERBOSE=1`).
+- [x] LLVM builder delegated only (builders/*); legacy fallback removed with clear debug on failure.
+- [x] Default outputs unified to `tmp/` (tools/build_llvm.sh, tools/build_aot.sh, llvm_builder.py CLI default).
+- [x] No behavior change: keep LLVM/PHI invariants and outputs semantics as-is.
+
 ## Next Milestones
 - DONE: Self‑host 前展開 既定化（auto）
   - 変更多: `NYASH_MACRO_SELFHOST_PRE_EXPAND` 未設定時に、マクロ有効かつ `NYASH_VM_USE_PY=1` で自動ON（安全策付き）。
@@ -75,6 +91,10 @@ Next (short)
 - DONE: for/foreach 正規化（コア正規化パスへ昇格）
   - 形: `for(fn(){init}, cond, fn(){step}, fn(){body})`, `foreach(arr, "x", fn(){body})`
   - 出力スモーク: tools/test/smoke/macro/for_foreach_output_smoke.sh（for: 0,1,2 / foreach: 1,2,3）
+- MacroCtx PoC（子ランナー経路のctx受け渡しを有効化）
+  - ctx JSON: `{ "caps": { "io|net|env": bool } }`
+  - 例マクロ: `apps/macros/examples/macro_ctx_demo.nyash`（identity、stdoutは使わない）
+  - Docs: guides/macro-system.md にMacroCtx節を追記
 - LoopForm MVP‑3: break/continue minimal handling (single‑level)
 - for/foreach pre‑desugaring → LoopForm normalization (limited)
 - LLVM IR hygiene for LoopForm / If / Match — PHI at block head, no empty PHIs (smoke)

README.md

@@ -14,6 +14,18 @@
 
 ---
 
+Execution Status (Phase Freeze)
+- Active
+  - `--backend llvm` (Python/llvmlite harness; AOT object emit)
+  - `--backend vm` (PyVM harness)
+- Inactive/Sealed
+  - `--backend cranelift`, `--jit-direct` (sealed; use LLVM harness)
+  - Rust VM (legacy opt‑in via features)
+
+Quick pointers
+- Emit object with harness: set `NYASH_LLVM_USE_HARNESS=1` and `NYASH_LLVM_OBJ_OUT=<path>` (defaults in tools use `tmp/`).
+- Run PyVM: `NYASH_VM_USE_PY=1 ./target/release/nyash --backend vm apps/APP/main.nyash`.
+
 Developer quickstart: see `docs/DEV_QUICKSTART.md`. Changelog highlights: `CHANGELOG.md`.
 User Macros (Phase 2): `docs/guides/user-macros.md`
 AST JSON v0 (macro/bridge): `docs/reference/ir/ast-json-v0.md`

apps/macros/examples/macro_ctx_demo.nyash

@@ -0,0 +1,14 @@
+// MacroCtx demo (PoC): shows how ctx (caps) is passed to expand(json, ctx).
+// This macro is identity; when caps.env=true it prints a trace line.
+
+static box MacroBoxSpec {
+  name() { return "MacroCtxDemo" }
+
+  // json: AST JSON v0 (string)
+  // ctx:  JSON string like {"caps":{"io":false,"net":false,"env":false}}
+  expand(json, ctx) {
+    // Identity demo: do not print to stdout (child output must remain pure JSON)
+    // You may inspect ctx (JSON string) and make decisions, but avoid side effects.
+    return json
+  }
+}

docs/DEV_QUICKSTART.md

@@ -33,3 +33,11 @@ See also
 ## Testing
 - Rust unit tests: `cargo test`
 - Targeted: e.g., tokenizer/sugar config `cargo test --lib sugar_basic_test -- --nocapture`
+
+## Acceptance Checklist (Phase Freeze)
+- cargo check (workspace) passes
+- Representative smokes are green:
+  - PyVM smokes: `tools/pyvm_stage2_smoke.sh`
+  - LLVM harness smokes: `tools/llvm_smoke.sh release`
+  - Macro goldens: `tools/ci_check_golden.sh`
+- No spec changes (compat maintained); changes are minimal and local

docs/guides/macro-system.md

@@ -92,6 +92,23 @@ Notes:
   - Splice `$...name` into call/array argument lists.
   - Array/Map nodes participate in pattern/unquote (map keys must match literally; values can bind via `$name`).
 
+## MacroCtx (PoC)
+
+User macros executed via the PyVM sandbox receive a second argument `ctx` in `expand(json, ctx)`:
+
+- Shape (string): `{ "caps": { "io": bool, "net": bool, "env": bool } }`
+- Policy: all caps default to false. The sandbox disables plugins and exposes a minimal Box API.
+- Do not print to stdout from macros: the child process stdout is reserved for the expanded JSON.
+  - For diagnostics use stderr in the future; for now prefer silent operation or trace via the parent process.
+
+Example (identity):
+```
+static box MacroBoxSpec {
+  name() { return "MacroCtxDemo" }
+  expand(json, ctx) { return json }
+}
+```
+
 ### JSON test args (advanced)
 
 For `--run-tests`, you can supply per-test arguments and instance construction details via `NYASH_TEST_ARGS_JSON`.

src/lib.rs

@@ -1,10 +1,9 @@
 /*!
- * Nyash Programming Language - Rust Implementation Library
- *
- * Everything is Box philosophy implemented in memory-safe Rust
- */
+ Nyash Programming Language — Rust library crate.
+ Provides parser, MIR, backends, runner, and supporting runtime.
+*/
 
-// 🌐 WebAssembly support
+// WebAssembly support
 #[cfg(target_arch = "wasm32")]
 use wasm_bindgen::prelude::*;
 
@@ -12,50 +11,47 @@ use wasm_bindgen::prelude::*;
 #[cfg(not(feature = "interpreter-legacy"))]
 mod interpreter_stub;
 
-pub mod ast; // Using old ast.rs for now
+pub mod ast; // using historical ast.rs
 pub mod box_arithmetic;
-pub mod box_factory; // 🏭 Unified Box Factory Architecture (Phase 9.78)
-pub mod box_operators; // 🚀 Operator implementations for basic Box types
+pub mod box_factory; // unified Box Factory
+pub mod box_operators; // operator implementations for basic Box types
 pub mod box_trait;
 pub mod boxes;
 pub mod channel_box;
-pub mod core; // Core models shared by backends
+pub mod core; // core models shared by backends
 pub mod environment;
 pub mod exception_box;
 pub mod finalization;
-pub mod instance_v2; // 🎯 Phase 9.78d: Simplified InstanceBox implementation
+pub mod instance_v2; // simplified InstanceBox implementation
 #[cfg(feature = "interpreter-legacy")]
 pub mod interpreter;
 #[cfg(not(feature = "interpreter-legacy"))]
 pub mod interpreter { pub use crate::interpreter_stub::*; }
 pub mod method_box;
-pub mod operator_traits; // 🚀 Rust-style trait-based operator overloading
-pub mod parser; // Using old parser.rs for now
-pub mod scope_tracker; // 🎯 Phase 9.78a: Box lifecycle tracking for VM
+pub mod operator_traits; // trait-based operator overloading
+pub mod parser; // using historical parser.rs
+pub mod scope_tracker; // Box lifecycle tracking for VM
 pub mod stdlib;
 pub mod tokenizer;
-pub mod type_box; // 🌟 TypeBox revolutionary system // 🚀 Arithmetic operations moved from box_trait.rs
+pub mod type_box; // TypeBox system (arithmetic moved from box_trait.rs)
 
-// 🔥 NyashValue Revolutionary System (NEW!)
 pub mod value;
 
-// 🌐 P2P Communication Infrastructure (NEW!)
 pub mod messaging;
 pub mod transport;
 
-// 🚀 MIR (Mid-level Intermediate Representation) Infrastructure (NEW!)
+// MIR (Mid-level Intermediate Representation)
 pub mod mir;
 #[cfg(feature = "aot-plan-import")]
 pub mod mir_aot_plan_import {
     pub use crate::mir::aot_plan_import::*;
 }
 
-// 🚀 Backend Infrastructure (NEW!)
+// Backends
 pub mod backend;
-pub mod jit; // Phase 10: Cranelift JIT subsystem (skeleton)
+pub mod jit; // Cranelift JIT subsystem (skeleton)
 pub mod semantics; // Unified semantics trait for MIR evaluation/lowering
 
-// 📊 Performance Benchmarks (NEW!)
 pub mod benchmarks;
 
 // BID-FFI / Plugin system (prototype)
@@ -71,9 +67,9 @@ pub mod cli;
 pub mod debug;
 pub mod runner_plugin_init;
 pub mod runtime;
-// Unified Grammar (Phase 11.9 scaffolding)
+// Unified Grammar scaffolding
 pub mod grammar;
-pub mod syntax; // Phase 12.7: syntax sugar config and helpers
+pub mod syntax; // syntax sugar config and helpers
 
 #[cfg(target_arch = "wasm32")]
 pub mod wasm_test;
@@ -105,14 +101,13 @@ pub use channel_box::{ChannelBox, MessageBox};
 pub use instance_v2::InstanceBox; // 🎯 新実装テスト（nyash_rustパス使用）
 pub use method_box::{BoxType, EphemeralInstance, FunctionDefinition, MethodBox};
 
-// 🔥 NyashValue Revolutionary System exports
 pub use value::NyashValue;
 
 // Direct canvas test export
 #[cfg(all(target_arch = "wasm32", feature = "interpreter-legacy"))]
 pub use wasm_test::wasm_test::test_direct_canvas_draw;
 
-// 🌐 WebAssembly exports for browser usage
+// WebAssembly exports for browser usage
 #[cfg(all(target_arch = "wasm32", feature = "interpreter-legacy"))]
 #[wasm_bindgen]
 pub struct NyashWasm {

src/llvm_py/llvm_builder.py

@@ -120,33 +120,13 @@ class NyashLLVMBuilder:
         for func_data in functions:
             self.lower_function(func_data)
 
-        # Create ny_main wrapper if necessary (extracted helper)
+        # Create ny_main wrapper if necessary (delegated builder; no legacy fallback)
         try:
             from builders.entry import ensure_ny_main as _ensure_ny_main
             _ensure_ny_main(self)
-        except Exception:
-            # Fallback to legacy in-place logic if helper import fails
+        except Exception as _e:
             try:
-                has_ny_main = any(f.name == 'ny_main' for f in self.module.functions)
-                fn_main_box = None
-                fn_main_plain = None
-                for f in self.module.functions:
-                    if f.name == 'Main.main/1':
-                        fn_main_box = f
-                    elif f.name == 'main':
-                        fn_main_plain = f
-                target_fn = fn_main_box or fn_main_plain
-                if target_fn is not None and not has_ny_main:
-                    ny_main_ty = ir.FunctionType(self.i64, [])
-                    ny_main = ir.Function(self.module, ny_main_ty, name='ny_main')
-                    entry = ny_main.append_basic_block('entry')
-                    b = ir.IRBuilder(entry)
-                    rv = ir.Constant(self.i64, 0)
-                    if fn_main_box is not None:
-                        rv = b.call(fn_main_box, [], name='call_Main_main_1')
-                    elif fn_main_plain is not None and len(fn_main_plain.args) == 0:
-                        rv = b.call(fn_main_plain, [], name='call_user_main')
-                    b.ret(rv)
+                trace_debug(f"[Python LLVM] ensure_ny_main failed: {_e}")
             except Exception:
                 pass
         
@@ -178,400 +158,16 @@ class NyashLLVMBuilder:
         return str(self.module)
     
     def lower_function(self, func_data: Dict[str, Any]):
-        """Lower a single MIR function to LLVM IR"""
-        # Prefer delegated helper (incremental split); fall back on failure
+        """Lower a single MIR function to LLVM IR (delegated, no legacy fallback)."""
         try:
             from builders.function_lower import lower_function as _lower
             return _lower(self, func_data)
         except Exception as _e:
             try:
-                trace_debug(f"[Python LLVM] helper lower_function failed, falling back: {_e}")
+                trace_debug(f"[Python LLVM] lower_function failed: {_e}")
             except Exception:
                 pass
-        name = func_data.get("name", "unknown")
-        self.current_function_name = name
-        import re
-        params = func_data.get("params", [])
-        blocks = func_data.get("blocks", [])
-        
-        # Determine function signature
-        if name == "ny_main":
-            # Special case: ny_main returns i32
-            func_ty = ir.FunctionType(self.i32, [])
-        else:
-            # Default: i64(i64, ...) signature; derive arity from '/N' suffix when params missing
-            m = re.search(r"/(\d+)$", name)
-            arity = int(m.group(1)) if m else len(params)
-            param_types = [self.i64] * arity
-            func_ty = ir.FunctionType(self.i64, param_types)
-        
-        # Reset per-function maps and resolver caches to avoid cross-function collisions
-        try:
-            self.vmap.clear()
-        except Exception:
-            self.vmap = {}
-        # Reset basic-block map per function (block ids are local to function)
-        try:
-            self.bb_map.clear()
-        except Exception:
-            self.bb_map = {}
-        # Reset resolver caches (they key by block name; avoid collisions across functions)
-        try:
-            self.resolver.i64_cache.clear()
-            self.resolver.ptr_cache.clear()
-            self.resolver.f64_cache.clear()
-            if hasattr(self.resolver, '_end_i64_cache'):
-                self.resolver._end_i64_cache.clear()
-            if hasattr(self.resolver, 'string_ids'):
-                self.resolver.string_ids.clear()
-            if hasattr(self.resolver, 'string_literals'):
-                self.resolver.string_literals.clear()
-            if hasattr(self.resolver, 'string_ptrs'):
-                self.resolver.string_ptrs.clear()
-        except Exception:
-            pass
-
-        # Create or reuse function
-        func = None
-        for f in self.module.functions:
-            if f.name == name:
-                func = f
-                break
-        if func is None:
-            func = ir.Function(self.module, func_ty, name=name)
-        
-        # Map parameters to vmap (value_id: 0..arity-1)
-        try:
-            arity = len(func.args)
-            for i in range(arity):
-                self.vmap[i] = func.args[i]
-        except Exception:
-            pass
-
-        # Build predecessor map from control-flow edges
-        self.preds = {}
-        for block_data in blocks:
-            bid = block_data.get("id", 0)
-            self.preds.setdefault(bid, [])
-        for block_data in blocks:
-            src = block_data.get("id", 0)
-            for inst in block_data.get("instructions", []):
-                op = inst.get("op")
-                if op == "jump":
-                    t = inst.get("target")
-                    if t is not None:
-                        self.preds.setdefault(t, []).append(src)
-                elif op == "branch":
-                    th = inst.get("then")
-                    el = inst.get("else")
-                    if th is not None:
-                        self.preds.setdefault(th, []).append(src)
-                    if el is not None:
-                        self.preds.setdefault(el, []).append(src)
-
-        # Create all blocks first
-        for block_data in blocks:
-            bid = block_data.get("id", 0)
-            block_name = f"bb{bid}"
-            bb = func.append_basic_block(block_name)
-            self.bb_map[bid] = bb
-        
-        # Build quick lookup for blocks by id
-        block_by_id: Dict[int, Dict[str, Any]] = {}
-        for block_data in blocks:
-            block_by_id[block_data.get("id", 0)] = block_data
-
-        # Determine entry block: first with no predecessors; fallback to first block
-        entry_bid = None
-        for bid, preds in self.preds.items():
-            if len(preds) == 0:
-                entry_bid = bid
-                break
-        if entry_bid is None and blocks:
-            entry_bid = blocks[0].get("id", 0)
-
-        # Compute a preds-first (approx topological) order
-        visited = set()
-        order: List[int] = []
-
-        def visit(bid: int):
-            if bid in visited:
-                return
-            visited.add(bid)
-            for p in self.preds.get(bid, []):
-                visit(p)
-            order.append(bid)
-
-        if entry_bid is not None:
-            visit(entry_bid)
-        # Include any blocks not reachable from entry
-        for bid in block_by_id.keys():
-            if bid not in visited:
-                visit(bid)
-
-        # Process blocks in the computed order
-        # Prepass: collect producer stringish hints and PHI metadata for all blocks
-        # and create placeholders at each block head so that resolver can safely
-        # return existing PHIs without creating new ones.
-        _setup_phi_placeholders(self, blocks)
-
-        # Optional: if-merge prepass → predeclare PHI for return-merge blocks
-        # Gate with NYASH_LLVM_PREPASS_IFMERGE=1
-        try:
-            if os.environ.get('NYASH_LLVM_PREPASS_IFMERGE') == '1':
-                plan = plan_ret_phi_predeclare(block_by_id)
-                if plan:
-                    # Ensure block_phi_incomings map exists
-                    if not hasattr(self, 'block_phi_incomings') or self.block_phi_incomings is None:
-                        self.block_phi_incomings = {}
-                    for bbid, ret_vid in plan.items():
-                        # Do not pre-materialize PHI here; record only metadata.
-                        # Record declared incoming metadata using the same value-id
-                        # for each predecessor; finalize_phis will resolve per-pred end values.
-                        try:
-                            preds_raw = [p for p in self.preds.get(bbid, []) if p != bbid]
-                        except Exception:
-                            preds_raw = []
-                        # Dedup while preserving order
-                        seen = set()
-                        preds_list = []
-                        for p in preds_raw:
-                            if p not in seen:
-                                preds_list.append(p)
-                                seen.add(p)
-                        try:
-                            # finalize_phis reads pairs as (decl_b, v_src) and maps to nearest predecessor.
-                            # We provide (bb_pred, ret_vid) for all preds.
-                            self.block_phi_incomings.setdefault(int(bbid), {})[int(ret_vid)] = [
-                                (int(p), int(ret_vid)) for p in preds_list
-                            ]
-                        except Exception:
-                            pass
-                        try:
-                            trace_debug(f"[prepass] if-merge: plan metadata at bb{bbid} for v{ret_vid} preds={preds_list}")
-                        except Exception:
-                            pass
-        except Exception:
-            pass
-
-        # Predeclare PHIs for values used in a block but defined in predecessors (multi-pred only).
-        # This keeps PHI nodes grouped at the top and avoids late synthesis during operand resolution.
-        try:
-            from cfg.utils import build_preds_succs
-            local_preds, _ = build_preds_succs(block_by_id)
-            def _collect_defs(block):
-                defs = set()
-                for ins in block.get('instructions') or []:
-                    try:
-                        dstv = ins.get('dst')
-                        if isinstance(dstv, int):
-                            defs.add(int(dstv))
-                    except Exception:
-                        pass
-                return defs
-            def _collect_uses(block):
-                uses = set()
-                for ins in block.get('instructions') or []:
-                    # Minimal keys: lhs/rhs (binop), value (ret/copy), cond (branch), box_val (boxcall)
-                    for k in ('lhs','rhs','value','cond','box_val'):
-                        try:
-                            v = ins.get(k)
-                            if isinstance(v, int):
-                                uses.add(int(v))
-                        except Exception:
-                            pass
-                return uses
-            # Ensure map for declared incomings exists
-            if not hasattr(self, 'block_phi_incomings') or self.block_phi_incomings is None:
-                self.block_phi_incomings = {}
-            for bid, blk in block_by_id.items():
-                # Only multi-pred blocks need PHIs
-                try:
-                    preds_raw = [p for p in local_preds.get(int(bid), []) if p != int(bid)]
-                except Exception:
-                    preds_raw = []
-                # Dedup preds preserve order
-                seen = set(); preds_list = []
-                for p in preds_raw:
-                    if p not in seen: preds_list.append(p); seen.add(p)
-                if len(preds_list) <= 1:
-                    continue
-                defs = _collect_defs(blk)
-                uses = _collect_uses(blk)
-                need = [u for u in uses if u not in defs]
-                if not need:
-                    continue
-                bb0 = self.bb_map.get(int(bid))
-                if bb0 is None:
-                    continue
-                b0 = ir.IRBuilder(bb0)
-                try:
-                    b0.position_at_start(bb0)
-                except Exception:
-                    pass
-                for vid in need:
-                    # Do not create placeholder here; let finalize_phis materialize
-                    # to keep PHIs strictly grouped at block heads and avoid dups.
-                    # Record incoming metadata for finalize_phis (pred -> same vid)
-                    try:
-                        self.block_phi_incomings.setdefault(int(bid), {}).setdefault(int(vid), [])
-                        # Overwrite with dedup list of (pred, vid)
-                        self.block_phi_incomings[int(bid)][int(vid)] = [(int(p), int(vid)) for p in preds_list]
-                    except Exception:
-                        pass
-            # Expose to resolver
-            try:
-                self.resolver.block_phi_incomings = self.block_phi_incomings
-            except Exception:
-                pass
-        except Exception:
-            pass
-
-        # Optional: simple loop prepass → synthesize a structured while body
-        loop_plan = None
-        try:
-            if os.environ.get('NYASH_LLVM_PREPASS_LOOP') == '1':
-                loop_plan = detect_simple_while(block_by_id)
-                if loop_plan is not None:
-                    trace_debug(f"[prepass] detect loop header=bb{loop_plan['header']} then=bb{loop_plan['then']} latch=bb{loop_plan['latch']} exit=bb{loop_plan['exit']}")
-        except Exception:
-            loop_plan = None
-
-        # No predeclared PHIs are materialized; resolver may ignore ret_phi_map
-
-        # Now lower blocks
-        skipped: set[int] = set()
-        if loop_plan is not None:
-            try:
-                for bskip in loop_plan.get('skip_blocks', []):
-                    if bskip != loop_plan.get('header'):
-                        skipped.add(int(bskip))
-            except Exception:
-                pass
-        for bid in order:
-            block_data = block_by_id.get(bid)
-            if block_data is None:
-                continue
-            # If loop prepass applies, lower while once at header and skip loop-internal blocks
-            if loop_plan is not None and bid == loop_plan.get('header'):
-                bb = self.bb_map[bid]
-                builder = ir.IRBuilder(bb)
-                try:
-                    self.resolver.builder = builder
-                    self.resolver.module = self.module
-                except Exception:
-                    pass
-                # Lower while via loopform (if enabled) or regular fallback
-                self.loop_count += 1
-                body_insts = loop_plan.get('body_insts', [])
-                cond_vid = loop_plan.get('cond')
-                from instructions.loopform import lower_while_loopform
-                ok = False
-                try:
-                    # Use a clean per-while vmap context seeded from global placeholders
-                    self._current_vmap = dict(self.vmap)
-                    ok = lower_while_loopform(
-                        builder,
-                        func,
-                        cond_vid,
-                        body_insts,
-                        self.loop_count,
-                        self.vmap,
-                        self.bb_map,
-                        self.resolver,
-                        self.preds,
-                        self.block_end_values,
-                        getattr(self, 'ctx', None),
-                    )
-                except Exception:
-                    ok = False
-                if not ok:
-                    # Prepare resolver backref for instruction dispatcher
-                    try:
-                        self.resolver._owner_lower_instruction = self.lower_instruction
-                    except Exception:
-                        pass
-                    lower_while_regular(builder, func, cond_vid, body_insts,
-                                        self.loop_count, self.vmap, self.bb_map,
-                                        self.resolver, self.preds, self.block_end_values)
-                # Clear while vmap context
-                try:
-                    delattr(self, '_current_vmap')
-                except Exception:
-                    pass
-                # Mark blocks to skip
-                for bskip in loop_plan.get('skip_blocks', []):
-                    skipped.add(bskip)
-                # Ensure skipped original blocks have a valid terminator: branch to while exit
-                try:
-                    exit_name = f"while{self.loop_count}_exit"
-                    exit_bb = None
-                    for bbf in func.blocks:
-                        try:
-                            if str(bbf.name) == exit_name:
-                                exit_bb = bbf
-                                break
-                        except Exception:
-                            pass
-                    if exit_bb is not None:
-                        # Connect while exit to original exit block if available
-                        try:
-                            orig_exit_bb = self.bb_map.get(loop_plan.get('exit'))
-                            if orig_exit_bb is not None and exit_bb.terminator is None:
-                                ibx = ir.IRBuilder(exit_bb)
-                                ibx.branch(orig_exit_bb)
-                        except Exception:
-                            pass
-                        for bskip in loop_plan.get('skip_blocks', []):
-                            if bskip == loop_plan.get('header'):
-                                continue
-                            bb_skip = self.bb_map.get(bskip)
-                            if bb_skip is None:
-                                continue
-                            try:
-                                if bb_skip.terminator is None:
-                                    ib = ir.IRBuilder(bb_skip)
-                                    ib.branch(exit_bb)
-                            except Exception:
-                                pass
-                except Exception:
-                    pass
-                continue
-            if bid in skipped:
-                continue
-            bb = self.bb_map[bid]
-            self.lower_block(bb, block_data, func)
-
-        # Provide lifetime hints to resolver (which blocks define which values)
-        try:
-            self.resolver.def_blocks = self.def_blocks
-            # Provide phi metadata for this function to resolver
-            self.resolver.block_phi_incomings = getattr(self, 'block_phi_incomings', {})
-            # Attach a BuildCtx object for future refactors (non-breaking)
-            try:
-                self.ctx = BuildCtx(
-                    module=self.module,
-                    i64=self.i64,
-                    i32=self.i32,
-                    i8=self.i8,
-                    i1=self.i1,
-                    i8p=self.i8p,
-                    vmap=self.vmap,
-                    bb_map=self.bb_map,
-                    preds=self.preds,
-                    block_end_values=self.block_end_values,
-                    resolver=self.resolver,
-                    trace_phi=os.environ.get('NYASH_LLVM_TRACE_PHI') == '1',
-                    verbose=os.environ.get('NYASH_CLI_VERBOSE') == '1',
-                )
-                # Also expose via resolver for convenience until migration completes
-                self.resolver.ctx = self.ctx
-            except Exception:
-                pass
-        except Exception:
-            pass
-        # Finalize PHIs for this function now that all snapshots for it exist
-        _finalize_phis(self)
+            raise
 
 
     def setup_phi_placeholders(self, blocks: List[Dict[str, Any]]):
@@ -1061,7 +657,7 @@ def main():
     # CLI:
     #   llvm_builder.py <input.mir.json> [-o output.o]
     #   llvm_builder.py --dummy [-o output.o]
-    output_file = "nyash_llvm_py.o"
+    output_file = os.path.join('tmp', 'nyash_llvm_py.o')
     args = sys.argv[1:]
     dummy = False
 
@@ -1085,6 +681,10 @@ def main():
         # Emit dummy ny_main
         ir_text = builder._create_dummy_main()
         trace_debug(f"[Python LLVM] Generated dummy IR:\n{ir_text}")
+        try:
+            os.makedirs(os.path.dirname(output_file), exist_ok=True)
+        except Exception:
+            pass
         builder.compile_to_object(output_file)
         print(f"Compiled to {output_file}")
         return
@@ -1100,6 +700,10 @@ def main():
     llvm_ir = builder.build_from_mir(mir_json)
     trace_debug("[Python LLVM] Generated LLVM IR (see NYASH_LLVM_DUMP_IR or tmp/nyash_harness.ll)")
 
+    try:
+        os.makedirs(os.path.dirname(output_file), exist_ok=True)
+    except Exception:
+        pass
     builder.compile_to_object(output_file)
     print(f"Compiled to {output_file}")
 

src/main.rs

@@ -1,73 +1,11 @@
 /*!
- * Nyash Rust Implementation - Everything is Box in Memory Safe Rust
- *
- * This is the main entry point for the Rust implementation of Nyash,
- * demonstrating the "Everything is Box" philosophy with Rust's ownership system.
- *
- * The main function serves as a thin entry point that delegates to the CLI
- * and runner modules for actual processing.
- */
-
-// Core modules
-pub mod ast;
-pub mod box_arithmetic; // 🚀 Moved from box_trait.rs for better organization
-pub mod box_factory; // 🏭 Unified Box Factory Architecture (Phase 9.78)
-pub mod box_trait;
-pub mod boxes;
-pub mod channel_box;
-pub mod core; // core::model (shared models)
-pub mod environment;
-pub mod exception_box;
-pub mod finalization;
-pub mod instance_v2; // 🎯 Phase 9.78d: Simplified InstanceBox implementation
-// Legacy interpreter module is not included by default; when disabled, re-export lib's stub
-#[cfg(feature = "interpreter-legacy")]
-pub mod interpreter;
-#[cfg(not(feature = "interpreter-legacy"))]
-pub mod interpreter { pub use nyash_rust::interpreter::*; }
-pub mod messaging; // 🌐 P2P Communication Infrastructure
-pub mod method_box;
-pub mod operator_traits;
-pub mod parser;
-pub mod scope_tracker; // VM scope lifecycle
-pub mod stdlib;
-pub mod tokenizer;
-pub mod transport;
-pub mod type_box; // 🌟 TypeBox revolutionary system
-pub mod value; // 🔥 NyashValue Revolutionary System // 🌐 P2P Communication Infrastructure
-
-// 🚀 MIR Infrastructure
-pub mod mir;
-
-// 🚀 Backend Infrastructure
-pub mod backend;
-// JIT subsystem (Phase 10)
-pub mod jit;
-pub mod semantics; // mirror library semantics module for crate path consistency in bin
-
-// 📊 Performance Benchmarks (lib provides; bin does not re-declare)
-
-// 🚀 Refactored modules for better organization
-pub mod cli;
-pub mod runner;
-pub mod r#macro;
-
-// BID-FFI / Plugin System (prototype)
-pub mod bid;
-
-// Configuration system
-pub mod config;
-
-// Runtime system (plugins, registry, etc.)
-pub mod debug;
-pub mod grammar; // Phase 11.9 unified grammar scaffolding
-pub mod runner_plugin_init;
-pub mod runtime;
-pub mod syntax; // Phase 12.7: syntax sugar config and helpers (mirror lib layout)
+ Minimal CLI entry point for Nyash.
+ Delegates to the library crate (`nyash_rust`) for all functionality.
+*/
 
 use nyash_rust::cli::CliConfig;
 use nyash_rust::config::env as env_config;
-use runner::NyashRunner;
+use nyash_rust::runner::NyashRunner;
 
 /// Thin entry point - delegates to CLI parsing and runner execution
 fn main() {
@@ -84,12 +22,13 @@ fn main() {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use box_trait::{BoxCore, NyashBox, StringBox};
+    use nyash_rust::box_trait::{BoxCore, NyashBox, StringBox};
 
     #[test]
     fn test_main_functionality() {
-        // This test ensures the module structure is correct
+        // Smoke: library module path wiring works
         let string_box = StringBox::new("test".to_string());
         assert_eq!(string_box.to_string_box().value, "test");
+        let _ = (); // CLI wiring exists via nyash_rust::cli
     }
 }

tools/build_aot.sh

@@ -5,7 +5,7 @@ ROOT_DIR=$(cd "$(dirname "$0")/.." && pwd)
 cd "$ROOT_DIR"
 
 APP=${1:-apps/tests/mir-branch-ret/main.nyash}
-OUT=${2:-app_aot}
+OUT=${2:-tmp/app_aot}
 OBJ_DIR=${OBJ_DIR:-target/aot_objects}
 OBJ_BASENAME=$(basename "$APP" .nyash)
 OBJ_PATH="$OBJ_DIR/$OBJ_BASENAME.o"
@@ -28,6 +28,8 @@ if [[ ! -f "$OBJ_PATH" ]]; then
 fi
 ls -l "$OBJ_PATH"
 
+# Ensure output directory exists
+mkdir -p "$(dirname "$OUT")"
 echo "[4/5] link with nyrt -> $OUT"
 cc "$OBJ_PATH" \
   -L crates/nyrt/target/release \

tools/build_llvm.sh

@@ -13,7 +13,7 @@ Compiles a Nyash program with the LLVM backend to an object (.o),
 links it with the NyRT static runtime, and produces a native executable.
 
 Options:
-  -o <output>   Output executable name (default: app)
+  -o <output>   Output executable path (default: tmp/app)
 
 Requirements:
   - LLVM 18 development (llvm-config-18)
@@ -24,7 +24,7 @@ USAGE
 if [[ $# -lt 1 ]]; then usage; exit 1; fi
 
 INPUT=""
-OUT="app"
+OUT="tmp/app"
 while [[ $# -gt 0 ]]; do
   case "$1" in
     -h|--help) usage; exit 0 ;;
@@ -129,6 +129,8 @@ else
   ( cd crates/nyrt && cargo build --release -j 24 >/dev/null )
 fi
 
+# Ensure output directory exists
+mkdir -p "$(dirname "$OUT")"
 echo "[4/4] Linking $OUT ..."
 cc "$OBJ" \
   -L target/release \

tools/test/golden/macro/for_basic.expanded.json

@@ -0,0 +1,8 @@
+{"kind":"Program","statements":[
+  {"kind":"Local","variables":["i"],"inits":[{"kind":"Literal","value":{"type":"int","value":0}}]},
+  {"kind":"Loop","condition":{"kind":"BinaryOp","op":"<","left":{"kind":"Variable","name":"i"},"right":{"kind":"Literal","value":{"type":"int","value":3}}},"body":[
+    {"kind":"Print","expression":{"kind":"Variable","name":"i"}},
+    {"kind":"Assignment","target":{"kind":"Variable","name":"i"},"value":{"kind":"BinaryOp","op":"+","left":{"kind":"Variable","name":"i"},"right":{"kind":"Literal","value":{"type":"int","value":1}}}}
+  ]}
+]}
+

tools/test/golden/macro/for_basic_user_macro_golden.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+root=$(cd "$(dirname "$0")"/../../../.. && pwd)
+bin="$root/target/release/nyash"
+src="apps/tests/macro/loopform/for_basic.nyash"
+golden="$root/tools/test/golden/macro/for_basic.expanded.json"
+
+if [ ! -x "$bin" ]; then
+  echo "nyash binary not found at $bin; build first (cargo build --release)" >&2
+  exit 1
+fi
+
+normalize_json() { python3 -c 'import sys,json; print(json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(",", ":")))'; }
+
+out_raw=$("$bin" --dump-expanded-ast-json "$src")
+out_norm=$(printf '%s' "$out_raw" | normalize_json)
+gold_norm=$(normalize_json < "$golden")
+
+if [ "$out_norm" != "$gold_norm" ]; then
+  echo "[FAIL] for_basic expanded JSON mismatch" >&2
+  diff -u <(echo "$out_norm") <(echo "$gold_norm") || true
+  exit 2
+fi
+
+echo "[OK] golden for_basic expansion matched"

tools/test/golden/macro/foreach_basic.expanded.json

@@ -0,0 +1,9 @@
+{"kind":"Program","statements":[
+  {"kind":"Local","variables":["arr"],"inits":[{"kind":"Array","elements":[{"kind":"Literal","value":{"type":"int","value":1}},{"kind":"Literal","value":{"type":"int","value":2}},{"kind":"Literal","value":{"type":"int","value":3}}]}]},
+  {"kind":"Local","variables":["__ny_i"],"inits":[{"kind":"Literal","value":{"type":"int","value":0}}]},
+  {"kind":"Loop","condition":{"kind":"BinaryOp","op":"<","left":{"kind":"Variable","name":"__ny_i"},"right":{"kind":"MethodCall","object":{"kind":"Variable","name":"arr"},"method":"size","arguments":[]}},"body":[
+    {"kind":"Print","expression":{"kind":"MethodCall","object":{"kind":"Variable","name":"arr"},"method":"get","arguments":[{"kind":"Variable","name":"__ny_i"}]}},
+    {"kind":"Assignment","target":{"kind":"Variable","name":"__ny_i"},"value":{"kind":"BinaryOp","op":"+","left":{"kind":"Variable","name":"__ny_i"},"right":{"kind":"Literal","value":{"type":"int","value":1}}}}
+  ]}
+]}
+

tools/test/golden/macro/foreach_basic_user_macro_golden.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+root=$(cd "$(dirname "$0")"/../../../.. && pwd)
+bin="$root/target/release/nyash"
+src="apps/tests/macro/loopform/foreach_basic.nyash"
+golden="$root/tools/test/golden/macro/foreach_basic.expanded.json"
+
+if [ ! -x "$bin" ]; then
+  echo "nyash binary not found at $bin; build first (cargo build --release)" >&2
+  exit 1
+fi
+
+normalize_json() { python3 -c 'import sys,json; print(json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(",", ":")))'; }
+
+out_raw=$("$bin" --dump-expanded-ast-json "$src")
+out_norm=$(printf '%s' "$out_raw" | normalize_json)
+gold_norm=$(normalize_json < "$golden")
+
+if [ "$out_norm" != "$gold_norm" ]; then
+  echo "[FAIL] foreach_basic expanded JSON mismatch" >&2
+  diff -u <(echo "$out_norm") <(echo "$gold_norm") || true
+  exit 2
+fi
+
+echo "[OK] golden foreach_basic expansion matched"