docs+runner+parser: SSOT+AST using finalized (legacy text inlining removed); provider verify reads nyash.toml; preflight warn hook; method-body guard removed; CURRENT_TASK updated for next JSON work
This commit is contained in:
@ -1,21 +0,0 @@
|
||||
# Nyash Architecture (Phase 15)
|
||||
|
||||
## Scope and Priorities
|
||||
- Primary execution path: LLVM AOT only. VM, Cranelift JIT/AOT, and the interpreter are not MIR14‑ready and are considered experimental in this phase.
|
||||
- Minimize fallback logic. Prefer simple, predictable lowering over clever heuristics that diverge across backends.
|
||||
|
||||
## Value Model
|
||||
- Box = handle (i64) as the canonical runtime representation.
|
||||
- Strings: LLVM AOT favors i8* for fast path operations and bridging with NyRT. Conversions between i8* and handle exist but are kept to the minimum required surfaces.
|
||||
|
||||
## Division of Responsibilities
|
||||
- NyRT (core, built‑in): fundamental boxes and operations essential for bootstrapping/self‑hosting.
|
||||
- IntegerBox, StringBox, ArrayBox, MapBox, BoolBox
|
||||
- Implemented as NyRT intrinsics (by‑id shims exist for plugin ABI compatibility).
|
||||
- Plugins: external or platform‑dependent functionality (File/Net/Regex/HTTP/DB/GUI etc.).
|
||||
- ExternCall: minimal window to the outside world (console print/log/error, debug trace, exit/now/readline); other APIs should route through BoxCall.
|
||||
|
||||
## Backend Policy (Phase 15)
|
||||
- LLVM is the source of truth. All new rules and ABIs are documented for LLVM. Other backends will adopt them after LLVM stabilizes.
|
||||
- Fallback logic must be narrow and documented. If behavior depends on type annotations, the (missing) annotations should be fixed at the MIR stage rather than widening fallback.
|
||||
|
||||
@ -1,37 +0,0 @@
|
||||
# LLVM Layer Overview (Phase 15)
|
||||
|
||||
Scope
|
||||
- Practical guide to LLVM lowering architecture and invariants used in Phase 15.
|
||||
- Complements LOWERING_LLVM.md (rules), RESOLVER_API.md (value resolution), and LLVM_HARNESS.md (harness).
|
||||
|
||||
Module Layout
|
||||
- `src/backend/llvm/compiler/codegen/`
|
||||
- `instructions/` split by concern: `flow.rs`, `blocks.rs`, `arith.rs`, `arith_ops.rs`, `mem.rs`,
|
||||
`strings.rs`, `arrays.rs`, `maps.rs`, `boxcall/`, `externcall/`, `call.rs`, `loopform.rs`, `resolver.rs`.
|
||||
- `builder_cursor.rs`: central insertion/terminator guard.
|
||||
|
||||
Core Invariants
|
||||
- Phase‑15 終盤(MIR13運用): MIR 生成層では PHI を生成しない。PHI 合成は LLVM 層(llvmlite/Resolver)が担う。
|
||||
- Resolver-only reads: lowerers fetch MIR values through `Resolver`(クロスBBの vmap 直接参照は禁止)。
|
||||
- Localize at block start: PHIs are created at the beginning of the current BB(non‑PHI より手前)で優位性を保証。
|
||||
- Cast placement: ptr↔int/幅変換は PHI の外側(BB先頭 or pred終端直前)に配置。
|
||||
- Sealed SSA: 後続ブロックの PHI は pred スナップショットと `seal_block` で配線し、branch/jump 自体は incoming を直接積まない。
|
||||
- Cursor discipline: 生成は `BuilderCursor` 経由のみ。terminator 後の挿入は禁止。
|
||||
|
||||
LoopForm(次フェーズ予定/MIR18)
|
||||
- Phase‑15 では LoopForm を MIR に導入しない。既存 CFG(preheader→header→{body|exit}; body→header)から llvmlite がループ搬送 PHI を合成。
|
||||
- 次フェーズで LoopForm(`LoopHeader/Enter/Latch` などの占位)を MIR に追加し、Resolver/PHI 合成は維持する。
|
||||
|
||||
Types and Bridges
|
||||
- Box handle is `i64` across NyRT boundary; strings prefer `i8*` fast paths.
|
||||
- Convert rules: `ensure_i64/ensure_i1/ensure_ptr` style helpers (planned extraction) to centralize casting.
|
||||
|
||||
Harness (optional)
|
||||
- llvmlite harness exists for fast prototyping and structural checks.
|
||||
- Gate: `NYASH_LLVM_USE_HARNESS=1` (planned wiring); target parity tested by Acceptance A5.
|
||||
|
||||
References
|
||||
- LOWERING_LLVM.md — lowering rules and runtime calls
|
||||
- RESOLVER_API.md — Resolver design and usage
|
||||
- LLVM_HARNESS.md — llvmlite harness interface and usage
|
||||
|
||||
@ -1,98 +0,0 @@
|
||||
# Lowering Contexts (Boxing the API)
|
||||
|
||||
Motivation
|
||||
- Current lowering functions carry too many parameters (10+), scattering responsibilities and making invariants (Resolver-only, localization discipline, dispatch-only PHI) hard to enforce.
|
||||
- Following Nyash’s “Everything is Box” philosophy, we group related data into small, composable context boxes to create clean boundaries and make invariants structural.
|
||||
|
||||
Core Context Boxes
|
||||
- LowerFnCtx<'ctx, 'b>
|
||||
- Holds: `codegen`, `func`, `cursor`, `resolver`, `vmap`, `bb_map`, `preds`, `block_end_values`, `phis_by_block`, optional `const_strs`, `box_type_ids`.
|
||||
- Utilities: `ensure_i64/ptr/f64` (Resolver-only), `with_block(..)`, `with_pred_end(..)`, `guard_post_terminator`.
|
||||
- Effect: All lowering entrypoints can accept a single `&mut LowerFnCtx`.
|
||||
|
||||
- BlockCtx<'ctx>
|
||||
- Holds: `cur_bid`, `cur_llbb` (from `bb_map`), optionally `succs`.
|
||||
- Role: The canonical site for “where to insert” decisions; pred-end casts are routed via this box.
|
||||
|
||||
- InvokeCtx
|
||||
- Holds: `method_id: u16`, `type_id: i64`, `recv_h: IntValue<'ctx>`, `args: &[ValueId]`.
|
||||
- Role: Unifies plugin invoke by-id/by-name into a single API surface.
|
||||
|
||||
- StringOps (lightweight types/helpers)
|
||||
- Types: `StrHandle(IntValue<'ctx>)`, `StrPtr(PointerValue<'ctx>)`.
|
||||
- Rule: across blocks keep `StrHandle`; convert to `StrPtr` only at call sites in the same BB; return values of string ops are kept as `StrHandle`.
|
||||
- Entry: `StringOps::concat(ctx, blk, lhs, rhs) -> LlResult<StrHandle>` etc.
|
||||
|
||||
API Sketch
|
||||
```
|
||||
pub type LlResult<T> = Result<T, String>;
|
||||
|
||||
pub struct LowerFnCtx<'ctx, 'b> { /* see above */ }
|
||||
pub struct BlockCtx<'ctx> { pub cur_bid: BasicBlockId, pub cur_llbb: BasicBlock<'ctx> }
|
||||
pub struct InvokeCtx<'ctx> { pub method_id: u16, pub type_id: i64, pub recv_h: IntValue<'ctx>, pub args: Vec<ValueId> }
|
||||
|
||||
impl<'ctx, 'b> LowerFnCtx<'ctx, 'b> {
|
||||
pub fn ensure_i64(&mut self, blk: &BlockCtx<'ctx>, v: ValueId) -> LlResult<IntValue<'ctx>> { /* Resolver-only */ }
|
||||
pub fn ensure_ptr(&mut self, blk: &BlockCtx<'ctx>, v: ValueId) -> LlResult<PointerValue<'ctx>> { /* i64 PHI -> inttoptr here */ }
|
||||
pub fn with_pred_end<R>(&mut self, pred: BasicBlockId, f: impl FnOnce(&Builder) -> R) -> R { /* insert before terminator */ }
|
||||
}
|
||||
|
||||
pub fn lower_boxcall(ctx: &mut LowerFnCtx, blk: &BlockCtx, inst: &BoxCallInst) -> LlResult<()> { /* … */ }
|
||||
pub fn try_handle_tagged_invoke(ctx: &mut LowerFnCtx, blk: &BlockCtx, call: &InvokeCtx) -> LlResult<()> { /* … */ }
|
||||
|
||||
pub mod string_ops {
|
||||
pub struct StrHandle<'ctx>(pub IntValue<'ctx>);
|
||||
pub struct StrPtr<'ctx>(pub PointerValue<'ctx>);
|
||||
pub fn concat(ctx: &mut LowerFnCtx, blk: &BlockCtx, lhs: ValueId, rhs: ValueId) -> LlResult<StrHandle> { /* … */ }
|
||||
}
|
||||
```
|
||||
|
||||
Invariants Enforced by Design
|
||||
- Resolver-only: `VMap` is not exposed; all value access goes through `LowerFnCtx` utilities.
|
||||
- Localization discipline: PHIs at BB head; casts at pred-end via `with_pred_end`.
|
||||
- Strings handle rule: Only `StrHandle` crosses block boundaries; `StrPtr` generated only in the same BB at call sites.
|
||||
- LoopForm rule: preheader mandatory, header condition built via Resolver; dispatch-only PHI. Dev guard checks enforce these.
|
||||
|
||||
Dev Guards (optional, recommended)
|
||||
- `PhiGuard::assert_dispatch_only(&LowerFnCtx)` to fail fast when non-dispatch PHIs appear.
|
||||
- `LoopGuard::assert_preheader(&LowerFnCtx)` to ensure preheader presence and header i1 formation point.
|
||||
- CI Deny-Direct: `rg -n "vmap\.get\(" src/backend/llvm/compiler/codegen/instructions | wc -l` must be `0`.
|
||||
|
||||
Migration Plan
|
||||
1) Introduce `LowerFnCtx`/`BlockCtx`/`InvokeCtx`; migrate `lower_boxcall` and invoke path first.
|
||||
2) Move string ops to `StringOps` with handle-only rule; clean call sites.
|
||||
3) Migrate BinOp/Compare/ExternCall to `LowerFnCtx + BlockCtx` API.
|
||||
4) Turn on dev guards; remove remaining fallback paths; simplify code.
|
||||
|
||||
Acceptance
|
||||
- Refactored entrypoints accept at most three boxed parameters.
|
||||
- Deny-Direct passes (no direct `vmap.get` in lowering/instructions).
|
||||
- Dominance: verifier green on representative functions (e.g., dep_tree_min_string).
|
||||
|
||||
## Context Stack Guide (If/Loop)
|
||||
|
||||
Purpose
|
||||
- Make control-flow merge/loop boundaries explicit and uniform across builders (MIR) and the JSON v0 bridge.
|
||||
|
||||
Stacks in MirBuilder
|
||||
- If merge: `if_merge_stack: Vec<BasicBlockId>`
|
||||
- Push the merge target before lowering branches, pop after wiring edge copies or Phi at the merge.
|
||||
- PHI-off: emit per-predecessor edge copies into the merge pred blocks; merge block itself must not add a self-copy.
|
||||
- PHI-on: place Phi(s) at the merge block head; inputs must cover all predecessors.
|
||||
- Loop context: `loop_header_stack` / `loop_exit_stack`
|
||||
- Header = re-check condition; Exit = after-loop block.
|
||||
- `continue` → jump to Header; `break` → jump to Exit. Both add predecessor metadata from the current block.
|
||||
- Builder captures variable-map snapshots on `continue` to contribute latch-like inputs when sealing the header.
|
||||
|
||||
JSON v0 Bridge parity
|
||||
- Bridge `LoopContext { cond_bb, exit_bb }` mirrors MIR loop stacks.
|
||||
- `continue` lowers to `Jump { target: cond_bb }`; `break` lowers to `Jump { target: exit_bb }`.
|
||||
|
||||
Verification hints
|
||||
- Use-before-def: delay copies that would reference later-defined values or route via Phi at block head.
|
||||
- Pred consistency: for every `Jump`/`Branch`, record the predecessor on the successor block.
|
||||
- PHI-off invariant: all merged values reach the merge via predecessor copies; the merge block contains no extra Copy to the same dst.
|
||||
|
||||
Snapshot rules (Loop/If)
|
||||
- Loop: take the latch snapshot at the actual latch block (end of body, after nested if merges). Use it as the backedge source when sealing header.
|
||||
- If: capture `pre_if_snapshot` before entering then/else; restore at merge and only bind merged variables (diff-based). Avoid self-copy at merge.
|
||||
@ -1,42 +0,0 @@
|
||||
# LLVM Lowering Rules (Phase 15)
|
||||
|
||||
This document describes the active LLVM lowering rules used in Phase 15. Only the LLVM path is authoritative at this time.
|
||||
|
||||
## General
|
||||
- Box values are represented as i64 handles when crossing the NyRT boundary.
|
||||
- String operations prefer i8* fast paths (AOT helpers) when possible. Handle conversions are done only at explicit boundaries.
|
||||
|
||||
## NewBox
|
||||
- StringBox:
|
||||
- When constructed from a constant string, lowering produces i8* via `nyash_string_new` and keeps it as i8* (no immediate handle conversion).
|
||||
- Builder skips redundant birth calls for StringBox.
|
||||
- Other boxes:
|
||||
- Minimal birth shims exist (e.g., `nyash.box.birth_h`, `nyash.box.birth_i64`) using Box type ids.
|
||||
|
||||
## BoxCall: String.concat fast path
|
||||
- If the receiver is annotated as String (or StringBox), lower to AOT helpers directly:
|
||||
- `concat_ss(i8*, i8*) -> i8*`
|
||||
- `concat_si(i8*, i64) -> i8*` (right operand is a handle coerced to string by NyRT)
|
||||
- `concat_is(i64, i8*) -> i8*`
|
||||
- For non‑String receivers or plugin cases, fall back to plugin/by‑id paths as needed.
|
||||
|
||||
## BinOp Add: String concatenation
|
||||
- Primary path: AOT helpers selected by operand shapes at IR time:
|
||||
- `i8* + i8* -> concat_ss`
|
||||
- `i8* + i64 -> concat_si`
|
||||
- `i64 + i8* -> concat_is`
|
||||
- Fallback policy: keep to the minimum. Do not add implicit conversions beyond the above without clear MIR type annotations. If mixed forms miscompile, fix MIR annotations first.
|
||||
|
||||
## ExternCall selection (console/debug)
|
||||
- `env.console.{log,warn,error}` and `env.debug.trace` inspect the argument at lowering time:
|
||||
- If argument is `i8*`, call the C‑string variant: `nyash.console.{log,warn,error}` / `nyash.debug.trace`.
|
||||
- Otherwise convert to `i64` and call the handle variant: `nyash.console.{log,warn,error}_handle` / `nyash.debug.trace_handle`.
|
||||
- The result values are ignored or zeroed as appropriate (side‑effecting I/O).
|
||||
|
||||
## Return/Result mapping
|
||||
- For plugin/by‑id calls that return an i64 handle but the destination is annotated as pointer‑like (String/Box/Array/Future/Unknown), the handle is cast to an opaque pointer for SSA flow. Integers/Bools remain integers.
|
||||
|
||||
## Backend Consistency Notes
|
||||
- VM/Cranelift/JIT are not MIR14‑ready and may not follow these rules yet. LLVM behavior takes precedence; other backends will be aligned later.
|
||||
- Any new fallback must be justified and scoped; wide catch‑alls are prohibited to prevent backend divergence.
|
||||
|
||||
@ -1,13 +1,11 @@
|
||||
# Nyash Design Notes
|
||||
# Design Notes Moved
|
||||
|
||||
Public, stable design documents and architecture explanations.
|
||||
`docs/design/` の設計ノートは `docs/development/design/legacy/` に移動しました。
|
||||
|
||||
Use for rationale, trade‑offs, and diagrams that are safe to cite.
|
||||
- 新しい場所: [../development/design/legacy/](../development/design/legacy/)
|
||||
- よく参照されるページ:
|
||||
- [flow-blocks.md](../development/design/legacy/flow-blocks.md)
|
||||
- [using-loader-integration.md](../development/design/legacy/using-loader-integration.md)
|
||||
|
||||
Contents to consolidate here:
|
||||
- Architecture overviews derived from ARCHITECTURE.md
|
||||
- Backend design (LLVM/Cranelift) summaries
|
||||
- MIR/IR evolution notes that are not drafts
|
||||
|
||||
Draft, exploratory, or long‑form papers should remain under `docs/private/` until finalized.
|
||||
このディレクトリは互換のための案内のみ残しています。
|
||||
|
||||
|
||||
@ -1,40 +0,0 @@
|
||||
# Resolver API (Minimal i64 Prototype)
|
||||
|
||||
Goals
|
||||
- Phase‑15(MIR13運用)における方針: MIR 生成層は PHI を出さず、LLVM 層で PHI を合成する。
|
||||
- Centralize "ValueId → current-block value" resolution.
|
||||
- Guarantee dominance by localizing values at the start of the block (before non-PHI).
|
||||
- De-duplicate per (block, value) to avoid redundant PHIs/casts.
|
||||
|
||||
Design
|
||||
- Resolver-only reads: lowerers must fetch cross-block values via `Resolver` (ban direct `vmap.get` for reads across BBs).
|
||||
- `Resolver` keeps small per-function caches keyed by `(BasicBlockId, ValueId)`.
|
||||
- `resolve_i64(...)` returns an `i64`-typed value, inserting a PHI at the beginning of the current block and wiring
|
||||
incoming from predecessor end-snapshots (sealed SSA). Any required casts are inserted in predecessor blocks just
|
||||
before their terminators to preserve dominance.
|
||||
- `resolve_ptr(...)` returns an `i8*` value localized to the current block; integer handles are bridged via `inttoptr`.
|
||||
- `resolve_f64(...)` returns an `f64` value localized to the current block; ints bridged via `sitofp`.
|
||||
- Internally uses sealed snapshots (`block_end_values`) to avoid referencing values that do not dominate the use.
|
||||
|
||||
Usage (planned wiring)
|
||||
- Create `let mut resolver = instructions::Resolver::new();` at function lowering start.
|
||||
- Replace all integer value fetches in lowerers with `resolver.resolve_i64(...)`.
|
||||
- Keep builder insertion discipline via `BuilderCursor`.
|
||||
|
||||
Ban: Direct `vmap.get(..)` for cross-BB reads
|
||||
- Direct reads from `vmap` are allowed only for values defined in the same block (after they are created).
|
||||
- For any value that may come from a predecessor, always go through `Resolver`.
|
||||
- CI guard: keep `rg "vmap\.get\(" src/backend/llvm` at zero for instruction paths (Resolver-only).
|
||||
|
||||
Next
|
||||
- Migrate remaining `localize_to_i64` call sites to the resolver.
|
||||
- Enforce vmap direct access ban in lowerers (Resolver-only for reads).
|
||||
- ループ(while 形 CFG)の検出と、ヘッダ BB での搬送 PHI 合成(preheader/backedge の 2 incoming)を実装。
|
||||
|
||||
Tracing
|
||||
- `NYASH_LLVM_TRACE_PHI=1`: log PHI creation/wiring in the Rust/inkwell path.
|
||||
- `NYASH_LLVM_TRACE_FINAL=1`: in the Python/llvmlite harness, trace selected final calls (e.g., `Main.node_json/3`,
|
||||
`Main.esc_json/1`) to correlate ON/OFF outputs during parity checks.
|
||||
|
||||
Acceptance tie-in
|
||||
- Combined with LoopForm: dispatch-only PHI + resolver-based value access → dominance violations drop to zero (A2.5).
|
||||
@ -1,37 +0,0 @@
|
||||
# AOT-Plan v1 Schema (Phase 15.1)
|
||||
|
||||
Status: draft-frozen for Phase 15.1 (extensions via `extensions` only)
|
||||
|
||||
- version: string, must be "1"
|
||||
- name: optional plan/module name
|
||||
- functions: array of PlanFunction
|
||||
- externs: optional array (reserved; not required in 15.1)
|
||||
- exports: optional array (reserved)
|
||||
- units: optional array (reserved)
|
||||
- extensions: optional object for forward-compatible keys
|
||||
|
||||
PlanFunction
|
||||
- name: string
|
||||
- params: array of { name: string, type?: string } (informational in 15.1)
|
||||
- return_type: optional string; one of: integer, float, bool, string, void (or omitted → Unknown)
|
||||
- body: optional tagged object
|
||||
- kind = "const_return": { value: any-json (int/bool/float/string) }
|
||||
- kind = "empty": returns default 0 with Unknown type (15.1 importer behavior)
|
||||
|
||||
Notes
|
||||
- 15.1 importer does not emit object code; it constructs MIR13 skeletons only.
|
||||
- If `return_type` is omitted, importer uses Unknown to keep VM dynamic display.
|
||||
- `extensions` is a free-form map; the importer ignores unknown keys.
|
||||
|
||||
Example
|
||||
```
|
||||
{
|
||||
"version": "1",
|
||||
"name": "mini_project",
|
||||
"functions": [
|
||||
{ "name": "main", "return_type": "integer", "body": { "kind": "const_return", "value": 42 }},
|
||||
{ "name": "greet", "return_type": "string", "body": { "kind": "const_return", "value": "hi" }}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
@ -1,57 +0,0 @@
|
||||
Cranelift AOT Box: 設計ノートと obj 出力 PoC(Phase 15 準備)
|
||||
|
||||
目的
|
||||
- Nyash → MIR → Cranelift AOT(C ABI)→ オブジェクト(.o/.obj)→ リンク → EXE の最小パイプラインを確立する前準備。
|
||||
- 本ブランチでは「設計と仕様の確定(ドキュメント化)」のみを行い、実装は別ブランチ `phase-15/self-host-aot-cranelift` で着手する。
|
||||
|
||||
対象範囲(P0)
|
||||
- PoC として `ny_main`(i64 → i64 返し)を定義する最小オブジェクトを Cranelift で生成できること。
|
||||
- 生成物を NyRT(`crates/nyrt`)と静的リンクして実行可能ファイルを作成できること。
|
||||
- 実行結果として `Result: 42` 等の既知の値確認を行うこと。
|
||||
|
||||
アーキテクチャ概要
|
||||
- CraneliftAotBox(本ドキュメントの主題)
|
||||
- 役割: MIR から Cranelift IR(CLIF)を生成し、`cranelift-object` でオブジェクトを出力する。
|
||||
- 出力: ターゲット環境に応じた COFF/ELF/Mach-O(`cranelift-object` の既定に従う)。
|
||||
- シグネチャ: PoC は `ny_main: () -> i64`(将来的には引数の受け渡しや NyRT 呼び出しを拡張)。
|
||||
- LinkerBox(別タスク、別文書で仕様化)
|
||||
- 役割: 生成された `.o/.obj` を NyRT(`libnyrt.a`/`nyrt.lib`)とリンクして EXE を作る。
|
||||
- Windows は `link.exe`/`lld-link`、Linux は `cc` 経由を想定(詳細は LinkerBox 仕様にて)。
|
||||
|
||||
ABI / 連携
|
||||
- エントリ: `ny_main` を EXE から呼び出す形。NyRT 側が `main()` 内から `ny_main()` を適切に呼び出して結果を表示(または検証)する想定。
|
||||
- ランタイム: PoC 段階では NyRT の最低限(起動/終了)に依存。将来的に checkpoint や GC バリアなどの外部関数を `extern "C"` で参照可能にする。
|
||||
|
||||
PoC 受入基準(P0)
|
||||
- `.o/.obj` に `ny_main` シンボルが定義されている。
|
||||
- `libnyrt.a`/`nyrt.lib` とリンクして実行可能ファイルが作成できる。
|
||||
- 実行すると標準出力に既知の値(例: `Result: 42`)が出力される。
|
||||
|
||||
想定コマンド(リンク例)
|
||||
- Linux: `cc -o app ny_main.o target/release/libnyrt.a -ldl -lpthread`
|
||||
- Windows (MSVC): `link ny_main.obj nyrt.lib /OUT:app.exe`
|
||||
- 実行時設定: 実行ファイルと同じディレクトリに `nyash.toml` を配置することでプラグイン解決を容易にする(NyRT は exe 直下→CWD の順で探索)。
|
||||
|
||||
CLI/ツール統合(案)
|
||||
- バックエンドキー: `--backend cranelift-aot`
|
||||
- PoC フラグ: `--poc-const N`(`ny_main` が `N` を返す単機能)
|
||||
- 補助スクリプト(設計のみ、本ブランチでは作成しない):
|
||||
- `tools/aot_smoke_cranelift.sh apps/APP/main.nyash -o app`
|
||||
- 流れ: Nyash → MIR → CraneliftAotBox → `.o` → LinkerBox/cc → `app`
|
||||
|
||||
ロードマップ
|
||||
- P0: PoC スタブ `ny_main` 定数返し、リンク/実行確認。
|
||||
- P1: 最小 MIR(`const_i64`/`add_i64`/`ret`)のマッピング。
|
||||
- P2: NyRT チェックポイント呼び出しなど最小の外部関数連携。
|
||||
- P3: Plugin 経由の I/O など実用的な呼び出しの一部導入。
|
||||
|
||||
既知のリスクと対策
|
||||
- プラットフォーム ABI 差異: 既定の呼出規約を使用し、まず Linux で動作確認。
|
||||
- オブジェクト形式差: `cranelift-object` の既定に寄り添う。必要に応じてターゲット指定を導入。
|
||||
- 重複実装の懸念: 既存のオブジェクトビルダ(JIT/emit系)の再利用・抽象化を検討。
|
||||
|
||||
実装方針(別ブランチで実施)
|
||||
- フィーチャ: `cranelift-aot = ["dep:cranelift-object"]`
|
||||
- モジュール: `src/backend/cranelift/aot_box.rs` を追加し、PoC 用 `compile_stub_ny_main_i64` を提供。
|
||||
- CLI 統合: `--backend cranelift-aot` と PoC フラグの導入(PoC 期間は一時的)。
|
||||
|
||||
@ -1,191 +0,0 @@
|
||||
# LLVM Backend Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
The LLVM backend provides native code compilation for Nyash programs through the inkwell LLVM wrapper. This document covers the implementation completed in Phase 9.78 Week 1.
|
||||
|
||||
## Current Implementation Status
|
||||
|
||||
### ✅ Completed (Phase 9.78 Week 1)
|
||||
|
||||
- **Infrastructure Setup**: Complete LLVM backend directory structure
|
||||
- **Mock Implementation**: Fully functional mock that demonstrates all integration points
|
||||
- **CLI Integration**: `--backend llvm` option support
|
||||
- **MIR Integration**: Connection between MIR compiler and LLVM backend
|
||||
- **Error Handling**: Proper error messages and user feedback
|
||||
|
||||
### 🔄 In Progress/Planned
|
||||
|
||||
- **Real LLVM Integration**: Requires LLVM development libraries
|
||||
- **Advanced Code Generation**: Full MIR instruction set support
|
||||
- **Optimization Passes**: LLVM optimization pipeline integration
|
||||
|
||||
## Architecture
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
src/backend/llvm/
|
||||
├── mod.rs # Main module interface and exports
|
||||
├── context.rs # LLVM context, module, and target management
|
||||
└── compiler.rs # MIR to LLVM IR compilation logic
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
1. **LLVMCompiler**: Main compilation orchestrator
|
||||
2. **CodegenContext**: LLVM context and target machine management
|
||||
3. **Integration Layer**: Connects with existing MIR → Backend pipeline
|
||||
|
||||
## Usage
|
||||
|
||||
### Mock Implementation (Current)
|
||||
|
||||
```bash
|
||||
# Run with mock LLVM backend
|
||||
cargo run -- --backend llvm test_program.nyash
|
||||
|
||||
# This will:
|
||||
# 1. Parse Nyash source to AST
|
||||
# 2. Compile AST to MIR
|
||||
# 3. Analyze MIR structure (mock)
|
||||
# 4. Display mock compilation results
|
||||
# 5. Return appropriate exit code
|
||||
```
|
||||
|
||||
### Real Implementation (Future)
|
||||
|
||||
```bash
|
||||
# Install LLVM development libraries first
|
||||
sudo apt install llvm-17-dev clang-17
|
||||
|
||||
# Enable LLVM feature and build
|
||||
cargo build --features llvm --release
|
||||
|
||||
# Run with real LLVM backend
|
||||
cargo run --features llvm -- --backend llvm test_program.nyash
|
||||
|
||||
# This will:
|
||||
# 1. Parse Nyash source to AST
|
||||
# 2. Compile AST to MIR
|
||||
# 3. Generate LLVM IR from MIR
|
||||
# 4. Compile to object file
|
||||
# 5. Link with system linker
|
||||
# 6. Execute native binary
|
||||
```
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Basic Return Test
|
||||
|
||||
**File**: `local_tests/test_return_42.nyash`
|
||||
```nyash
|
||||
static box Main {
|
||||
main() {
|
||||
return 42
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Expected Behavior**:
|
||||
- Mock: Analyzes MIR and returns appropriate exit code
|
||||
- Real: Compiles to native code that returns exit code 42
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Test mock implementation
|
||||
cargo run -- --backend llvm local_tests/test_return_42.nyash
|
||||
echo "Exit code: $?"
|
||||
|
||||
# Should show mock execution and exit code 0 (42 when real implementation is complete)
|
||||
```
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Mock vs Real Implementation
|
||||
|
||||
The current implementation uses conditional compilation to provide both mock and real implementations:
|
||||
|
||||
```rust
|
||||
#[cfg(feature = "llvm")]
|
||||
// Real inkwell-based implementation
|
||||
use inkwell::context::Context;
|
||||
|
||||
#[cfg(not(feature = "llvm"))]
|
||||
// Mock implementation for demonstration
|
||||
pub struct CodegenContext { /* ... */ }
|
||||
```
|
||||
|
||||
### MIR Integration
|
||||
|
||||
The LLVM backend integrates with the existing MIR compilation pipeline:
|
||||
|
||||
1. **AST → MIR**: Uses existing `MirCompiler`
|
||||
2. **MIR → LLVM**: New `LLVMCompiler` handles MIR instruction translation
|
||||
3. **LLVM → Native**: Uses LLVM's code generation and system linker
|
||||
|
||||
### Error Handling
|
||||
|
||||
The implementation provides comprehensive error handling:
|
||||
|
||||
- **Environment Errors**: Missing LLVM libraries, compilation failures
|
||||
- **User Feedback**: Clear messages about requirements and next steps
|
||||
- **Graceful Degradation**: Mock implementation when LLVM unavailable
|
||||
|
||||
## Dependencies
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **LLVM 17+**: Development libraries and headers
|
||||
- **Clang**: System C compiler for linking
|
||||
- **Target Support**: Native target architecture support
|
||||
|
||||
### Rust Dependencies
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
inkwell = { version = "0.5", features = ["target-x86"], optional = true }
|
||||
|
||||
[features]
|
||||
llvm = ["dep:inkwell"]
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Week 2+ Roadmap
|
||||
|
||||
1. **Real LLVM Integration**: Replace mock with inkwell implementation
|
||||
2. **Instruction Support**: Complete MIR instruction set coverage
|
||||
3. **Type System**: Proper LLVM type mapping for Nyash Box types
|
||||
4. **Optimization**: LLVM optimization pass integration
|
||||
5. **Debug Info**: Debug symbol generation for debugging support
|
||||
|
||||
### Performance Goals
|
||||
|
||||
- **Compilation Speed**: Sub-second compilation for small programs
|
||||
- **Runtime Performance**: 2x+ improvement over VM backend
|
||||
- **Binary Size**: Reasonable binary sizes with optional optimization
|
||||
|
||||
## Contributing
|
||||
|
||||
### Development Setup
|
||||
|
||||
1. Install LLVM development libraries
|
||||
2. Enable LLVM feature flag
|
||||
3. Run tests with `cargo test --features llvm`
|
||||
4. Implement missing MIR instruction handlers
|
||||
|
||||
### Code Style
|
||||
|
||||
- Follow existing Nyash code patterns
|
||||
- Use conditional compilation for feature gates
|
||||
- Provide comprehensive error messages
|
||||
- Include tests for new functionality
|
||||
|
||||
## References
|
||||
|
||||
- [inkwell Documentation](https://thedan64.github.io/inkwell/)
|
||||
- [LLVM Language Reference](https://llvm.org/docs/LangRef.html)
|
||||
- [Nyash MIR Specification](../mir/README.md)
|
||||
- [Phase 9.78 Implementation Plan](../../予定/native-plan/llvm/issue/001-setup-inkwell-hello-world.md)
|
||||
@ -1,57 +0,0 @@
|
||||
CraneliftAotBox インタフェース草案(Phase 15 準備)
|
||||
|
||||
前提
|
||||
- 本ブランチでは「仕様化(ドキュメント)」のみ行い、実装は別ブランチ `phase-15/self-host-aot-cranelift` で行う。
|
||||
- Cargo フィーチャ: `cranelift-aot = ["dep:cranelift-object"]` を追加し、同フィーチャ時のみ AOT モジュールを有効化する。
|
||||
|
||||
モジュール構成(案)
|
||||
- パス: `src/backend/cranelift/aot_box.rs`
|
||||
- 依存: `cranelift-object`(オブジェクト出力)、既存のオブジェクトビルダ/ヘルパを再利用可能なら抽象化して流用。
|
||||
|
||||
公開型(案)
|
||||
- `pub struct CraneliftAotConfig {`
|
||||
- `pub opt_level: u8` // 0..3 程度(実装は後続)
|
||||
- `pub target: Option<String>` // target triple 等(未指定でホスト)
|
||||
`}`
|
||||
|
||||
- `pub struct CraneliftAotBox {`
|
||||
- `obj: <object builder>`
|
||||
- `cfg: CraneliftAotConfig`
|
||||
`}`
|
||||
|
||||
- `#[derive(Debug)] pub enum CraneliftAotError {`
|
||||
- `Codegen(String)`, `IO(String)`
|
||||
`}`
|
||||
|
||||
主要メソッド(案)
|
||||
- `impl CraneliftAotBox {`
|
||||
- `pub fn new(cfg: CraneliftAotConfig) -> Result<Self, CraneliftAotError>`
|
||||
- `pub fn compile_stub_ny_main_i64(&mut self, val: i64, out_obj: impl AsRef<Path>) -> Result<(), CraneliftAotError>`
|
||||
- 役割: PoC。`ny_main` 関数を定義し、即値 `val` を返すオブジェクトを生成。
|
||||
- `pub fn compile_mir_to_obj(&mut self, mir: MirModule, out_obj: impl AsRef<Path>) -> Result<(), CraneliftAotError>`
|
||||
- 役割: P1〜。最小 MIR(`const_i64`/`add_i64`/`ret`)から CLIF を組み立てて出力。
|
||||
`}`
|
||||
|
||||
使用例(PoC フロー)
|
||||
1) NyRT ビルド: `cargo build -p nyrt --release`
|
||||
2) オブジェクト出力(CLIイメージ):
|
||||
- `nyash --backend cranelift-aot --poc-const 42 apps/hello/main.nyash -o ny_main.o`
|
||||
3) リンク:
|
||||
- Linux: `cc -o app ny_main.o target/release/libnyrt.a -ldl -lpthread`
|
||||
- Windows: `link ny_main.obj nyrt.lib /OUT:app.exe`
|
||||
4) 実行: `./app` → `Result: 42` を確認。
|
||||
|
||||
エラーモデル(案)
|
||||
- 環境・設定: フィーチャ未有効や未対応ターゲット → 分かりやすいメッセージ。
|
||||
- 生成・出力: `CraneliftAotError::Codegen(_)`/`CraneliftAotError::IO(_)` で大別。
|
||||
|
||||
補助スクリプトの仕様(設計のみ)
|
||||
- ファイル: `tools/aot_smoke_cranelift.sh`
|
||||
- 目的: `.o/.obj` を生成→リンク→実行して PoC を自動検証。
|
||||
- 主要引数: `apps/APP/main.nyash -o app`、必要に応じ `--const` を透過的に渡す。
|
||||
|
||||
今後の拡張(非ブロッキング)
|
||||
- NyRT の外部関数呼び出し(checkpoint など)の導入。
|
||||
- MIR 命令カバレッジの拡大、BoxCall/Plugin 経由の I/O。
|
||||
- ターゲットトリプルとオブジェクト形式の明示的制御。
|
||||
|
||||
@ -1,80 +1,5 @@
|
||||
# Flow Blocks and Arrow Piping (Design Draft)
|
||||
# Moved: Flow Blocks (Design Draft)
|
||||
|
||||
Status: design-only during the feature‑pause (no implementation)
|
||||
このドキュメントは再編により移動しました。
|
||||
新しい場所: [../development/design/legacy/flow-blocks.md](../development/design/legacy/flow-blocks.md)
|
||||
|
||||
Goal
|
||||
- Make control/data flow visually obvious while keeping the core minimal.
|
||||
- Core = anonymous `{ ... }` blocks + `->` chaining with `_` or `|args|` as the input placeholder.
|
||||
- Always desugar to plain sequential let/if/call; zero new runtime constructs.
|
||||
|
||||
Core Syntax
|
||||
- Serial (value flow):
|
||||
```nyash
|
||||
{ readConfig() }
|
||||
-> { |cfg| validate(cfg) }
|
||||
-> { |cfg| normalize(cfg) }
|
||||
-> { |cfg| save(cfg) }
|
||||
```
|
||||
- Placeholder short form:
|
||||
```nyash
|
||||
{ fetch() } -> { process(_) } -> { output(_) }
|
||||
```
|
||||
- If/Else with horizontal flow:
|
||||
```nyash
|
||||
if cond -> { doA() } else -> { doB() }
|
||||
```
|
||||
|
||||
Semantics
|
||||
- `{ ... }` is an anonymous scope usable as expression or statement.
|
||||
- `->` passes the left result as the first parameter of the right block.
|
||||
- Left returns `Void` → right cannot use `_`/`|x|` (compile-time error in MVP spec).
|
||||
- `_` and `|x,...|` are exclusive; mixing is an error.
|
||||
|
||||
Lowering (always zero-cost sugar)
|
||||
- Chain desugars to temporaries and calls:
|
||||
```nyash
|
||||
# {A} -> { |x| B(x) } -> { |y| C(y) }
|
||||
t0 = A();
|
||||
t1 = B(t0);
|
||||
t2 = C(t1);
|
||||
```
|
||||
- If/Else chain desugars to standard if/else blocks; merges follow normal PHI wiring rules.
|
||||
|
||||
Match normalization via guard chains
|
||||
- Prefer a single readable form:
|
||||
```nyash
|
||||
guard cond1 -> { A }
|
||||
guard cond2 -> { B }
|
||||
else -> { C }
|
||||
```
|
||||
- Lowers to first-match if/else chain. No new pattern engine is introduced.
|
||||
|
||||
Range and CharClass guards (design)
|
||||
- Range: `guard ch in '0'..'9' -> { ... }` → `('0' <= ch && ch <= '9')`.
|
||||
- CharClass: `guard ch in Digit -> { ... }` → expands to ranges (e.g., '0'..'9').
|
||||
- Multiple ranges combine with OR.
|
||||
|
||||
Formatting (nyfmt guidance)
|
||||
- Align arrows vertically; one step per line:
|
||||
```
|
||||
{ fetch() }
|
||||
-> { validate(_) }
|
||||
-> { save(_) }
|
||||
```
|
||||
- Suggest factoring when chains exceed N steps; prefer naming a scope helper.
|
||||
|
||||
Observability (design only)
|
||||
- `NYASH_FLOW_TRACE=1` prints the desugared steps (`t0=...; t1=...;`).
|
||||
|
||||
Constraints (MVP)
|
||||
- No new closures; anonymous blocks inline when capture-free.
|
||||
- Recursion not required; focus on linear/branching chains.
|
||||
- ASI: treat `->` as a low-precedence line-continue operator.
|
||||
|
||||
Tests (syntax-only smokes; design)
|
||||
- flow_linear: `read→validate→save` matches expected value.
|
||||
- flow_placeholder: `{f()} -> { process(_) } -> { out(_) }`.
|
||||
- flow_if: `if cond -> {A} else -> {B}` behaves like standard if.
|
||||
|
||||
Pause note
|
||||
- Documentation and design intent only. Implementation is deferred until after the feature‑pause (post‑bootstrap).
|
||||
|
||||
@ -1,48 +0,0 @@
|
||||
LinkerBox 仕様(Phase 15 準備 / Windows優先)
|
||||
|
||||
目的
|
||||
- AOT 生成されたオブジェクト(`.o/.obj`)を NyRT とリンクして実行可能ファイルを得る統一レイヤ。
|
||||
- 初期実装は「外部リンカー呼び出し」(MSVC `link.exe` または `lld-link`、Unix は `cc`)で動作し、将来的に lld 内蔵連携へ置換可能な設計とする。
|
||||
|
||||
前提/エントリポイント
|
||||
- 既定のエントリポイントは NyRT 側の `main()`(`crates/nyrt` に実装)。
|
||||
- AOT 側は `ny_main`(想定: `() -> i64`)を定義し、NyRT の `main()` が `ny_main()` を呼び出す。
|
||||
- よって通常は `/ENTRY:` 指定は不要(NyRT をリンクしない特殊構成でのみ上書き可能とする)。
|
||||
|
||||
入力と出力
|
||||
- 入力: 一つ以上のオブジェクト(`.obj`/`.o`)、追加ライブラリ群、ライブラリ検索パス。
|
||||
- 既定ライブラリ: NyRT(Windows: `nyrt.lib`、Unix: `libnyrt.a`)。
|
||||
- 出力: 実行ファイル(Windows: `*.exe`、Unix: 実行ビット付きバイナリ)。
|
||||
|
||||
環境変数(LinkerBox が解釈)
|
||||
- `NYASH_LINKER`: 使用リンカーを強制。`link` | `lld-link` | `cc`(未指定は OS/環境から自動推定)
|
||||
- `NYASH_LINK_FLAGS`: 追加フラグ(空白区切り)。
|
||||
- `NYASH_LINK_VERBOSE=1`: 実コマンドラインを表示。
|
||||
- `NYASH_LINK_ENTRY=<symbol>`: エントリポイントを明示的に指定(既定は未指定で NyRT の `main` を使用)。
|
||||
- `NYASH_LINK_OUT=<path>`: 出力先を明示的に指定(CLI引数 `-o` が優先)。
|
||||
|
||||
Windows(MSVC / lld-link)
|
||||
- 既定探索順: `link.exe` → `lld-link`。
|
||||
- 代表フラグ:
|
||||
- `link.exe`: `/OUT:<exe>` `/SUBSYSTEM:CONSOLE`(既定) `/LIBPATH:<dir>` `nyrt.lib` 他
|
||||
- `lld-link`: `-OUT:<exe>` `-SUBSYSTEM:CONSOLE` `-LIBPATH:<dir>` `nyrt.lib`
|
||||
- `PATH`/`LIB`/`LIBPATH` の整合に注意(Developer Command Prompt を推奨)。
|
||||
|
||||
Unix(参考)
|
||||
- 代表フラグ: `cc -o <exe> <objs...> <lib paths> -L... -lnyrt -ldl -lpthread`
|
||||
- 既定のオブジェクト形式/ターゲットはホストに従う。
|
||||
|
||||
CLI API(想定)
|
||||
- `nyash --linker link|lld-link|cc --libpath <dir> --lib nyrt [--entry nyash_main] -o app <objs...>`
|
||||
- AOT パスでは内部的に LinkerBox を呼び出し、上記環境変数も透過的に反映する。
|
||||
|
||||
エラー方針
|
||||
- ツールチェーン未検出(リンカー不在): わかりやすい対処案を表示(MSVC のセットアップ/lld の導入)。
|
||||
- 未解決シンボル: `ny_main`/NyRT 関連の欠落を優先表示。
|
||||
- 引数/パスのクォート: 空白を含むパスは安全にクォートして実行。
|
||||
|
||||
将来拡張
|
||||
- 内蔵 lld を採用した一体化(外部プロセス呼び出しからの置換)。
|
||||
- ターゲットトリプルの明示指定とクロスリンク(フェーズ後半)。
|
||||
- 追加ランタイムやプラグイン静的リンクのオプション化。
|
||||
|
||||
@ -1,32 +1,5 @@
|
||||
# Using → Loader Integration (Minimal)
|
||||
# Moved: Using → Loader Integration (Minimal)
|
||||
|
||||
Goal
|
||||
- Keep `using` simple: strip lines and resolve names to paths/aliases.
|
||||
- Add the minimal integration so userland boxes referenced via `using` are actually available at compile/run time.
|
||||
このドキュメントは再編により移動しました。
|
||||
新しい場所: [../development/design/legacy/using-loader-integration.md](../development/design/legacy/using-loader-integration.md)
|
||||
|
||||
Scope (pause‑safe)
|
||||
- Parser stays Phase‑0: keep `nyashstd` restriction; do not widen grammar.
|
||||
- Integration is a runner step (pre‑parse): resolve and register modules; do not change language semantics.
|
||||
|
||||
Design
|
||||
- Strip `using` lines when `NYASH_ENABLE_USING=1` (already implemented).
|
||||
- For each `using ns [as alias]?`:
|
||||
- Resolve `ns` → path via: [modules] → aliases → using.paths (apps/lib/.) → context dir.
|
||||
- Register mapping in `modules_registry` as `alias_or_ns -> path` (already implemented).
|
||||
- Minimal loader hook (defer heavy linking):
|
||||
- Compile/execute entry file as today.
|
||||
- Userland boxes are accessed via tools/runners that read from `modules_registry` where needed (e.g., PyVM harness/tests).
|
||||
|
||||
Notes
|
||||
- Entry thin‑ization (Mini‑VM) waits until loader reads userland boxes on demand.
|
||||
- Keep docs small: this note serves as the canonical link; avoid duplicating details in other pages.
|
||||
|
||||
Preprocessing invariants (runner)
|
||||
- `using` lines are stripped and resolved prior to parse; dependencies are inlined before `Main` so names are available without changing language semantics.
|
||||
- Line‑head `@name[:T] = expr` is normalized to `local name[:T] = expr` as a purely textual pre‑expand (no semantic change). Inline `@` is not recognized; keep `@` at line head.
|
||||
- These steps are pause‑safe: they do not alter AST semantics; they only simplify authoring and module wiring.
|
||||
|
||||
Links
|
||||
- Runner pipeline: src/runner/pipeline.rs
|
||||
- Using strip/resolve: src/runner/modes/common_util/resolve.rs
|
||||
- Env: NYASH_ENABLE_USING, NYASH_USING_STRICT, NYASH_SKIP_TOML_ENV
|
||||
|
||||
Reference in New Issue
Block a user