tomoaki/hakorune
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
797 Commits 1 Branch 0 Tags
c8063c9e41485ed9f976cbf9cd71c04bfb7f57b2
Commit Graph

306 Commits

Author SHA1 Message Date
Selfhosting Dev
c8063c9e41 pyvm: split op handlers into ops_core/ops_box/ops_ctrl; add ops_flow + intrinsic; delegate vm.py without behavior change 
net-plugin: modularize constants (consts.rs) and sockets (sockets.rs); remove legacy commented socket code; fix unused imports
mir: move instruction unit tests to tests/mir_instruction_unit.rs (file lean-up); no semantic changes
runner/pyvm: ensure using pre-strip; misc docs updates

Build: cargo build ok; legacy cfg warnings remain as before
 2025-09-21 08:53:00 +09:00
Selfhosting Dev
497da75f90 macroctx: pass ctx JSON (caps) to user macros; add demo macro; docs: MacroCtx section and sandbox Box API; golden for for/foreach normalized AST 2025-09-20 09:11:52 +09:00
Selfhosting Dev
daa5309ea9 docs: fix macro sandbox policy and lock minimal Box API (Console/String/Array/Map); clarify plugin-off applies to macro child only 2025-09-20 09:05:20 +09:00
Selfhosting Dev
8a84339ac2 core: for/foreach -> Loop normalization (always-on); LoopForm MVP-3 per-segment reorder; smokes stabilized (VM + LLVM PHI); docs updated (macro-system, loopform); quiet macro load logs 2025-09-20 08:39:40 +09:00
Selfhosting Dev
1805ab3df2 mir(hints): add JoinResult trace smoke; add assign_both_branches case; docs: trace usage in scope-hints; all smokes pass 2025-09-20 05:55:25 +09:00
Selfhosting Dev
166c374eec macro(if/match): stabilize PeekExpr→If normalization via internal-child; default runner off; propagate child stderr; fix JsonBuilder local_decl; add scope-hints doc; extend PHI smoke; golden for match literal passes 2025-09-20 05:00:31 +09:00
Selfhosting Dev
3c1486e411 tests(macro): organize under apps/tests/macro/{if,loopform,collections,types,strings,identity,test_runner} with thin include wrappers; update all golden/smoke scripts and docs to new paths 2025-09-20 02:20:02 +09:00
Selfhosting Dev
9b9080d0a3 macro/pattern: add type_is basic golden (MethodCall .is → MIR TypeOp mapping); docs: AST JSON v0 note for is/as mapping 2025-09-20 01:51:55 +09:00
Selfhosting Dev
c5ab95d082 docs(cf/pattern): add ControlFlowBuilder and PatternBuilder guides; reference from If/Match normalization; plan updated in CURRENT_TASK 2025-09-20 01:15:26 +09:00
Selfhosting Dev
d185811541 docs: add three breakthrough papers Q, R, S for academic publication 
- Paper Q: Unified Grammar Engine for AI-Language Collaboration (緊急性高)
- Paper R: ScopeBox Theory - Zero-Cost Abstraction (Gemini絶賛)
- Paper S: LoopForm Revolution - PHI Problem Solution (技術革新)

Updated PAPER_INDEX.md with new papers and reorganized priorities.
Added comprehensive README documentation for each new paper.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-20 01:11:36 +09:00
Selfhosting Dev
9999c7c456 llvm: add LoopForm PHI hygiene smoke (no empty PHI); docs: MIR hints (zero-cost); macros: scaffold If/Match normalize (identity) 2025-09-20 01:06:54 +09:00
Selfhosting Dev
02c8244964 scopebox: add design guide; scaffold no-op scope/defer macro; add MIR hint plan to CURRENT_TASK 2025-09-20 00:59:50 +09:00
Selfhosting Dev
1d965c7241 docs(cf-normalize): add If/Match normalization guide and note in CURRENT_TASK Next; confirm pipeline order before LoopForm 2025-09-20 00:36:25 +09:00
Selfhosting Dev
7a57abfb95 docs(loopform): add carrier normalization examples and local run tips; task: Next short + 48h Action Items updated for LoopForm MVP-2/3 and LLVM PHI smoke 2025-09-20 00:32:29 +09:00
Selfhosting Dev
65227cb56f loopform: add loop_simple golden and test; prepare while normalization pathway via LoopNormalize + JsonBuilder 2025-09-19 23:18:07 +09:00
Selfhosting Dev
0c09460286 loopform: MVP normalization in LoopNormalize macro (canonicalize Loop via JsonBuilder); docs touch 2025-09-19 23:15:35 +09:00
Selfhosting Dev
1d309283b6 jsonbuilder: add apps/lib/json_builder.nyash and docs; wire loop_normalize_macro to prepare for usage; prefer Nyash runner route 2025-09-19 23:06:28 +09:00
Selfhosting Dev
9a0fe71215 docs: prefer Nyash runner route (self-hosting); switch to NYASH_MACRO_PATHS; deprecations noted; update smokes to new envs 2025-09-19 22:59:29 +09:00
Selfhosting Dev
65cb504c7d macro: default to Nyash runner route for user macros (self-hosting first); docs note 2025-09-19 22:55:37 +09:00
Selfhosting Dev
c4dda4ce01 macro(loopform): add LoopNormalize behavior routing (identity for MVP); AST JSON Local support 2025-09-19 22:52:22 +09:00
Selfhosting Dev
9d40e9137a macro(ast-json): add Loop/Break/Continue to AST JSON v0; add loop_normalize_macro (MVP identity) and loopform smoke; docs update 2025-09-19 22:47:12 +09:00
Selfhosting Dev
c55f9d3689 docs(phase-17): add LoopForm Self-Hosting phase with schedule/design/guide; link from AGENTS\n\n- New: docs/development/roadmap/phases/phase-17-loopform-selfhost/{README,SCHEDULE,loopform-design}.md\n- Guide: docs/guides/loopform.md\n- AGENTS: link phase-17 + loopform guide\n 2025-09-19 22:39:04 +09:00
Selfhosting Dev
da32455afc freeze: macro platform complete; default ON with profiles; env consolidation; docs + smokes\n\n- Profiles: --profile {lite|dev|ci|strict} (dev-like default for macros)\n- Macro paths: prefer NYASH_MACRO_PATHS (legacy envs deprecated with warnings)\n- Selfhost pre-expand: auto mode, PyVM-only, add smokes (array/map)\n- Docs: user-macros updated; new macro-profiles guide; AGENTS freeze note; CURRENT_TASK freeze\n- Compat: non-breaking; legacy envs print deprecation notices\n 2025-09-19 22:27:59 +09:00
Selfhosting Dev
7dfd55bfdb pyvm: implement TypeOp(check) + strict match-guard smokes; parser: guard support in match; llvm: PHI wiring at block head + incoming normalization; docs: AGENTS LLVM/PHI + guard policy; add tests; plan: refactor parse_box_declaration + TODO triage + clone reduction + CLI split + LLVM builder split; update CURRENT_TASK.md 2025-09-19 10:52:57 +09:00
Selfhosting Dev
9142476484 parser(match): add MVP type patterns (IntegerBox(x)/StringBox(s)) via AST If-chain; keep literal-only path using PeekExpr; add smoke app (apps/tests/match_type_pattern_basic.nyash); build + stage-2 smokes green 2025-09-19 08:34:29 +09:00
Selfhosting Dev
45e1d57536 docs(lang): adopt match as unified pattern matching; remove peek from docs; update EBNF and guides 
- Decision recorded in CURRENT_TASK: replace  with  (no compat alias)
- EBNF: add match_expr/pattern/guard; note legacy peek removal
- Guides/Reference: update examples and wording (null-coalesce/safe-access sugar via match)
 2025-09-19 07:42:38 +09:00
Selfhosting Dev
5e818eeb7e stage3: unify to cleanup; MIR return-defer; docs+smokes updated; LLVM(harness): finalize_phis ownership, ret.py simplified, uses-predeclare; cleanup return override green; method-postfix cleanup return WIP (PHI head) 2025-09-19 02:07:38 +09:00
Selfhosting Dev
3fe908eb0d docs/runtime: externcall + println normalization; NyRT silent result flag; PyVM console warn/error/trace routing\nselfhost(M2): add MirEmitterBox (Return(Int)->const+ret) and --emit-mir switch; initial selfhost smoke\nrunner: child args/env hardening for selfhost (NYASH_NY_COMPILER_CHILD_ARGS under --; NYASH_JSON_ONLY=1, NYASH_VM_USE_PY=1, NYASH_DISABLE_PLUGINS=1); VM path strips using lines when NYASH_ENABLE_USING=1\nmeta: clarify PyVM/llvmlite as primary (vm/interpreter legacy OFF) in CURRENT_TASK.md; README links to externcall docs 2025-09-18 06:35:49 +09:00
Selfhosting Dev
df66ea3ecb cli: add --emit-exe (+ --emit-exe-nyrt/--emit-exe-libs) to build native exe via ny-llvmc; docs updated 2025-09-18 04:23:11 +09:00
Selfhosting Dev
847a1f09e9 docs: update LLVM_HARNESS.md and selfhost-pilot for ny-llvmc exe; add tools/crate_exe_smoke.sh 2025-09-18 03:58:36 +09:00
Selfhosting Dev
b420929f53 chore(legacy): remove unused legacy plugin loader and C-ABI nyash-file plugin; unify plugin headers to TypeBox v2 2025-09-17 22:32:38 +09:00
Selfhosting Dev
d93595ab83 docs(plugins): update headers to TypeBox v2; expand v2 doc with Regex/Net method tables; remove BID mentions where v2 exists 2025-09-17 22:30:19 +09:00
Selfhosting Dev
fcf8042622 docs(abi): align v2 with Phase‑12 blueprint; add roadmap/forward‑compat notes and Phase‑15 status pointer 2025-09-17 21:58:47 +09:00
Selfhosting Dev
4ea3ca2685 docs/ci: selfhost bootstrap/exe-first workflows; add ny-llvmc scaffolding + JSON v0 schema validation; plan: unify to Nyash ABI v2 (no backwards compat) 2025-09-17 20:33:19 +09:00
Selfhosting Dev
d99b941218 docs: add MIR13 mode doc and set PHI-off as default; bridge lowering split (if/loop/try); llvmlite resolver stabilization; curated runner default PHI-off; refresh CURRENT_TASK.md 2025-09-17 10:58:12 +09:00
Selfhosting Dev
9dc5c9afb9 refactor: split optimizer/verifier/parser modules (mainline); add runner trace/directives; add LLVM terminator/select scaffolds; extract AST Span; update CURRENT_TASK with remaining plan 2025-09-17 05:56:33 +09:00
Selfhosting Dev
5c9213cd03 smokes: add curated LLVM runner; archive legacy smokes; PHI-off unified across Bridge/Builder; LLVM resolver tracing; minimal Throw lowering; config env getters; dev profile and root cleaner; docs updated; CI workflow runs curated LLVM (PHI-on/off) 2025-09-16 23:49:36 +09:00
Selfhosting Dev
97a76c0571 selfhost: stub stage3 parser nodes under gate 2025-09-16 18:36:54 +09:00
Selfhosting Dev
47f4ca0e44 📚 docs: Record field declaration design discussion in papers 
## Summary
Documented the "init block vs fields-at-top" design discussion as a valuable example of AI-human collaboration in language design.

## Changes

### Paper G (AI Collaboration)
- Added field-declaration-design.md documenting the entire discussion flow
- Showcased how complex init block proposal evolved to simple "fields at top" rule
- Demonstrates AI's tendency toward complexity vs human intuition for simplicity

### Paper H (AI Practical Patterns)
- Added Pattern #17: "Gradual Refinement Pattern" (段階的洗練型)
- Documents the process: Complex AI proposal → Detailed analysis → Human insight → Convergence
- Field declaration design as a typical example

### Paper K (Explosive Incidents)
- Added Incident #046: "init block vs fields-at-top incident"
- Updated total count to 46 incidents
- Shows how a single human comment redirected entire design approach

## Design Decision
After analysis, decided that BoxIndex should remain a compiler-internal structure, not a core Box:
- Core Boxes: User-instantiable runtime values (String, Integer, Array, Map)
- Compiler internals: BoxIndex for name resolution (compile-time only)
- Clear separation of concerns between language features and compiler tools

## Philosophy
This discussion exemplifies key principles:
- The best design needs no explanation
- Constraints provide clarity, not limitation
- "Everything is Box" doesn't mean "compiler internals are Boxes"
- AI tends toward theoretical completeness; humans toward practical simplicity

🐱 Sometimes the simplest answer is right in front of us\!
 2025-09-16 14:57:05 +09:00
Selfhosting Dev
6ca56b0652 feat: 配列/Mapリテラル糖衣構文の実装とネームスペース解決の改善計画 
- ArrayLiteral/MapLiteralのAST定義追加
- パーサーで[...]と{...}構文をサポート
- MIR Builderでnew Box() + push/setへのdesugaring実装
- テストケースとスモークスクリプト追加
- CURRENT_TASK.mdにネームスペース解決Phase-1計画を追記
- 3段階解決順序（ローカル→エイリアス→プラグイン）の設計合意
 2025-09-16 06:13:44 +09:00
Selfhosting Dev
94d95dfbcd 🚀 Break/Continue/Try-Catch構文のサポート追加とMIRループ制御強化 
## 主な変更点

### 🎯 MIRループ制御の実装（根治対応）
- src/mir/loop_builder.rs: Break/Continue対応のループコンテキスト管理
  - ループのbreak/continueターゲットブロック追跡
  - ネストループの適切な処理
- src/mir/builder.rs: Break/Continue文のMIR生成実装
- src/tokenizer.rs: Break/Continue/Tryトークン認識追加

### 📝 セルフホストパーサーの拡張
- apps/selfhost-compiler/boxes/parser_box.nyash:
  - Stage-3: break/continue構文受理（no-op実装）
  - Stage-3: try-catch-finally構文受理（構文解析のみ）
  - エラー処理構文の将来対応準備

### 📚 ドキュメント更新
- 論文K（爆速事件簿）: 45事例に更新（4件追加）
  - PyVM迂回路の混乱事件
  - Break/Continue無限ループ事件
  - EXE-first戦略の再発見
- 論文I（開発秘話）: Day 38の重要決定追加

### 🧪 テストケース追加
- apps/tests/: ループ制御とPHIのテストケース
  - nested_loop_inner_break_isolated.nyash
  - nested_loop_inner_continue_isolated.nyash
  - loop_phi_one_sided.nyash
  - shortcircuit関連テスト

## 技術的詳細
- Break/ContinueをMIRレベルで適切に処理
- 無限ループ問題（CPU 99.9%暴走）の根本解決
- 将来の例外処理機能への準備

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-15 22:14:42 +09:00
Selfhosting Dev
d90216e9c4 📚 Phase 15 - セルフホスティング戦略の明確化とEXE-first実装 
## 主な変更点

### 🎯 戦略の転換と明確化
- PyVMを開発ツールとして位置づけ（本番経路ではない）
- EXE-first戦略を明確に優先（build_compiler_exe.sh実装済み）
- Phase順序の整理: 15.2(LLVM)→15.3(コンパイラ)→15.4(VM)

### 🚀 セルフホスティング基盤の実装
- apps/selfhost-compiler/にNyashコンパイラMVP実装
  - compiler.nyash: メインエントリー（位置引数対応）
  - boxes/: parser_box, emitter_box, debug_box分離
- tools/build_compiler_exe.sh: ネイティブEXEビルド＋dist配布
- Python MVPパーサーStage-2完成（local/if/loop/call/method/new）

### 📝 ドキュメント整備
- Phase 15 README/ROADMAP更新（Self-Hosting優先明記）
- docs/guides/exe-first-wsl.md: WSLクイックスタート追加
- docs/private/papers/: 論文G～L、爆速事件簿41事例収録

### 🔧 技術的改善
- JSON v0 Bridge: If/Loop PHI生成実装（ChatGPT協力）
- PyVM/llvmliteパリティ検証スイート追加
- using/namespace機能（gated実装、Phase 15では非解決）

## 次のステップ
1. パーサー無限ループ修正（未実装関数の実装）
2. EXEビルドとセルフホスティング実証
3. c0→c1→c1'ブートストラップループ確立

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-15 18:44:49 +09:00
Selfhosting Dev
8f11c79f19 📚 Add complete 41-incident collection to Paper K 
- 世界記録級TOP10事件の詳細記録
- 16パターン別に全41事例を分類・整理
- JIT1日完成、20日でEXE、AIが人間に相談等の歴史的事件
- 統計データ: 45日間、破綻0回の奇跡
- 印象的なエピソード（こらー！事件、AIパニック等）収録

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-15 04:12:24 +09:00
Selfhosting Dev
af11c6855b 🚀 Start Phase 15.3: Nyash compiler MVP implementation 
Major milestone:
- Set up apps/selfhost-compiler/ directory structure
- Implement basic Nyash compiler in Nyash (CompilerBox)
- Stage-1: Basic arithmetic parser (int/string/+/-/*/括弧/return)
- JSON v0 output compatible with --ny-parser-pipe
- Runner integration with NYASH_USE_NY_COMPILER=1 flag
- Comprehensive smoke tests for PHI/Bridge/Stage-2

Technical updates:
- Updated CLAUDE.md with Phase 15.3 status and MIR14 details
- Statement separation policy: newline-based with minimal ASI
- Fixed runaway ny-parser-pipe processes (CPU 94.9%)
- Clarified MIR14 as canonical instruction set (not 13/18)
- LoopForm strategy: PHI auto-generation during reverse lowering

Collaborative development:
- ChatGPT5 implementing compiler skeleton
- Codex provided LoopForm PHI generation guidance
- Claude maintaining documentation and coordination

🎉 セルフホスティングの歴史的一歩！自分自身をコンパイルする日が近いにゃ！

Co-Authored-By: ChatGPT <noreply@openai.com>
 2025-09-15 01:21:37 +09:00
Selfhosting Dev
5cad0ab20c 🔧 Fix peek expression terminator issues and add ternary operator support 
- Fix LLVM IR terminator missing in peek expression entry blocks
- Add proper jump instructions between peek blocks
- Implement ternary operator (? :) as syntactic sugar for peek
- Update Python LLVM externcall handling for improved compatibility
- Add comprehensive test cases for peek and ternary expressions
- Update language guide with ternary operator documentation

ChatGPTが頑張って修正してくれたにゃ！🐱

Co-Authored-By: ChatGPT <noreply@openai.com>
 2025-09-14 20:30:38 +09:00
Selfhosting Dev
3ba96d9a03 🚀 feat: Multiple improvements for Nyash parser and LLVM backend 
Parser improvements:
- Added expression statement fallback in parse_statement() for flexible syntax
- Fixed ternary operator to use PeekExpr instead of If AST (better lowering)
- Added peek_token() check to avoid ?/?: operator conflicts

LLVM Python improvements:
- Added optional ESC_JSON_FIX environment flag for string concatenation
- Improved PHI generation with better default handling
- Enhanced substring tracking for esc_json pattern

Documentation updates:
- Updated language guide with peek expression examples
- Added box theory diagrams to Phase 15 planning
- Clarified peek vs when syntax differences

These changes enable cleaner parser implementation for self-hosting,
especially for handling digit conversion with peek expressions instead
of 19-line if-else chains.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 19:16:32 +09:00
Selfhosting Dev
4c0e6726e3 🔧 refactor(llvm-py): Fix resolver PHI handling and add trace improvements 
Changes to resolver.py:
- Improved PHI value tracking in _value_at_end_i64() (lines 268-285)
- Added trace logging for snap hits with PHI detection
- Fixed PHI placeholder reuse logic to preserve dominance
- PHI values now returned directly from snapshots when valid

Changes to llvm_builder.py:
- Fixed externcall instruction parsing (line 522: 'func' instead of 'name')
- Improved block snapshot tracing (line 439)
- Added PHI incoming metadata tracking (lines 316-376)
- Enhanced definition tracking for lifetime hints

This should help debug the string carry=0 issue in esc_dirname_smoke where
PHI values were being incorrectly coerced instead of preserved.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 16:25:21 +09:00
Selfhosting Dev
3e07763af8 docs: update CURRENT_TASK with Box Theory PHI plan (defer/finalize) and MIR v0.5 type meta; add parity tooling and PyVM scaffolding 
impl(pyvm/llvmlite):
- add tools/parity.sh; tools/pyvm_runner.py; src/llvm_py/pyvm/*
- emit string const as handle type in MIR JSON; add dst_type hints
- unify '+' to concat_hh with from_i64/from_i8_string bridges; console print via to_i8p_h
- add runtime bridges: nyash.box.from_i64, nyash.string.to_i8p_h

tests:
- add apps/tests/min_str_cat_loop (minimal repro for string cat loop)
 2025-09-14 04:51:33 +09:00
Selfhosting Dev
2a9aa5368d harness(llvm/py): fix PHI/dominance via Resolver-only; per-pred localization and constant GEPs; stabilize Main.esc_json/1, dirname/1, node_json/3; docs: add NYASH_LLVM_TRACE_FINAL and Resolver-only invariants 
- Resolver-only reads across BBs; remove vmap fallbacks
- Create PHIs at block start; insert casts in preds before terminators
- Re-materialize int in preds to satisfy dominance (add/zext/trunc)
- Use constant GEP for method strings to avoid order dependency
- Order non-PHI lowering to preserve producer→consumer dominance
- Update docs: RESOLVER_API.md, LLVM_HARNESS.md
- compare_harness_on_off: ON/OFF exits match; linking green
 2025-09-13 19:49:03 +09:00
Selfhosting Dev
1d6fab4eda 📚 Phase 15計画を詳細化・更新: Python/llvmlite正式採用とプラグイン全方向ビルド戦略 
✅ 主な更新内容:
- Python/llvmlite実装の正式採用を明記（開発速度10倍、~2400行）
- プラグイン全方向ビルド戦略（.so/.o/.a同時生成）で単一EXE生成可能に
- 各実装の予想コード量を具体化（パーサー800行、MIR Builder 2500行、VM 5000行）
- 循環依存問題の解決を明記（nyrtがC ABI経由で提供）
- 現実的なスケジュール調整（2025年9月～2026年3月）

🎉 最新進捗:
- dep_tree_min_string.nyashオブジェクト生成成功（10.4KB）
- LLVM verifier green - dominance違反解決
- Resolver patternでSSA安全性確保

🚀 次のマイルストーン:
- Python/llvmliteでEXE生成パイプライン完成
- nyash-llvm-compiler分離設計
- NyashパーサーMVP実装開始

Everything is Boxの究極形が、ついに実現へ！

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-13 15:37:58 +09:00