tomoaki/hakorune
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,240 Commits 1 Branch 0 Tags
2f07ab6a302550ed686d0978532471697d55152e
Commit Graph

299 Commits

Author SHA1 Message Date
Selfhosting Dev
8e4cadd349 selfhost(pyvm): MiniVmPrints – prefer JSON route early-return (ok==1) to avoid fallback loops; keep default behavior unchanged elsewhere 2025-09-22 07:54:25 +09:00
Selfhosting Dev
c8063c9e41 pyvm: split op handlers into ops_core/ops_box/ops_ctrl; add ops_flow + intrinsic; delegate vm.py without behavior change 
net-plugin: modularize constants (consts.rs) and sockets (sockets.rs); remove legacy commented socket code; fix unused imports
mir: move instruction unit tests to tests/mir_instruction_unit.rs (file lean-up); no semantic changes
runner/pyvm: ensure using pre-strip; misc docs updates

Build: cargo build ok; legacy cfg warnings remain as before
 2025-09-21 08:53:00 +09:00
Selfhosting Dev
f50f79994f loopform(hints): detect up to 2 assigned vars in loop body (no break/continue) and emit LoopCarrier hint; add smoke for two-vars case 2025-09-20 06:24:33 +09:00
Selfhosting Dev
a811857cc8 mir(hints): add runtime tracing via NYASH_MIR_TRACE_HINTS; wire scope enter/leave at function entry/exit; add smoke to validate trace 2025-09-20 05:50:45 +09:00
Selfhosting Dev
d27ca541d8 mir(hints): wire scope_enter/leave at function entry/exit; llvm(smoke): add guard-literal-or case with skip when mock; all smokes/goldens passing 2025-09-20 05:44:57 +09:00
Selfhosting Dev
8cb93b9f1f tests(macro): inline samples into new directory hierarchy and drop legacy macro_golden_* sources 2025-09-20 03:37:20 +09:00
Selfhosting Dev
486adfbec5 mir(hints): add no-op hint types and sink; expose module (scaffold, zero-cost) 2025-09-20 02:08:52 +09:00
Selfhosting Dev
811e3eb3f8 feat: comprehensive development progress 
- Pattern matching implementation extended in match_expr.rs
- CLI configuration structured with categorized groups (task recommendation completed)
- Python LLVM builder split into function_lower.py (task recommendation completed)
- parse_box_declaration massive function refactored (task recommendation completed)
- Phase 16 Macro Revolution comprehensive planning and documentation
- Archive legacy phase documentation for clean structure
- HTTP message box improvements and performance optimizations
- MIR builder enhancements and control flow improvements

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-19 15:11:57 +09:00
Selfhosting Dev
7dfd55bfdb pyvm: implement TypeOp(check) + strict match-guard smokes; parser: guard support in match; llvm: PHI wiring at block head + incoming normalization; docs: AGENTS LLVM/PHI + guard policy; add tests; plan: refactor parse_box_declaration + TODO triage + clone reduction + CLI split + LLVM builder split; update CURRENT_TASK.md 2025-09-19 10:52:57 +09:00
Selfhosting Dev
9142476484 parser(match): add MVP type patterns (IntegerBox(x)/StringBox(s)) via AST If-chain; keep literal-only path using PeekExpr; add smoke app (apps/tests/match_type_pattern_basic.nyash); build + stage-2 smokes green 2025-09-19 08:34:29 +09:00
Selfhosting Dev
52ab2957e6 refactor(optimizer, parser/box): prune legacy optimizer bodies; delegate public/private single-line fields 
- optimizer: remove unreachable old bodies in normalize_legacy_instructions/normalize_ref_field_access and delegate to optimizer_passes only
- parser/box: route public/private single-line fields through members::fields (supports = init, => computed, {..}+postfix) and keep visibility vectors consistent
- box_def facade: silence dead_code warning for scaffold

Smokes:
- PyVM Stage-2 PASS
- Stage-2 short-circuit PASS
- PyVM collections PASS
(NOTE) using_e2e_smoke is optional and currently failing; unrelated to this change
 2025-09-19 07:19:46 +09:00
Selfhosting Dev
5e818eeb7e stage3: unify to cleanup; MIR return-defer; docs+smokes updated; LLVM(harness): finalize_phis ownership, ret.py simplified, uses-predeclare; cleanup return override green; method-postfix cleanup return WIP (PHI head) 2025-09-19 02:07:38 +09:00
Selfhosting Dev
951a050592 selfhost: introduce using-based imports for compiler/parser/tools; keep includes temporarily. llvm: add PHI wiring JSON trace + unit/integration tests; fast test suite extended. runner: split selfhost helpers, small cleanups. 2025-09-18 13:35:38 +09:00
Selfhosting Dev
5d51086530 python-plugin: RAII (PyOwned/PyBorrowed) + autodecode enum; crate: ny-llvmc --emit exe with NyRT link; tools: build_llvm.sh crate-exe path + crate_exe_smoke; CURRENT_TASK update 2025-09-18 03:57:25 +09:00
Selfhosting Dev
4ea3ca2685 docs/ci: selfhost bootstrap/exe-first workflows; add ny-llvmc scaffolding + JSON v0 schema validation; plan: unify to Nyash ABI v2 (no backwards compat) 2025-09-17 20:33:19 +09:00
Selfhosting Dev
a5054a271b llvm(py): introduce BuildCtx + trace hub; refactor if-merge prepass + PHI wiring into module; unify logs; ctx-enable compare/ret/call/boxcall/externcall/typeop/newbox/safepoint; curated smoke option for if-merge; README updates; keep behavior stable 2025-09-17 16:11:01 +09:00
Selfhosting Dev
2720884a20 bridge/json_v0: split expr lowering; add Ternary/Peek lowering + AST; selfhost Peek JSON emit; add selfhost Peek smoke; warning cleanup in lowering/optimizer/verification 
- Split expr lowering into ; route calls from stmt lowering
- Implement ternary/peek lowering (MIR13 PHI-off=Copy, PHI-on=Phi)
- Extend JSON v0 AST (ExprV0::{Ternary,Peek}, PeekArmV0)
- Selfhost parser_box: emit Peek JSON; add Stage-2 'Peek basic' smoke
- Reduce warnings: remove unused imports/vars in several modules
- current_task: update plan for legacy VM/Interpreter offboarding
 2025-09-17 11:45:57 +09:00
Selfhosting Dev
d99b941218 docs: add MIR13 mode doc and set PHI-off as default; bridge lowering split (if/loop/try); llvmlite resolver stabilization; curated runner default PHI-off; refresh CURRENT_TASK.md 2025-09-17 10:58:12 +09:00
Selfhosting Dev
31f90012e0 refactor: extract helpers (builder_calls: math/env/me; json_v0_bridge: new_block/merge_values) — non-functional cleanup; cargo check + smokes green 2025-09-17 08:20:44 +09:00
Selfhosting Dev
dd53d38c8f builder: widen AST wrapper adoption (Binary/Call/Method/Field) and loop helper usage; non-functional refactor with cargo check passing 2025-09-17 08:11:47 +09:00
Selfhosting Dev
e2379b35b3 ast: add MethodCallExpr/FieldAccessExpr wrappers; builder: adopt wrappers for method/field, and add loop helpers (create_loop_blocks/add_predecessor). No behavior change. 2025-09-17 08:10:21 +09:00
Selfhosting Dev
6b505b5435 builder: add loop helpers (create_loop_blocks/add_predecessor) and adopt in LoopBuilder; use BinaryExpr/CallExpr wrappers in expr lowering (no behavior change) 2025-09-17 07:59:41 +09:00
Selfhosting Dev
f3be8ea5f6 builder/exprs: use AssignStmt and ReturnStmt wrappers for clearer destructuring (no behavior change) 2025-09-17 07:57:29 +09:00
Selfhosting Dev
3f930a0057 builder/stmts: use CallExpr wrapper in build_print_statement for isType/asType pattern (no behavior change) 2025-09-17 07:54:45 +09:00
Selfhosting Dev
e47ee65a40 mir(builder/loops): add in_loop()/depth() helpers; clean up Break lowering comment (no behavior change) 2025-09-17 07:50:15 +09:00
Selfhosting Dev
adbb0201a9 chore(fmt): add legacy stubs and strip trailing whitespace to unblock cargo fmt 2025-09-17 07:43:07 +09:00
Selfhosting Dev
fcf8ce1f3c mir: split build_module phases 2025-09-17 07:29:28 +09:00
Selfhosting Dev
c553f2952d runner/env: centralize CLI/env getters; parser expr split (call/primary); verifier utils direct; optimizer: boxfield peephole; LLVM: branch cond normalize hook; add trace macro scaffolding; refactor common.rs verbose checks 2025-09-17 06:55:39 +09:00
Selfhosting Dev
9dc5c9afb9 refactor: split optimizer/verifier/parser modules (mainline); add runner trace/directives; add LLVM terminator/select scaffolds; extract AST Span; update CURRENT_TASK with remaining plan 2025-09-17 05:56:33 +09:00
Selfhosting Dev
154778fc57 mir: extract CSE into passes::cse and wire it in optimizer. Behavior preserved (count-only, no SSA rewrites). Build + PyVM Stage-2 smokes PASS. 2025-09-17 03:41:04 +09:00
Selfhosting Dev
56463c75f6 mir: introduce passes::dce and minimal MirPass trait; rewire optimizer to use modular DCE pass. Build + PyVM Stage-2 smokes PASS. 2025-09-17 03:32:05 +09:00
Selfhosting Dev
5c9213cd03 smokes: add curated LLVM runner; archive legacy smokes; PHI-off unified across Bridge/Builder; LLVM resolver tracing; minimal Throw lowering; config env getters; dev profile and root cleaner; docs updated; CI workflow runs curated LLVM (PHI-on/off) 2025-09-16 23:49:36 +09:00
Selfhosting Dev
6ca56b0652 feat: 配列/Mapリテラル糖衣構文の実装とネームスペース解決の改善計画 
- ArrayLiteral/MapLiteralのAST定義追加
- パーサーで[...]と{...}構文をサポート
- MIR Builderでnew Box() + push/setへのdesugaring実装
- テストケースとスモークスクリプト追加
- CURRENT_TASK.mdにネームスペース解決Phase-1計画を追記
- 3段階解決順序（ローカル→エイリアス→プラグイン）の設計合意
 2025-09-16 06:13:44 +09:00
Selfhosting Dev
18bc386bc5 chore: 未使用インポートの削除と軽微な警告修正 2025-09-16 04:23:20 +09:00
Selfhosting Dev
40db299bab refactor: Gemini/ChatGPT collaborative refactoring + build fixes 
Major changes:
- Split runner module: 1358→580 lines (via Gemini)
- Create new modules: dispatch.rs, selfhost.rs, pipeline.rs, pipe_io.rs
- Fix build errors from incomplete method migrations
- Add warning to CLAUDE.md about JIT/Cranelift not working
- Create interpreter.rs mode module
- Refactor loop builder into separate module

Build status:
- ✅ Executable builds successfully
- ✅ Basic execution works (tested with print)
- ⚠️ 106 warnings remain (to be cleaned up next)
- ⚠️ execute_mir_mode still in mod.rs (needs further migration)

Note: ChatGPT correctly fixed runner.execute_mir_mode() calls
that I incorrectly changed to super::modes::mir::

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-16 03:54:44 +09:00
Selfhosting Dev
ab76cd35fa refactor: 大規模なモジュールを分割し、コードの構造を改善 
runner, mir/builder, backend/llvm の各モジュールが肥大化していたため、責務ごとにファイルを分割し、見通しを改善するリファクタリングを実施。

### `runner`
- `mod.rs` に集中していたロジックを、以下のモジュールに分割:
    - `tasks.rs`: `nyash.toml` のタスク実行処理
    - `build.rs`: AOTビルドパイプラインの実装
    - `pipeline.rs`: `using` の解決など、パイプライン中のユーティリティ
    - `demos.rs`: デモの実行処理

### `mir/builder`
- `if/else` 文のPHIノード生成ロジックを `stmts.rs` から `phi.rs` へ切り出し。
- `utils.rs` にあったPHI関連のヘルパーも `phi.rs` に集約。
- ASTから自由変数を収集するロジックを `vars.rs` へ切り出し。

### `backend/llvm/compiler/codegen`
- 巨大だった `lower_one_function` 関数を、`function.rs` モジュールとして分離。
- `sanitize_symbol` などのヘルパー関数を `utils.rs` へ移動。
 2025-09-16 01:54:00 +09:00
Selfhosting Dev
94d95dfbcd 🚀 Break/Continue/Try-Catch構文のサポート追加とMIRループ制御強化 
## 主な変更点

### 🎯 MIRループ制御の実装（根治対応）
- src/mir/loop_builder.rs: Break/Continue対応のループコンテキスト管理
  - ループのbreak/continueターゲットブロック追跡
  - ネストループの適切な処理
- src/mir/builder.rs: Break/Continue文のMIR生成実装
- src/tokenizer.rs: Break/Continue/Tryトークン認識追加

### 📝 セルフホストパーサーの拡張
- apps/selfhost-compiler/boxes/parser_box.nyash:
  - Stage-3: break/continue構文受理（no-op実装）
  - Stage-3: try-catch-finally構文受理（構文解析のみ）
  - エラー処理構文の将来対応準備

### 📚 ドキュメント更新
- 論文K（爆速事件簿）: 45事例に更新（4件追加）
  - PyVM迂回路の混乱事件
  - Break/Continue無限ループ事件
  - EXE-first戦略の再発見
- 論文I（開発秘話）: Day 38の重要決定追加

### 🧪 テストケース追加
- apps/tests/: ループ制御とPHIのテストケース
  - nested_loop_inner_break_isolated.nyash
  - nested_loop_inner_continue_isolated.nyash
  - loop_phi_one_sided.nyash
  - shortcircuit関連テスト

## 技術的詳細
- Break/ContinueをMIRレベルで適切に処理
- 無限ループ問題（CPU 99.9%暴走）の根本解決
- 将来の例外処理機能への準備

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-15 22:14:42 +09:00
Selfhosting Dev
5cad0ab20c 🔧 Fix peek expression terminator issues and add ternary operator support 
- Fix LLVM IR terminator missing in peek expression entry blocks
- Add proper jump instructions between peek blocks
- Implement ternary operator (? :) as syntactic sugar for peek
- Update Python LLVM externcall handling for improved compatibility
- Add comprehensive test cases for peek and ternary expressions
- Update language guide with ternary operator documentation

ChatGPTが頑張って修正してくれたにゃ！🐱

Co-Authored-By: ChatGPT <noreply@openai.com>
 2025-09-14 20:30:38 +09:00
Selfhosting Dev
3ba96d9a03 🚀 feat: Multiple improvements for Nyash parser and LLVM backend 
Parser improvements:
- Added expression statement fallback in parse_statement() for flexible syntax
- Fixed ternary operator to use PeekExpr instead of If AST (better lowering)
- Added peek_token() check to avoid ?/?: operator conflicts

LLVM Python improvements:
- Added optional ESC_JSON_FIX environment flag for string concatenation
- Improved PHI generation with better default handling
- Enhanced substring tracking for esc_json pattern

Documentation updates:
- Updated language guide with peek expression examples
- Added box theory diagrams to Phase 15 planning
- Clarified peek vs when syntax differences

These changes enable cleaner parser implementation for self-hosting,
especially for handling digit conversion with peek expressions instead
of 19-line if-else chains.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 19:16:32 +09:00
Selfhosting Dev
13298126c8 fix(llvm): MapBox core-first implementation with plugin fallback by ChatGPT 
Implemented elegant solution for MapBox as core box with plugin fallback:

1. Core-first Strategy:
   - Removed MapBox type_id from nyash_box.toml
   - MapBox now uses env.box.new fallback (core implementation)
   - Consistent with self-hosting goals

2. Plugin Fallback Option:
   - Added NYASH_LLVM_FORCE_PLUGIN_MAP=1 environment variable
   - Allows forcing MapBox to plugin path when needed
   - Preserves flexibility during transition

3. MIR Type Inference:
   - Added MapBox method type inference (size/has/get)
   - Ensures proper return type handling

4. Documentation:
   - Added core vs plugin box explanation in nyrt
   - Clarified the transition strategy

This aligns with Phase 15 goals where basic boxes will eventually
be implemented in Nyash itself for true self-hosting.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-11 23:09:16 +09:00
Selfhosting Dev
e7ad2191de fix(llvm): String/Plugin return value handling improvements by ChatGPT 
Major fixes for LLVM backend string and plugin return value handling:

1. MIR Type Annotations:
   - Added StringBox method return types (substring/concat/replace/trim/toUpper/toLower)
   - Enhanced type inference for BoxCall operations

2. LLVM BinOp String Concatenation:
   - Added safe handle-to-pointer conversion paths
   - Support for ptr+i64 and i64+ptr concatenation patterns
   - Uses nyash.string.concat_hh for handle-based concatenation

3. ExternCall Selection:
   - Smart selection between C string (i8*) and handle (i64) variants
   - Improved print/log function selection based on argument types

4. StringBox Fast-path Optimization:
   - Direct AOT concatenation for StringBox.concat
   - Bypasses plugin path for better performance

5. Consistent String Representation:
   - AOT uses i8* (C string) as primary representation
   - Handles used for print/concat auxiliary paths

6. Build Fix:
   - Removed duplicate plugin.rs to resolve nyrt build conflicts

Results: LLVM plugin return smoke tests now pass (NYASH_LLVM_PLUGIN_RET_SMOKE=1)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-11 20:18:53 +09:00
Selfhosting Dev
faf79c5d52 🔧 Fix all unused_mut warnings: 16 instances cleaned up 
- Remove unnecessary 'mut' from variable declarations
- Clean up code in boxes/, interpreter/, mir/, backend/, and runtime/
- No functional changes, just cleaner code
 2025-09-11 16:34:22 +09:00
moe-charm
c4e1728b8e refactor: centralize box type metadata 2025-09-11 05:22:52 +09:00
moe-charm
4201c63001 mir: read plugin method signatures from nyash_box (#134) 
* mir: load plugin method signatures

* llvm: read box type ids from nyash_box

* mir: show value types in MIR printer
 2025-09-11 03:33:33 +09:00
Selfhosting Dev
c014e78fb4 feat(llvm): Complete plugin system unification and environment variable elimination 
🎉 Major Achievement: LLVM Plugin Environment Variable Problem Completely Resolved

## ✅ Completed Major Features:
1. **Plugin Implementation** ✅ - nyash.plugin.invoke_* functions in nyrt library working
2. **Plugin Calls** ✅ - Method calls working without environment variables
3. **Return Value Type Inference Fix** ✅ - Added plugin method type inference to MIR builder
4. **by-id Unification Complete** ✅ - Removed by-name fallback, unified to method_id system
5. **Environment Variable Elimination** ✅ - Removed NYASH_LLVM_ALLOW_BY_NAME=1 requirement
6. **Simple Execution Achieved** ✅ - ./target/release/nyash --backend llvm program.nyash

## 🔧 Technical Changes:

### Core Fixes:
- **src/mir/builder.rs**: Added plugin method return type inference
  - CounterBox.get() -> Integer
  - MathBox.sqrt() -> Float
  - FileBox.read() -> String
  - FileBox.exists() -> Bool

- **src/backend/llvm/compiler.rs**: Removed by-name fallback completely
  - Deleted NYASH_LLVM_ALLOW_BY_NAME environment variable check
  - Removed ~50 lines of fallback logic
  - Unified to method_id-based calls only

### Documentation Updates:
- **CLAUDE.md**: Updated all plugin examples to remove environment variables
- **README.md/README.ja.md**: Removed environment variable documentation
- **tools/llvm_smoke.sh**: Removed NYASH_LLVM_ALLOW_BY_NAME from all test scripts

### Performance & Maintainability:
- **Better Performance**: method_id calls more efficient than by-name lookups
- **Type Safety**: method_id system provides compile-time guarantees
- **Code Simplification**: Removed complex fallback logic
- **User Experience**: No environment variables to remember

## 🧪 Verification:
- ✅ Plugin execution without environment variables
- ✅ method_id injection working: [LLVM] method_id injected: 4-5 places
- ✅ Type inference working: [BUILDER] Type inference: CounterBox get -> Integer
- ✅ Compilation success with LLVM backend

## 🔍 Remaining Investigation:
Plugin return value display issue identified as separate runtime layer problem
(plugin methods execute and side effects work, but return values not displayed in print())

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-10 23:24:02 +09:00
moe-charm
1b5a55c795 mir: include continue snapshots in phi sealing (#130) 2025-09-10 17:39:46 +09:00
moe-charm
06c77c1026 chore(mir): prune legacy instruction sets (#129) 2025-09-10 17:23:26 +09:00
Tomoaki
08d9b71297 grammar: introduce bitwise ops & shifts; retire legacy >> ARROW 
- Tokenizer: add tokens for << >> & | ^ (SHIFT_LEFT/RIGHT, BIT_AND/OR/XOR); keep ||, &&, |>
- Parser: precedence layers for bit ops (| ^ &) and shift (<< >>); remove >> Arrow production
- AST: add BitAnd/BitOr/BitXor/Shr with Display
- MIR builder: map bit ops and Shr to MIR BinaryOp
- Interpreter: implement integer-only bit ops and shifts, mask shift count to 0..63

Compatibility: legacy >> Arrow removed from parser; use |> for pipeline.
 2025-09-08 03:54:34 +09:00
Tomoaki
7c2b09c647 grammar: add '<<' shift-left safely (Phase 1); map to AST/MIR/VM; keep '>>' as legacy ARROW 
- Tokenizer: add SHIFT_LEFT for '<<' (before <= detection)
- Parser: introduce parse_shift() layer; left-associative; only '<<' handled in Phase 1
- AST: add BinaryOperator::Shl and display
- MIR builder: map Shl -> BinaryOp::Shl
- Interpreter: implement Shl in both execution paths (integer-only, type error otherwise)

No change to '>>' (legacy ARROW remains). No runtime gate yet for SHL as it does not collide with existing syntax.
 2025-09-08 03:44:55 +09:00
Tomoaki
864473336e merge: resolve conflicts (prefer cranelift-dev for Core-13 defaults; drop modules/using, add env.local/env.box shims) 2025-09-08 01:28:39 +09:00