tomoaki/hakorune
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,423 Commits 1 Branch 0 Tags
c117a0403594ca10ef81cf35c67e405f20b6ad80
Commit Graph

38 Commits

Author SHA1 Message Date
tomoaki
f740e6542f feat(phase285): Complete weak reference implementation (VM + LLVM harness) 
Phase 285LLVM-1.1 to 1.4 + weak reference infrastructure:

**LLVM Harness** (Phase 285LLVM-1.x):
- 285LLVM-1.1: User Box registration & debug output
- 285LLVM-1.2: WeakRef basic operations (identity deferred)
- 285LLVM-1.3: InstanceBox field access (getField/setField)
- 285LLVM-1.4: print Handle resolution (type tag propagation)

**VM Runtime** (nyash_kernel):
- FFI functions: nyrt_weak_new, nyrt_weak_to_strong, nyrt_weak_drop
  (crates/nyash_kernel/src/lib.rs: +209 lines)
- WeakRef plugin invoke support
  (crates/nyash_kernel/src/plugin/invoke.rs: +250 lines)
- weak_handles.rs: WeakRef handle registry (NEW)

**LLVM Python Backend**:
- WeakRef instruction lowering (weak.py: NEW)
- Entry point integration (entry.py: +93 lines)
- Instruction lowering (instruction_lower.py: +13 lines)
- LLVM harness runner script (tools/run_llvm_harness.sh: NEW)

**MIR & Runtime**:
- WeakRef emission & validation
- MIR JSON export for weak instructions
- Environment variable support (NYASH_WEAK_*, HAKO_WEAK_*)

**Documentation**:
- CLAUDE.md: Phase 285 completion notes
- LANGUAGE_REFERENCE_2025.md: Weak reference syntax
- 10-Now.md & 30-Backlog.md: Phase 285 status updates

Total: +864 lines, 24 files changed

SSOT: docs/reference/language/lifecycle.md
Related: Phase 285W-Syntax-0, Phase 285W-Syntax-0.1
 2025-12-25 00:11:34 +09:00
tomoaki
9a76a199ee feat(phase276-p0): Quick Win improvements - type helper SSOT + debug cleanup 
Phase 276 P0 post-Phase 275 robustness improvements:

1. Debug cleanup (wiring.py L100-103):
   - Remove traceback.format_stack() output
   - Cleaner debug logs after Phase 275 completion

2. box_from_f64 usage investigation:
   - Confirmed ZERO usage (Rust/Python/MIR)
   - Ready for deletion (Phase 275 unboxed double SSOT)
   - crates/nyash_kernel/src/lib.rs L244-252, L971-982

3. Type resolution SSOT (⭐ most important):
   - New file: src/llvm_py/phi_wiring/type_helper.py (72 lines)
   - Unified 3 duplicate logic sites:
     * tagging.py: inst.get("dst_type") → type_helper.get_phi_dst_type()
     * llvm_builder.py: 9 lines → 2 lines (-7)
     * wiring.py: 18 lines → 5 lines (-13)
   - Priority: MIR JSON dst_type > resolver.value_types
   - Benefits: SSOT principle, bug prevention, easy extension

4. Type mismatch warning enhancement (wiring.py L63-79):
   - CRITICAL warning for predeclared PHI type mismatch
   - PhiManager.invalidate_phi() notification
   - Early bug detection with ⚠️  marker

Effects:
- Type resolution logic: 3 sites → 1 SSOT (type_helper.py)
- Code reduction: -20 lines duplicate logic
- Robustness: SSOT principle, CRITICAL warnings, Fail-Fast
- Debuggability: cleaner logs, prominent type mismatch alerts

Testing:
- LLVM harness: exit=3 verified (/tmp/test_p275_debug2.hako)
- NYASH_PHI_TYPE_DEBUG=1: correct type resolution confirmed

Docs:
- Phase 276 P0 completion: docs/.../phase-276/P0-COMPLETION.md
- 10-Now.md: Phase 276 P0 ✅, Phase 275 P0 completed section

Next steps (Phase 277 P0 recommended):
- Delete box_from_f64 (nyash.box.from_f64, nyash.float.box_from_f64)
- Adopt dst_type_to_llvm_type() helper

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-22 13:39:55 +09:00
tomoaki
60cffb1948 feat(phase275-p0): Float型PHI完全対応 - MIR型伝播→LLVM IR生成 
Phase 275 P0: Float型PHIノードの完全実装

【問題】
- PHI v36 が i64 型で生成されるが、incoming value v12 が f64 型
- LLVM IR parsing error: 型不一致により実行失敗

【解決】
1. MIR型伝播順序修正 (lifecycle.rs)
   - BinOp repropagation → PHI resolution に変更
   - Int+Float → Float promotion を PHI推論前に実行

2. MIR JSON Float型出力 (mir_json_emit.rs)
   - PHI dst_type に Float case 追加（2箇所）

3. LLVM PHI型対応 (wiring.py, tagging.py, llvm_builder.py)
   - ensure_phi に dst_type パラメータ追加
   - f64 の場合 ir.DoubleType() PHI 生成
   - 型互換性チェック付き predeclared PHI 再利用

4. LLVM Float演算対応
   - compare.py: ensure_double helper (Float unboxing for fcmp)
   - binop.py: Int+Float → sitofp→fadd (unboxed double)

【検証】
- LLVM IR: phi double [%"float_int_add", %"bb0"] ✅
- VM/LLVM parity: exit=3 ✅
- PHI duplicate 解消: 型チェックで再利用 ✅

【影響範囲】
- Rust MIR builder: 3ファイル (型伝播ロジック)
- LLVM harness: 5ファイル (PHI型対応)
- Kernel: 1ファイル (box_from_f64 helper - 予備)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-22 13:31:53 +09:00
nyash-codex
e0fb6fecf6 refactor(llvm): Phase 132-P1 - FunctionLowerContext Box isolation 
Structural fix for cross-function state leakage bugs discovered in Phase 131-132.

Problem:
- Function-local state (block_end_values, def_blocks, phi_manager, caches)
  was globally managed, causing collisions across functions
- Required manual clearing via _clear_function_local_state() (~80 lines)
- Tuple-key workaround (func_name, block_id) added complexity

Solution: FunctionLowerContext Box
- Encapsulates all function-scoped state in a dedicated Box
- Automatic lifetime management (created at entry, destroyed at exit)
- Eliminates manual state clearing
- Simplifies from tuple-key to simple block_id access

Implementation:
1. Created src/llvm_py/context/function_lower_context.py (150+ lines)
   - block_end_values, def_blocks, jump_only_blocks
   - phi_manager, resolver_caches
   - Helper methods: get/set_block_snapshot(), register_def(), etc.

2. Updated function_lower.py
   - Creates context at function entry
   - Binds resolver to context for cache isolation
   - Removed _clear_function_local_state() (~80 lines)

3. Updated block_lower.py
   - Changed tuple-key (func_name, block_id) to simple block_id
   - Access via context.get_block_snapshot() / context.set_block_snapshot()

4. Updated resolver.py, phi_wiring/wiring.py, phi_wiring/tagging.py
   - All state access now through context

5. Fixed critical bug in phi_wiring/tagging.py
   - setup_phi_placeholders() was breaking context connection
   - Changed reassignment to .clear()/.update() to preserve reference

Benefits:
- ✅ Automatic state isolation (no manual clearing)
- ✅ Clear ownership (one context per function)
- ✅ Eliminated tuple-keys (simpler code)
- ✅ Prevents bugs by design (cross-function leakage impossible)

Test results:
- ✅ Phase 87 smoke test: PASS
- ✅ Phase 132 smoke test: PASS (both cases)
- ✅ STRICT mode: Works with multi-function inputs
- ✅ No regression

Files modified: 8 (2 new, 6 updated)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-15 11:26:10 +09:00
nyash-codex
3f58f34592 feat(llvm): Phase 132-P0 - block_end_values tuple-key fix for cross-function isolation 
## Problem
`block_end_values` used block ID only as key, causing collisions when
multiple functions share the same block IDs (e.g., bb0 in both
condition_fn and main).

## Root Cause
- condition_fn's bb0 → block_end_values[0]
- main's bb0 → block_end_values[0] (OVERWRITES!)
- PHI resolution gets wrong snapshot → dominance error

## Solution (Box-First principle)
Change key from `int` to `Tuple[str, int]` (func_name, block_id):

```python
# Before
block_end_values: Dict[int, Dict[int, ir.Value]]

# After
block_end_values: Dict[Tuple[str, int], Dict[int, ir.Value]]
```

## Files Modified (Python - 6 files)

1. `llvm_builder.py` - Type annotation update
2. `function_lower.py` - Pass func_name to lower_blocks
3. `block_lower.py` - Use tuple keys for snapshot save/load
4. `resolver.py` - Add func_name parameter to resolve_incoming
5. `wiring.py` - Thread func_name through PHI wiring
6. `phi_manager.py` - Debug traces

## Files Modified (Rust - cleanup)

- Removed deprecated `loop_to_join.rs` (297 lines deleted)
- Updated pattern lowerers for cleaner exit handling
- Added lifecycle management improvements

## Verification

- ✅ Pattern 1: VM RC: 3, LLVM Result: 3 (no regression)
- ⚠️ Case C: Still has dominance error (separate root cause)
  - Needs additional scope fixes (phi_manager, resolver caches)

## Design Principles

- **Box-First**: Each function is an isolated Box with scoped state
- **SSOT**: (func_name, block_id) uniquely identifies block snapshots
- **Fail-Fast**: No cross-function state contamination

## Known Issues (Phase 132-P1)

Other function-local state needs same treatment:
- phi_manager.predeclared
- resolver caches (i64_cache, ptr_cache, etc.)
- builder._jump_only_blocks

## Documentation

- docs/development/current/main/investigations/phase132-p0-case-c-root-cause.md
- docs/development/current/main/investigations/phase132-p0-tuple-key-implementation.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-15 05:36:50 +09:00
nyash-codex
447d4ea246 feat(llvm): Phase 132 - Pattern 1 exit value parity fix + Box-First refactoring 
## Phase 132: Exit PHI Value Parity Fix

### Problem
Pattern 1 (Simple While) returned 0 instead of final loop variable value (3)
- VM: RC: 3 ✅ (correct)
- LLVM: Result: 0 ❌ (wrong)

### Root Cause (Two Layers)
1. **JoinIR/Boundary**: Missing exit_bindings → ExitLineReconnector not firing
2. **LLVM Python**: block_end_values snapshot dropping PHI values

### Fix
**JoinIR** (simple_while_minimal.rs):
- Jump(k_exit, [i_param]) passes exit value

**Boundary** (pattern1_minimal.rs):
- Added LoopExitBinding with carrier_name="i", role=LoopState
- Enables ExitLineReconnector to update variable_map

**LLVM** (block_lower.py):
- Use predeclared_ret_phis for reliable PHI filtering
- Protect builder.vmap PHIs from overwrites (SSOT principle)

### Result
- ✅ VM: RC: 3
- ✅ LLVM: Result: 3
- ✅ VM/LLVM parity achieved

## Phase 132-Post: Box-First Refactoring

### Rust Side
**JoinModule::require_function()** (mod.rs):
- Encapsulate function search logic
- 10 lines → 1 line (90% reduction)
- Reusable for Pattern 2-5

### Python Side
**PhiManager Box** (phi_manager.py - new):
- Centralized PHI lifecycle management
- 47 lines → 8 lines (83% reduction)
- SSOT: builder.vmap owns PHIs
- Fail-Fast: No silent overwrites

**Integration**:
- LLVMBuilder: Added phi_manager
- block_lower.py: Delegated to PhiManager
- tagging.py: Register PHIs with manager

### Documentation
**New Files**:
- docs/development/architecture/exit-phi-design.md
- docs/development/current/main/investigations/phase132-llvm-exit-phi-wrong-result.md
- docs/development/current/main/phases/phase-132/

**Updated**:
- docs/development/current/main/10-Now.md
- docs/development/current/main/phase131-3-llvm-lowering-inventory.md

### Design Principles
- Box-First: Logic encapsulated in classes/methods
- SSOT: Single Source of Truth (builder.vmap for PHIs)
- Fail-Fast: Early explicit failures, no fallbacks
- Separation of Concerns: 3-layer architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-15 03:17:31 +09:00
nyash-codex
eb70dfc5bb refactor(llvm): Phase 131-12-P2 - block_end_values SSOT 化（WIP） 
## 実装内容
- get_end_values() API 追加
- _value_at_end_i64() を snapshot-only に変更
- def_blocks 即時更新
- PHI incoming を snapshot から取得

## 発見された問題
- 同一ブロック内の def→use が predecessor snapshot を見てしまう
- これは次フェーズで resolve_cur / resolve_incoming 分離で修正

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-14 21:55:44 +09:00
nyash-codex
22753f8cc3 fix(llvm): Phase 131-7 - Multi-pass vmap sync 修正（Branch bug 解消） 
Phase 131-7: Multi-pass block lowering の vmap sync バグ修正

問題:
- Pass A (body instructions) が vmap_cur に値を保存
- Pass C (deferred terminators) が builder.vmap を参照
- → Pass A の値が Pass C に届かない
- → Branch condition が 0 に fallback → 無限ループ

修正:
- src/llvm_py/builders/block_lower.py (lines 237-248):
  - PHI-only フィルタを削除
  - 全ての値を global vmap に sync

変更ファイル:
- src/llvm_py/builders/block_lower.py: vmap sync 修正
- src/llvm_py/instructions/controlflow/branch.py: trace logging 追加
- src/llvm_py/instructions/unop.py: trace logging 追加
- src/llvm_py/llvm_builder.py: debug helpers 追加
- src/llvm_py/phi_wiring/wiring.py: trace logging 追加
- src/runner/modes/common_util/exec.rs: Python stdout 転送

結果:
- ✅ Branch bug 修正: ループが正しく終了
- ✅ 出力: 無限ループ → 3回で停止
- ❌ 新 bug 発見: ExternCall が null を受け取る（PHI 値の代わりに）

Next: Phase 131-8 (ExternCall argument handling 修正)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-14 07:38:21 +09:00
nyash-codex
d7b8f79a75 feat(phase131): LLVM backend re-enable & PHI ordering bug discovery 
Phase 131: JoinIR → LLVM individual fixes (minimal scope)

## Modifications

### 1. LLVM Backend Re-enable ✅
- Built with --features llvm to enable real LLVM execution
- Verified Python/llvmlite environment (llvmlite 0.45.1)
- Fixed llvmlite deprecated API: removed llvm.initialize() call
- Successfully moved from Mock backend to real LLVM harness execution

### 2. PHI Instruction Ordering Bug Discovery ⚠️
- Discovered critical bug: PHI nodes placed AFTER terminator instructions
- LLVM IR constraint: PHI must be at block start, before any non-PHI
- Root cause: finalize_phis() in llvm_builder.py emits PHIs after block termination
- Affects all 6 tests with control flow merges

### 3. ConsoleBox LLVM Integration ⚠️
- Confirmed ConsoleBox not registered in Rust VM environment
- Deferred to Phase 132 (prerequisite issue)

## Results

Modified files:
- src/llvm_py/llvm_builder.py: Removed deprecated llvm.initialize()
- docs/development/current/main/phase130_joinir_llvm_baseline.md: Added Phase 131 results
- CURRENT_TASK.md: Added Phase 131 completion report

Test results:
- ✅ peek_expr_block.hako: LLVM execution success (Result: 1)
- ❌ 6/7 tests: PHI ordering bug (requires Phase 132 refactoring)

## Success Criteria

- ✅ LLVM backend minimal re-enable (1/7 test passing)
- ⚠️ PHI ordering bug discovered and documented
- ⚠️ ConsoleBox integration deferred to Phase 132

## Phase 132 Handoff

Priority 1: Fix PHI instruction ordering bug (finalize_phis() refactoring)
Priority 2: Resolve ConsoleBox registration issue
Priority 3: Enable remaining test cases for LLVM

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-12-04 11:04:29 +09:00
nyash-codex
dda65b94b7 Phase 21.7 normalization: optimization pre-work + bench harness expansion 
- Add opt-in optimizations (defaults OFF)
  - Ret purity verifier: NYASH_VERIFY_RET_PURITY=1
  - strlen FAST enhancement for const handles
  - FAST_INT gate for same-BB SSA optimization
  - length cache for string literals in llvmlite
- Expand bench harness (tools/perf/microbench.sh)
  - Add branch/call/stringchain/arraymap/chip8/kilo cases
  - Auto-calculate ratio vs C reference
  - Document in benchmarks/README.md
- Compiler health improvements
  - Unify PHI insertion to insert_phi_at_head()
  - Add NYASH_LLVM_SKIP_BUILD=1 for build reuse
- Runtime & safety enhancements
  - Clarify Rust/Hako ownership boundaries
  - Strengthen receiver localization (LocalSSA/pin/after-PHIs)
  - Stop excessive PluginInvoke→BoxCall rewrites
- Update CURRENT_TASK.md, docs, and canaries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-13 16:40:58 +09:00
nyash-codex
9e2fa1e36e Phase 21.6 solidification: chain green (return/binop/loop/call); add Phase 21.7 normalization plan (methodize static boxes). Update CURRENT_TASK.md and docs. 2025-11-11 22:35:45 +09:00
nyash-codex
7b1f791395 feat(phase21.5): Loop FORCE direct assembly + PHI/compare fixes 
## Loop FORCE Direct Assembly ✅
- Added: Direct MIR assembly bypass when HAKO_MIR_BUILDER_LOOP_FORCE_JSONFRAG=1
- Implementation: Extracts limit from Program(JSON), generates minimal while-form
- Structure: entry(0) → loop(1) → body(2) → exit(3)
- PHI: i = {i0, entry} | {i_next, body}
- Location: tools/hakorune_emit_mir.sh:70-126
- Tag: [selfhost-direct:ok] Direct MIR assembly (FORCE=1)

## PHI/Compare Fixes (ny-llvmc) ✅
- Fixed: vmap maintenance for PHI results across instructions
- Fixed: PHI placeholder name consistency (bytes vs str)
- Fixed: ensure_phi_alloca creates unique placeholders per block
- Fixed: resolve_i64_strict properly looks up PHI results
- Files:
  - src/llvm_py/phi_wiring/tagging.py
  - src/llvm_py/phi_wiring/wiring.py
  - src/llvm_py/instructions/compare.py
  - src/llvm_py/resolver.py

## Testing Results
- VM backend: ✅ rc=10 (correct)
- Direct assembly MIR: ✅ Structurally correct
- Crate backend: ⚠️ PHI/compare issues (being investigated)

## Implementation Principles
- 既定挙動不変 (FORCE=1 gated)
- Dev toggle controlled
- Minimal diff, surgical changes
- Bypasses using resolution when FORCE=1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-11 17:04:33 +09:00
nyash-codex
f6c5dc9e43 Phase 22.x WIP: LLVM backend improvements + MIR builder enhancements 
LLVM backend improvements:
- Add native LLVM backend support (NYASH_LLVM_BACKEND=native)
- Add crate backend selector with priority (crate > llvmlite)
- Add native_llvm_builder.py for native IR generation
- Add NYASH_LLVM_NATIVE_TRACE=1 for IR dump

MIR builder enhancements:
- Refactor lower_if_compare_* boxes for better code generation
- Refactor lower_return_* boxes for optimized returns
- Refactor lower_loop_* boxes for loop handling
- Refactor lower_method_* boxes for method calls
- Update pattern_util_box for better pattern matching

Smoke tests:
- Add phase2100 S3 backend selector tests (17 new tests)
- Add phase2120 native backend tests (4 new tests)
- Add phase2034 MIR builder internal tests (2 new tests)
- Add phase2211 TLV shim parity test

Documentation:
- Update ENV_VARS.md with LLVM backend variables
- Update CURRENT_TASK.md with progress
- Update README.md and CHANGELOG.md

Config:
- Add NYASH_LLVM_BACKEND env support in src/config/env.rs
- Update ny_mir_builder.sh for backend selection
- Update dispatch.rs for backend routing

Tools:
- Add tools/native_llvm_builder.py
- Update smokes/v2/profiles/quick/core/phase2100/run_all.sh

Known: Many Hako builder internal files modified for optimization
 2025-11-09 23:40:36 +09:00
nyash-codex
0455307418 refactor(phase-a): remove Cranelift/JIT backend legacy code (~373 lines) 
Phase A cleanup - Safe deletions with zero risk:

## Deleted Files (6 files, 373 lines total)
1. Cranelift/JIT Backend (321 lines):
   - src/runner/modes/cranelift.rs (45 lines)
   - src/runner/modes/aot.rs (55 lines)
   - src/runner/jit_direct.rs (152 lines)
   - src/tests/core13_smoke_jit.rs (42 lines)
   - src/tests/core13_smoke_jit_map.rs (27 lines)

2. Legacy MIR Builder (52 lines):
   - src/mir/builder/exprs_legacy.rs
   - Functionality inlined into exprs.rs (control flow constructs)

## Module Reference Cleanup
- src/backend/mod.rs: Removed cranelift feature gate exports
- src/runner/mod.rs: Removed jit_direct module reference
- src/runner/modes/mod.rs: Removed aot module reference
- src/mir/builder.rs: Removed exprs_legacy module

## Impact Analysis
- Build: Success (cargo build --release)
- Tests: All passing
- Risk Level: None (feature already archived, code unused)
- Related: Phase 15 JIT archival (archive/jit-cranelift/)

## BID Copilot Status
- Already removed in previous cleanup
- Not part of this commit

Total Reduction: 373 lines (~0.4% of codebase)
Next: Phase B - Dead code investigation

Related: #phase-21.0-cleanup
Part of: Legacy Code Cleanup Initiative
 2025-11-06 22:34:18 +09:00
nyash-codex
8b71f25dd4 Stage-A selfhost emitter fixes and LLVM opt toggle 2025-10-31 23:16:27 +09:00
nyash-codex
cb236b7f5a json(vm): fix birth dispatch; unify constructor naming (Box.birth/N); JsonNode factories return JsonNodeInstance; quick: enable heavy JSON with probe; builder: NYASH_BUILDER_DEBUG_LIMIT guard; json_query_min(core) harness; docs/tasks updated 2025-09-27 08:45:25 +09:00
Selfhosting Dev
041cef875a json-native: token positions (line/column); escape utils BMP coverage + surrogate guard; add smokes for string escapes, nested, and error cases (AST/VM) 2025-09-26 00:42:55 +09:00
Selfhosting Dev
497da75f90 macroctx: pass ctx JSON (caps) to user macros; add demo macro; docs: MacroCtx section and sandbox Box API; golden for for/foreach normalized AST 2025-09-20 09:11:52 +09:00
Selfhosting Dev
da32455afc freeze: macro platform complete; default ON with profiles; env consolidation; docs + smokes\n\n- Profiles: --profile {lite|dev|ci|strict} (dev-like default for macros)\n- Macro paths: prefer NYASH_MACRO_PATHS (legacy envs deprecated with warnings)\n- Selfhost pre-expand: auto mode, PyVM-only, add smokes (array/map)\n- Docs: user-macros updated; new macro-profiles guide; AGENTS freeze note; CURRENT_TASK freeze\n- Compat: non-breaking; legacy envs print deprecation notices\n 2025-09-19 22:27:59 +09:00
Selfhosting Dev
811e3eb3f8 feat: comprehensive development progress 
- Pattern matching implementation extended in match_expr.rs
- CLI configuration structured with categorized groups (task recommendation completed)
- Python LLVM builder split into function_lower.py (task recommendation completed)
- parse_box_declaration massive function refactored (task recommendation completed)
- Phase 16 Macro Revolution comprehensive planning and documentation
- Archive legacy phase documentation for clean structure
- HTTP message box improvements and performance optimizations
- MIR builder enhancements and control flow improvements

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-19 15:11:57 +09:00
Selfhosting Dev
3c7a5de900 runner(cli): adopt CliConfig::as_groups across runner modules (dispatch/common/pipe_io/mir/bench). llvm-builder: extract ny_main wrapper to builders.entry; add optional env-gated function_lower delegation; keep default behavior unchanged 2025-09-19 14:29:02 +09:00
Selfhosting Dev
4f82edf901 llvm: gate empty PHI sanitize behind NYASH_LLVM_SANITIZE_EMPTY_PHI=1; docs: AGENTS.md note. perf: begin clone() reduction plan — TypeBox.name/type_parameters to Arc<str>, adjusted builders/registry; no behavior change 2025-09-19 11:54:28 +09:00
Selfhosting Dev
7dfd55bfdb pyvm: implement TypeOp(check) + strict match-guard smokes; parser: guard support in match; llvm: PHI wiring at block head + incoming normalization; docs: AGENTS LLVM/PHI + guard policy; add tests; plan: refactor parse_box_declaration + TODO triage + clone reduction + CLI split + LLVM builder split; update CURRENT_TASK.md 2025-09-19 10:52:57 +09:00
Selfhosting Dev
2d870664d5 current_task: refresh notes; cleanup unified; LLVM(harness): finalize_phis ownership, ret simplified, uses-predeclare; method-postfix cleanup return: PHI head grouping final TODOs 2025-09-19 02:43:33 +09:00
Selfhosting Dev
5e818eeb7e stage3: unify to cleanup; MIR return-defer; docs+smokes updated; LLVM(harness): finalize_phis ownership, ret.py simplified, uses-predeclare; cleanup return override green; method-postfix cleanup return WIP (PHI head) 2025-09-19 02:07:38 +09:00
Selfhosting Dev
4ea3ca2685 docs/ci: selfhost bootstrap/exe-first workflows; add ny-llvmc scaffolding + JSON v0 schema validation; plan: unify to Nyash ABI v2 (no backwards compat) 2025-09-17 20:33:19 +09:00
Selfhosting Dev
a5054a271b llvm(py): introduce BuildCtx + trace hub; refactor if-merge prepass + PHI wiring into module; unify logs; ctx-enable compare/ret/call/boxcall/externcall/typeop/newbox/safepoint; curated smoke option for if-merge; README updates; keep behavior stable 2025-09-17 16:11:01 +09:00
Selfhosting Dev
d99b941218 docs: add MIR13 mode doc and set PHI-off as default; bridge lowering split (if/loop/try); llvmlite resolver stabilization; curated runner default PHI-off; refresh CURRENT_TASK.md 2025-09-17 10:58:12 +09:00
Selfhosting Dev
ee4224d7d2 llvm_py: extract setup_phi_placeholders() and keep snapshots in lower_block (no behavior change) 2025-09-17 07:45:50 +09:00
Selfhosting Dev
63c8fda808 🔍 Research: GPT-5-Codex capabilities and GitHub PR integration 
## Summary
Investigated OpenAI's new GPT-5-Codex model and Codex GitHub PR review integration capabilities.

## GPT-5-Codex Analysis

### Benchmark Performance (Good)
- SWE-bench Verified: 74.5% (vs GPT-5's 72.8%)
- Refactoring tasks: 51.3% (vs GPT-5's 33.9%)
- Code review: Higher developer ratings

### Real-World Issues (Concerning)
- Users report degraded coding performance
- Scripts that previously worked now fail
- Less consistent than GPT-4.5
- Longer response times (minutes vs instant)
- "Creatively and emotionally flat"
- Basic errors (e.g., counting letters incorrectly)

### Key Finding
Classic case of "optimizing for benchmarks vs real usability" - scores well on tests but performs poorly in practice.

## Codex GitHub PR Integration

### Setup Process
1. Enable MFA and connect GitHub account
2. Authorize Codex GitHub app for repos
3. Enable "Code review" in repository settings

### Usage Methods
- **Manual**: Comment '@codex review' in PR
- **Automatic**: Triggers when PR moves from draft to ready

### Current Limitations
- One-way communication (doesn't respond to review comments)
- Prefers creating new PRs over updating existing ones
- Better for single-pass reviews than iterative feedback

## 'codex resume' Feature
New session management capability:
- Resume previous codex exec sessions
- Useful for continuing long tasks across days
- Maintains context from interrupted work

🐱 The investigation reveals that while GPT-5-Codex shows benchmark improvements, practical developer experience has declined - a reminder that metrics don't always reflect real-world utility\!
 2025-09-16 16:28:25 +09:00
Selfhosting Dev
3ba96d9a03 🚀 feat: Multiple improvements for Nyash parser and LLVM backend 
Parser improvements:
- Added expression statement fallback in parse_statement() for flexible syntax
- Fixed ternary operator to use PeekExpr instead of If AST (better lowering)
- Added peek_token() check to avoid ?/?: operator conflicts

LLVM Python improvements:
- Added optional ESC_JSON_FIX environment flag for string concatenation
- Improved PHI generation with better default handling
- Enhanced substring tracking for esc_json pattern

Documentation updates:
- Updated language guide with peek expression examples
- Added box theory diagrams to Phase 15 planning
- Clarified peek vs when syntax differences

These changes enable cleaner parser implementation for self-hosting,
especially for handling digit conversion with peek expressions instead
of 19-line if-else chains.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 19:16:32 +09:00
Selfhosting Dev
4c0e6726e3 🔧 refactor(llvm-py): Fix resolver PHI handling and add trace improvements 
Changes to resolver.py:
- Improved PHI value tracking in _value_at_end_i64() (lines 268-285)
- Added trace logging for snap hits with PHI detection
- Fixed PHI placeholder reuse logic to preserve dominance
- PHI values now returned directly from snapshots when valid

Changes to llvm_builder.py:
- Fixed externcall instruction parsing (line 522: 'func' instead of 'name')
- Improved block snapshot tracing (line 439)
- Added PHI incoming metadata tracking (lines 316-376)
- Enhanced definition tracking for lifetime hints

This should help debug the string carry=0 issue in esc_dirname_smoke where
PHI values were being incorrectly coerced instead of preserved.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 16:25:21 +09:00
Selfhosting Dev
3e07763af8 docs: update CURRENT_TASK with Box Theory PHI plan (defer/finalize) and MIR v0.5 type meta; add parity tooling and PyVM scaffolding 
impl(pyvm/llvmlite):
- add tools/parity.sh; tools/pyvm_runner.py; src/llvm_py/pyvm/*
- emit string const as handle type in MIR JSON; add dst_type hints
- unify '+' to concat_hh with from_i64/from_i8_string bridges; console print via to_i8p_h
- add runtime bridges: nyash.box.from_i64, nyash.string.to_i8p_h

tests:
- add apps/tests/min_str_cat_loop (minimal repro for string cat loop)
 2025-09-14 04:51:33 +09:00
Selfhosting Dev
658a0d46da feat(llvm-py): Major breakthrough in Python LLVM backend! 🎉 
✅ Print and FileBox paths now working correctly
✅ Resolver simplified by removing overly aggressive fast-path optimization
✅ Both OFF/ON in compare_harness_on_off.sh now use Python version
✅ String handle propagation issues resolved

Key changes:
- Removed instruction reordering in llvm_builder.py (respecting MIR order)
- Resolver now more conservative but reliable
- compare_harness_on_off.sh updated to use Python backend for both paths

This marks a major milestone towards Phase 15 self-hosting with Python/llvmlite!

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-14 00:44:28 +09:00
Selfhosting Dev
2a9aa5368d harness(llvm/py): fix PHI/dominance via Resolver-only; per-pred localization and constant GEPs; stabilize Main.esc_json/1, dirname/1, node_json/3; docs: add NYASH_LLVM_TRACE_FINAL and Resolver-only invariants 
- Resolver-only reads across BBs; remove vmap fallbacks
- Create PHIs at block start; insert casts in preds before terminators
- Re-materialize int in preds to satisfy dominance (add/zext/trunc)
- Use constant GEP for method strings to avoid order dependency
- Order non-PHI lowering to preserve producer→consumer dominance
- Update docs: RESOLVER_API.md, LLVM_HARNESS.md
- compare_harness_on_off: ON/OFF exits match; linking green
 2025-09-13 19:49:03 +09:00
Selfhosting Dev
1d6fab4eda 📚 Phase 15計画を詳細化・更新: Python/llvmlite正式採用とプラグイン全方向ビルド戦略 
✅ 主な更新内容:
- Python/llvmlite実装の正式採用を明記（開発速度10倍、~2400行）
- プラグイン全方向ビルド戦略（.so/.o/.a同時生成）で単一EXE生成可能に
- 各実装の予想コード量を具体化（パーサー800行、MIR Builder 2500行、VM 5000行）
- 循環依存問題の解決を明記（nyrtがC ABI経由で提供）
- 現実的なスケジュール調整（2025年9月～2026年3月）

🎉 最新進捗:
- dep_tree_min_string.nyashオブジェクト生成成功（10.4KB）
- LLVM verifier green - dominance違反解決
- Resolver patternでSSA安全性確保

🚀 次のマイルストーン:
- Python/llvmliteでEXE生成パイプライン完成
- nyash-llvm-compiler分離設計
- NyashパーサーMVP実装開始

Everything is Boxの究極形が、ついに実現へ！

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-09-13 15:37:58 +09:00
Selfhosting Dev
ef44801fa6 ✨ Python LLVM backend implementation (experimental) 
- Created llvmlite-based LLVM backend in src/llvm_py/
- Implemented all MIR14 instructions (const, binop, jump, branch, ret, compare, phi, call, boxcall, externcall, typeop, newbox, safepoint, barrier)
- Experimental LoopForm support
- ~2000 lines of clean Python code vs complex Rust/inkwell
- Useful for PHI/SSA validation and rapid prototyping
- Added documentation to CLAUDE.md

This was created while waiting for ChatGPT's investigation of BuilderCursor issues.
 2025-09-12 20:55:13 +09:00
Selfhosting Dev
38aea59fc1 llvm: unify lowering via Resolver and Cursor; remove non-sealed PHI wiring; apply Resolver to extern/call/boxcall/arrays/maps/mem; add llvmlite harness docs; add LLVM layer overview; add LoopForm preheader 2025-09-12 20:40:48 +09:00