Commit Graph

19 Commits

Author SHA1 Message Date
187edfcaaf 🏗️ Phase 22: Revolutionary Nyash LLVM Compiler vision
- Create Phase 22 documentation for Nyash-based LLVM compiler
- C++ thin wrapper (20-30 functions) + Nyash implementation (100-200 lines)
- Gemini & Codex discussions: Both AIs confirm technical feasibility
- Build time revolution: 5-7min → instant changes
- Code reduction: 2,500 lines → 100-200 lines (95% reduction\!)
- User insight: 'Why worry about memory leaks for a 3-second batch process?'
- Ultimate 'Everything is Box' philosophy: Even the compiler is a Box\!

🌟 Vision: After Phase 15 LLVM stabilization, we can build anything\!
2025-09-12 04:03:43 +09:00
b120e4a26b refactor(llvm): Complete Call instruction modularization + Phase 21 organization
## LLVM Call Instruction Modularization
- Moved MirInstruction::Call lowering to separate instructions/call.rs
- Follows the principle of one MIR instruction per file
- Call implementation was already complete, just needed modularization

## Phase 21 Documentation
- Moved all Phase 21 content to private/papers/paper-f-self-parsing-db/
- Preserved AI evaluations from Gemini and Codex
- Academic paper potential confirmed by both AIs
- Self-parsing AST database approach validated

## Next Steps
- Continue monitoring ChatGPT5's LLVM improvements
- Consider creating separate nyash-llvm-compiler crate when LLVM layer is stable
- This will reduce build times by isolating LLVM dependencies

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-12 01:58:07 +09:00
40d0cac0f1 feat(llvm): Complete function call system implementation by ChatGPT5
Major improvements to LLVM backend function call infrastructure:

## Key Changes

### Function Call System Complete
- All MIR functions now properly lowered to LLVM (not just entry)
- Function parameter binding to LLVM arguments implemented
- ny_main() wrapper added for proper entry point handling
- Callee resolution from ValueId to function symbols working

### Call Instruction Analysis
- MirInstruction::Call was implemented but system was incomplete
- Fixed "rhs missing" errors caused by undefined Call return values
- Function calls now properly return values through the system

### Code Modularization (Ongoing)
- BoxCall → instructions/boxcall.rs ✓
- ExternCall → instructions/externcall.rs ✓
- Call remains in mod.rs (to be refactored)

### Phase 21 Documentation
- Added comprehensive AI evaluation from Gemini and Codex
- Both AIs confirm academic paper potential for self-parsing AST DB approach
- "Code as Database" concept validated as novel contribution

Co-authored-by: ChatGPT5 <noreply@openai.com>

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-12 01:45:00 +09:00
4f4c6397a9 🏗️ Refactor: Major LLVM codegen modularization + Phase 15 docs cleanup + Phase 21 DDD concept
## LLVM Codegen Refactoring (by ChatGPT5)
- Split massive boxcall.rs into focused submodules:
  - strings.rs: String method optimizations (concat, length)
  - arrays.rs: Array operations (get, set, push, length)
  - maps.rs: Map operations (get, set, has, size)
  - fields.rs: getField/setField handling
  - invoke.rs: Tagged invoke implementation
  - marshal.rs: Helper functions for marshaling
- Improved code organization and maintainability
- No functional changes, pure refactoring

## Phase 15 Documentation Cleanup
- Restructured phase-15 folder:
  - implementation/: Technical implementation docs
  - planning/: Planning and sequence docs
  - archive/: Redundant/old content
- Removed duplicate content (80k→20k line reduction mentioned 5 times)
- Converted all .txt files to .md for consistency
- Fixed broken links in README.md
- Removed redundant INDEX.md

## Phase 21: Database-Driven Development (New)
- Revolutionary concept: Source code in SQLite instead of files
- Instant refactoring with SQL transactions
- Structured management of boxes, methods, dependencies
- Technical design with security considerations
- Vision: World's first DB-driven programming language

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-12 00:35:11 +09:00
3ac4a383e4 refactor(llvm): Modularize instructions.rs into focused submodules by ChatGPT
Successfully split the massive instructions.rs (1400+ lines) into organized submodules:

Structure:
- instructions/mod.rs - Module exports and wiring
- instructions/blocks.rs - Basic block creation and PHI setup
- instructions/flow.rs - Control flow (Return, Jump, Branch)
- instructions/externcall.rs - External call handling
- instructions/newbox.rs - NewBox operations
- instructions/boxcall.rs - BoxCall lowering (main dispatch)
- instructions/strings.rs - String fast-paths (concat, length)
- instructions/arrays.rs - Array operations (get/set/push/length)
- instructions/maps.rs - Map operations (size/get/set/has)
- instructions/arith.rs - Arithmetic operations (UnaryOp, BinOp, Compare)
- instructions/mem.rs - Memory operations (Load, Store)
- instructions/consts.rs - Constant value handling

Benefits:
- Improved maintainability (each file ~200-400 lines)
- Clear separation of concerns
- No behavior changes (pure refactoring)
- All existing smoke tests pass

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 23:58:10 +09:00
13298126c8 fix(llvm): MapBox core-first implementation with plugin fallback by ChatGPT
Implemented elegant solution for MapBox as core box with plugin fallback:

1. Core-first Strategy:
   - Removed MapBox type_id from nyash_box.toml
   - MapBox now uses env.box.new fallback (core implementation)
   - Consistent with self-hosting goals

2. Plugin Fallback Option:
   - Added NYASH_LLVM_FORCE_PLUGIN_MAP=1 environment variable
   - Allows forcing MapBox to plugin path when needed
   - Preserves flexibility during transition

3. MIR Type Inference:
   - Added MapBox method type inference (size/has/get)
   - Ensures proper return type handling

4. Documentation:
   - Added core vs plugin box explanation in nyrt
   - Clarified the transition strategy

This aligns with Phase 15 goals where basic boxes will eventually
be implemented in Nyash itself for true self-hosting.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 23:09:16 +09:00
89e6fbf010 feat(llvm): Comprehensive LLVM backend improvements by ChatGPT
Major enhancements to LLVM code generation and type handling:

1. String Operations:
   - Added StringBox length fast-path (length/len methods)
   - Converts i8* to handle when needed for len_h call
   - Consistent handle-based string operations

2. Array/Map Fast-paths:
   - ArrayBox: get/set/push/length operations
   - MapBox: get/set/has/size with handle-based keys
   - Optimized paths for common collection operations

3. Field Access:
   - getField/setField implementation with handle conversion
   - Proper i64 handle to pointer conversions

4. NewBox Improvements:
   - StringBox/IntegerBox pass-through optimizations
   - Fallback to env.box.new when type_id unavailable
   - Support for dynamic box creation

5. Documentation:
   - Added ARCHITECTURE.md for overall design
   - Added EXTERNCALL.md for external call specs
   - Added LOWERING_LLVM.md for LLVM lowering rules
   - Added PLUGIN_ABI.md for plugin interface

6. Type System:
   - Added UserBox type registration in nyash_box.toml
   - Consistent handle (i64) representation across system

Results: More robust LLVM code generation with proper type handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 22:30:26 +09:00
0ac22427e5 docs: Architecture decision - Box/ExternCall boundary design
Documented the architectural decision for Nyash runtime design:

1. Core boxes (String/Integer/Array/Map/Bool) built into nyrt
   - Essential for self-hosting
   - Available at boot without plugin loader
   - High performance (no FFI overhead)

2. All other boxes as plugins (File/Net/User-defined)
   - Extensible ecosystem
   - Clear separation of concerns

3. Minimal ExternCall (only 5 functions)
   - print/error (output)
   - panic/exit (process control)
   - now (time)

Key principle: Everything goes through BoxCall interface
- No special fast paths
- Unified architecture
- "Everything is Box" philosophy maintained

This design balances self-hosting requirements with architectural purity.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 20:58:18 +09:00
e7ad2191de fix(llvm): String/Plugin return value handling improvements by ChatGPT
Major fixes for LLVM backend string and plugin return value handling:

1. MIR Type Annotations:
   - Added StringBox method return types (substring/concat/replace/trim/toUpper/toLower)
   - Enhanced type inference for BoxCall operations

2. LLVM BinOp String Concatenation:
   - Added safe handle-to-pointer conversion paths
   - Support for ptr+i64 and i64+ptr concatenation patterns
   - Uses nyash.string.concat_hh for handle-based concatenation

3. ExternCall Selection:
   - Smart selection between C string (i8*) and handle (i64) variants
   - Improved print/log function selection based on argument types

4. StringBox Fast-path Optimization:
   - Direct AOT concatenation for StringBox.concat
   - Bypasses plugin path for better performance

5. Consistent String Representation:
   - AOT uses i8* (C string) as primary representation
   - Handles used for print/concat auxiliary paths

6. Build Fix:
   - Removed duplicate plugin.rs to resolve nyrt build conflicts

Results: LLVM plugin return smoke tests now pass (NYASH_LLVM_PLUGIN_RET_SMOKE=1)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 20:18:53 +09:00
b9f9e81c72 refactor(llvm): Major progress on codegen modularization by ChatGPT
Major changes:
- Removed 617 lines of duplicate/legacy code from mod.rs (lines 351-967)
- All BoxCall handling now properly delegated to instructions::lower_boxcall
- Updated CURRENT_TASK.md with new findings:
  - String concatenation issue (BinOp type mismatch)
  - Plugin return value smoke test added
  - Clear next steps for fixing return value display

Key improvements:
- Clean separation between dispatch (mod.rs) and implementation (instructions.rs)
- Legacy code marked as unreachable and ready for removal
- Better error visibility with modularized code structure
- llvm_smoke.sh updated with new plugin return value tests

Next steps:
1. Fix BinOp string concatenation type handling
2. Investigate MIR value_types for BoxCall returns
3. Further split lower_boxcall function (still 260+ lines)
2025-09-11 18:16:08 +09:00
89dd518408 refactor(llvm): Further modularization progress by ChatGPT
- BoxCall handling now properly delegated to instructions::lower_boxcall
- Removed duplicate code in mod.rs (lines 351+ were unreachable after continue)
- Clean separation between dispatch (mod.rs) and implementation (instructions.rs)
- Preparing for further BoxCall function breakdown

Work in progress - ChatGPT continuing refactoring efforts
2025-09-11 17:59:51 +09:00
1fd37bf14a refactor(llvm): Complete modularization of codegen.rs by Codex
- Split 2522-line codegen.rs into modular structure:
  - mod.rs (1330 lines) - main compilation flow and instruction dispatch
  - instructions.rs (1266 lines) - all MIR instruction implementations
  - types.rs (189 lines) - type conversion and classification helpers
  - helpers.rs retained for shared utilities

- Preserved all functionality including:
  - Plugin return value handling (BoxCall/ExternCall)
  - Handle-to-pointer conversions for proper value display
  - Type-aware return value processing based on MIR metadata
  - All optimization paths (ArrayBox fast-paths, string concat, etc.)

- Benefits:
  - Better code organization and maintainability
  - Easier to locate specific functionality
  - Reduced cognitive load when working on specific features
  - Cleaner separation of concerns

No functional changes - pure refactoring to improve code structure.
2025-09-11 17:51:43 +09:00
335aebb041 🏗️ Refactor: Split massive codegen.rs (2522 lines) into modular structure
Thanks to Codex's powerful refactoring\!
- codegen.rs → codegen/ directory with 3 focused modules
- mod.rs (1498 lines) - main compilation flow
- instructions.rs (1121 lines) - MIR instruction implementations
- types.rs (189 lines) - type conversion helpers

Benefits:
- Much easier to locate errors and debug
- Better separation of concerns
- Enables parallel development
- Maintains API compatibility

Co-authored-by: Codex <codex@openai.com>
2025-09-11 17:34:30 +09:00
faf79c5d52 🔧 Fix all unused_mut warnings: 16 instances cleaned up
- Remove unnecessary 'mut' from variable declarations
- Clean up code in boxes/, interpreter/, mir/, backend/, and runtime/
- No functional changes, just cleaner code
2025-09-11 16:34:22 +09:00
ba33431f02 🎯 Complete ptr_type deprecation fixes: 200+ → 96 warnings (53% reduction!)
- Fix all deprecated ptr_type() warnings (30+ instances)
- LLVM 15.0 migration: Type::ptr_type() → Context::ptr_type()
- Remove unused i64p variables in codegen.rs and helpers.rs
- Clean up remaining unused imports

Significant milestone achieved in warning cleanup campaign!
2025-09-11 16:30:32 +09:00
c6a8d0b686 🧹 Major warning cleanup: 200+ → 102 warnings (50% reduction)
- Fix TokenType enum naming convention (SNAKE_CASE → CamelCase)
- Remove 60+ unused imports across multiple modules
- Clean up interpreter, backend, and box modules
- LLVM build now passes with significantly fewer warnings
2025-09-11 16:24:18 +09:00
94156251a5 llvm: restore codegen and interpreter pipeline 2025-09-11 12:14:11 +09:00
93e571103f fix(llvm): Implement handle-based console.log in MIR interpreter
- MIR interpreter now converts NyashBox to handles for console methods
- Added debug output to trace handle conversion
- Fixed plugin return values not being displayed
- Part of ongoing investigation into LLVM backend plugin issues

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-11 11:38:08 +09:00
c4e1728b8e refactor: centralize box type metadata 2025-09-11 05:22:52 +09:00