1198 lines
36 KiB
Markdown
1198 lines
36 KiB
Markdown
# @enum Implementation Module Architecture
|
|
|
|
**Goal**: Production-ready, clean, maintainable architecture following Hakorune conventions
|
|
|
|
**Date**: 2025-10-08
|
|
**Status**: Design Phase - Ready for Implementation
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
This document defines the complete module placement strategy for @enum implementation (Day 1 scope). The design follows **Separation of Concerns** principle, integrates naturally with existing codebase patterns, and provides clear boundaries for testing and future expansion.
|
|
|
|
**Key Decisions**:
|
|
- Parser: `src/parser/declarations/enum_parser.rs` (NEW)
|
|
- AST: Add 3 structs to `src/ast.rs` (existing monolithic file)
|
|
- Macro Expander: `src/macro/enum_expander.rs` (NEW - Rust for Day 1)
|
|
- Tests: `apps/tests/enum/` (NEW directory for integration tests)
|
|
- Docs: `docs/reference/language/enum-syntax.md` (NEW user guide)
|
|
|
|
---
|
|
|
|
## 1. Current Project Structure Analysis
|
|
|
|
### src/ Structure (Rust Implementation)
|
|
```
|
|
src/
|
|
├── lib.rs [exports all modules]
|
|
├── main.rs [CLI entry point]
|
|
├── ast.rs [monolithic AST definitions - 16,868 bytes]
|
|
├── ast/ [utility modules]
|
|
│ ├── nodes.rs [additional AST utilities]
|
|
│ ├── span.rs [Span implementation]
|
|
│ └── utils.rs [AST helpers]
|
|
├── parser/
|
|
│ ├── mod.rs [main parser - 15,287 bytes]
|
|
│ ├── common.rs [ParserUtils trait]
|
|
│ ├── cursor.rs [token cursor]
|
|
│ ├── declarations/ [DECLARATION PARSERS - TARGET LOCATION]
|
|
│ │ ├── mod.rs [exports: box_def, box_definition, static_box, etc.]
|
|
│ │ ├── box_definition.rs [box/static box parser]
|
|
│ │ ├── box_def/ [box parser internals]
|
|
│ │ ├── static_box.rs [static box parser]
|
|
│ │ ├── static_def/ [static box internals]
|
|
│ │ ├── dependency_helpers.rs [circular dependency detection]
|
|
│ │ └── flow.rs [control flow parsing]
|
|
│ ├── expr/
|
|
│ ├── expressions.rs
|
|
│ ├── items/
|
|
│ └── statements/
|
|
├── macro/ [MACRO SYSTEM - TARGET LOCATION]
|
|
│ ├── mod.rs [macro engine entry]
|
|
│ ├── engine.rs [macro expansion engine]
|
|
│ ├── macro_box.rs [built-in macro boxes]
|
|
│ ├── macro_box_ny.rs [Hakorune macro boxes]
|
|
│ ├── ast_json.rs [AST↔JSON conversion]
|
|
│ ├── pattern.rs [pattern matching]
|
|
│ └── ctx.rs [macro context]
|
|
├── tokenizer/
|
|
├── mir/
|
|
├── backend/
|
|
└── ...
|
|
```
|
|
|
|
**Pattern Observed**:
|
|
- **AST**: Monolithic `ast.rs` (16KB) with some utilities in `ast/` subdirectory
|
|
- **Parser**: Modular by concern (`declarations/`, `expressions/`, `statements/`)
|
|
- **Macro**: Flat structure in `macro/` directory (6-7 files)
|
|
|
|
### apps/ Structure (Hakorune Programs)
|
|
```
|
|
apps/
|
|
├── lib/
|
|
│ └── boxes/ [standard library boxes]
|
|
│ ├── array_std.hako
|
|
│ ├── console_std.hako
|
|
│ ├── map_std.hako
|
|
│ ├── string_std.hako
|
|
│ ├── option.hako [❗ 2-variant enum-like, 1755 bytes]
|
|
│ └── result.hako [❗ 2-variant enum-like, 1874 bytes]
|
|
├── tests/ [integration test programs]
|
|
│ ├── (48 subdirectories)
|
|
│ ├── core/ [core language tests]
|
|
│ ├── array_literal_basic.nyash
|
|
│ └── ...
|
|
├── macros/ [macro implementations]
|
|
│ ├── examples/
|
|
│ └── selfhost_min/
|
|
├── selfhost/ [self-hosting compiler]
|
|
├── selfhost-compiler/
|
|
├── benchmarks/
|
|
└── ...
|
|
```
|
|
|
|
**Pattern Observed**:
|
|
- **Boxes**: Standard library in `apps/lib/boxes/` (`.hako` extension)
|
|
- **Tests**: Flat + categorized subdirectories in `apps/tests/`
|
|
- **Macros**: Some exist in `apps/macros/` but primary impl is Rust (`src/macro/`)
|
|
|
|
---
|
|
|
|
## 2. Architecture Decision: Hybrid Modular Approach
|
|
|
|
### Rationale
|
|
Based on existing patterns:
|
|
1. **Parser follows declarations/ pattern** - Add `enum_parser.rs` to existing `src/parser/declarations/`
|
|
2. **AST stays monolithic** - Add structs to `src/ast.rs` (consistent with current 16KB file)
|
|
3. **Macro in Rust (Day 1)** - Fast prototyping, type-safe, easier debugging
|
|
4. **Tests in dedicated subdirectory** - `apps/tests/enum/` for clear organization
|
|
|
|
### Benefits
|
|
- ✅ **Zero architectural surprises** - follows existing patterns exactly
|
|
- ✅ **Minimal file changes** - only 4 new files + modifications
|
|
- ✅ **Easy to find** - predictable locations for future developers
|
|
- ✅ **Testable** - clear separation between parser/expander/integration tests
|
|
|
|
---
|
|
|
|
## 3. Module Placement Design
|
|
|
|
### 3.1 Parser Module (Token → AST)
|
|
|
|
**Location**: `src/parser/declarations/enum_parser.rs` (NEW)
|
|
|
|
**Responsibility**: Parse `@enum Name { ... }` syntax into AST nodes
|
|
|
|
**Exports**:
|
|
```rust
|
|
// src/parser/declarations/enum_parser.rs
|
|
|
|
use crate::ast::{ASTNode, EnumDeclaration, EnumVariant, EnumField, Span};
|
|
use crate::parser::{NyashParser, ParseError};
|
|
use crate::parser::common::ParserUtils;
|
|
|
|
impl NyashParser {
|
|
/// Parse @enum declaration
|
|
///
|
|
/// Syntax:
|
|
/// @enum EnumName {
|
|
/// VariantA(field1: TypeBox, field2: TypeBox)
|
|
/// VariantB()
|
|
/// VariantC(value: IntegerBox)
|
|
/// }
|
|
pub(crate) fn parse_enum_declaration(&mut self) -> Result<ASTNode, ParseError> {
|
|
// Implementation here
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
// Unit tests for enum parsing only
|
|
// Test AST node construction, error cases, edge cases
|
|
}
|
|
```
|
|
|
|
**Integration**:
|
|
```rust
|
|
// src/parser/declarations/mod.rs
|
|
pub mod enum_parser; // ADD THIS LINE
|
|
|
|
// src/parser/mod.rs (or statements.rs)
|
|
// In parse_statement() or parse_top_level():
|
|
if self.match_token(&TokenType::MacroKeyword) &&
|
|
self.peek_ahead_matches("enum") {
|
|
return self.parse_enum_declaration();
|
|
}
|
|
```
|
|
|
|
**Dependencies**:
|
|
- `crate::ast::{EnumDeclaration, EnumVariant, EnumField}` (new AST nodes)
|
|
- `crate::tokenizer::TokenType::MacroKeyword` (existing or new)
|
|
- `crate::parser::common::ParserUtils` (existing trait)
|
|
|
|
---
|
|
|
|
### 3.2 AST Module (Data Structures)
|
|
|
|
**Location**: `src/ast.rs` (MODIFY - add 3 structs)
|
|
|
|
**Responsibility**: Define AST node structures for enum declarations
|
|
|
|
**New Structures**:
|
|
```rust
|
|
// src/ast.rs
|
|
|
|
/// Enum declaration AST node (parsed from @enum syntax)
|
|
#[derive(Debug, Clone, PartialEq)]
|
|
pub struct EnumDeclaration {
|
|
pub name: String,
|
|
pub variants: Vec<EnumVariant>,
|
|
pub span: Span,
|
|
}
|
|
|
|
/// Enum variant (e.g., "Some(value: T)" or "None()")
|
|
#[derive(Debug, Clone, PartialEq)]
|
|
pub struct EnumVariant {
|
|
pub name: String,
|
|
pub fields: Vec<EnumField>,
|
|
pub span: Span,
|
|
}
|
|
|
|
/// Enum variant field (e.g., "value: IntegerBox")
|
|
#[derive(Debug, Clone, PartialEq)]
|
|
pub struct EnumField {
|
|
pub name: String,
|
|
pub type_name: String, // "IntegerBox", "StringBox", "T", etc.
|
|
pub span: Span,
|
|
}
|
|
|
|
// Add to ASTNode enum:
|
|
pub enum ASTNode {
|
|
// ... existing variants ...
|
|
|
|
/// Enum declaration (macro syntax: @enum Name { ... })
|
|
EnumDeclaration {
|
|
name: String,
|
|
variants: Vec<EnumVariant>,
|
|
span: Span,
|
|
},
|
|
|
|
// ... rest of variants ...
|
|
}
|
|
```
|
|
|
|
**Rationale for Monolithic**:
|
|
- Current `ast.rs` is 16KB - still manageable size
|
|
- No `src/ast/declarations.rs` exists yet
|
|
- Adding 50-80 lines doesn't justify new file
|
|
- **Later** (Phase 19.2+): Consider `src/ast/declarations/` if file grows beyond 25KB
|
|
|
|
**Alternative (Future Consideration)**:
|
|
```
|
|
src/ast/
|
|
├── mod.rs [main ASTNode enum + re-exports]
|
|
├── declarations.rs [EnumDeclaration, BoxDeclaration, etc.]
|
|
├── expressions.rs [ExpressionNode variants]
|
|
└── statements.rs [StatementNode variants]
|
|
```
|
|
|
|
---
|
|
|
|
### 3.3 Tokenizer Integration
|
|
|
|
**Location**: `src/tokenizer/mod.rs` or equivalent (MODIFY)
|
|
|
|
**Responsibility**: Recognize `@enum` keyword
|
|
|
|
**Changes Required**:
|
|
```rust
|
|
// Option A: Reuse existing MacroKeyword token
|
|
// (if @enum, @macro, etc. all use same token)
|
|
|
|
// Option B: Add specific EnumKeyword token
|
|
pub enum TokenType {
|
|
// ... existing tokens ...
|
|
MacroKeyword, // for @ prefix
|
|
// ... or ...
|
|
EnumKeyword, // specifically for @enum
|
|
}
|
|
|
|
// In tokenizer loop:
|
|
'@' => {
|
|
// Check if followed by 'enum'
|
|
if self.peek_word() == "enum" {
|
|
tokens.push(Token::new(TokenType::MacroKeyword, "@", line));
|
|
tokens.push(Token::new(TokenType::Identifier, "enum", line));
|
|
}
|
|
}
|
|
```
|
|
|
|
**Decision**: Use **Option A** (reuse `MacroKeyword`) for consistency with macro system.
|
|
|
|
---
|
|
|
|
### 3.4 Macro Expander Module (AST → Box AST)
|
|
|
|
**Location**: `src/macro/enum_expander.rs` (NEW)
|
|
|
|
**Responsibility**: Transform `EnumDeclaration` AST into `BoxDeclaration` + `StaticBoxDeclaration` AST
|
|
|
|
**Architecture** (Two-Layer):
|
|
```
|
|
Parser Layer:
|
|
parse_enum_declaration()
|
|
↓ produces
|
|
EnumDeclaration AST
|
|
|
|
Macro Expander Layer:
|
|
expand_enum(EnumDeclaration)
|
|
↓ produces
|
|
Vec<ASTNode> [BoxDeclaration, StaticBoxDeclaration]
|
|
```
|
|
|
|
**Implementation**:
|
|
```rust
|
|
// src/macro/enum_expander.rs
|
|
|
|
use crate::ast::{ASTNode, EnumDeclaration, EnumVariant, Span};
|
|
|
|
/// Expand @enum declaration into Box + StaticBox AST nodes
|
|
///
|
|
/// Transformation:
|
|
/// @enum Result { Ok(value: T), Err(error: E) }
|
|
///
|
|
/// Generates:
|
|
/// box ResultBox<T, E> {
|
|
/// variant: StringBox
|
|
/// ok_value: T
|
|
/// err_error: E
|
|
///
|
|
/// birth(variant, ...) { ... }
|
|
/// is_ok() { ... }
|
|
/// is_err() { ... }
|
|
/// unwrap() { ... }
|
|
/// }
|
|
///
|
|
/// static box Result {
|
|
/// Ok(value) { return new ResultBox("Ok", value, null) }
|
|
/// Err(error) { return new ResultBox("Err", null, error) }
|
|
/// }
|
|
pub fn expand_enum(decl: EnumDeclaration) -> Result<Vec<ASTNode>, EnumExpansionError> {
|
|
let box_node = generate_box_declaration(&decl)?;
|
|
let static_node = generate_static_box(&decl)?;
|
|
|
|
Ok(vec![box_node, static_node])
|
|
}
|
|
|
|
fn generate_box_declaration(decl: &EnumDeclaration) -> Result<ASTNode, EnumExpansionError> {
|
|
// Generate BoxDeclaration with:
|
|
// - variant: StringBox field
|
|
// - All variant fields (with nullable types for variants that don't use them)
|
|
// - birth(variant, ...) constructor
|
|
// - is_* query methods
|
|
// - unwrap/unwrap_or/map/... utility methods
|
|
}
|
|
|
|
fn generate_static_box(decl: &EnumDeclaration) -> Result<ASTNode, EnumExpansionError> {
|
|
// Generate StaticBoxDeclaration with:
|
|
// - Constructor methods for each variant (Ok, Err, Some, None, etc.)
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
pub enum EnumExpansionError {
|
|
DuplicateVariant(String),
|
|
DuplicateField(String),
|
|
InvalidTypeName(String),
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
// Unit tests for enum expansion logic
|
|
// Test generated AST structure
|
|
// Test error cases (duplicate variants, invalid names, etc.)
|
|
}
|
|
```
|
|
|
|
**Integration**:
|
|
```rust
|
|
// src/macro/engine.rs (MODIFY)
|
|
use crate::macro::enum_expander;
|
|
|
|
impl MacroEngine {
|
|
pub fn expand(&mut self, ast: &ASTNode) -> (ASTNode, Vec<Patch>) {
|
|
match ast {
|
|
ASTNode::EnumDeclaration { .. } => {
|
|
// Expand enum and replace node
|
|
let expanded = enum_expander::expand_enum(ast)?;
|
|
// Replace single EnumDeclaration with multiple nodes
|
|
}
|
|
// ... other macro expansions ...
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Rationale for Rust Implementation (Day 1-2)**:
|
|
- ✅ **Faster prototyping** - Rich type system, IDE support, compiler checks
|
|
- ✅ **Easier debugging** - Standard Rust tooling
|
|
- ✅ **Type-safe codegen** - Constructing AST nodes with compiler verification
|
|
- ❌ **Not dogfooding** - But acceptable for MVP
|
|
|
|
**Future Migration** (Phase 19.3+):
|
|
- Rewrite in Hakorune (`apps/macros/enum_macro.hako`)
|
|
- Demonstrate self-hosting capability
|
|
- String-based codegen (less type-safe but proves language capability)
|
|
|
|
---
|
|
|
|
### 3.5 Test Organization
|
|
|
|
**Structure**:
|
|
```
|
|
apps/tests/
|
|
└── enum/ [NEW - dedicated enum tests]
|
|
├── test_enum_basic.hako [2-variant Result-like]
|
|
├── test_enum_option.hako [2-variant Option-like]
|
|
├── test_enum_multi.hako [3+ variants]
|
|
├── test_enum_zero_field.hako [unit variants only]
|
|
├── test_enum_is_methods.hako [is_ok, is_err queries]
|
|
├── test_enum_unwrap.hako [unwrap, unwrap_or]
|
|
└── test_enum_error_cases.hako [duplicate variants, etc.]
|
|
|
|
src/parser/declarations/enum_parser.rs
|
|
└── #[cfg(test)] mod tests { ... } [parser unit tests]
|
|
|
|
src/macro/enum_expander.rs
|
|
└── #[cfg(test)] mod tests { ... } [expander unit tests]
|
|
```
|
|
|
|
**Test Separation**:
|
|
|
|
| Test Type | Location | What to Test |
|
|
|-----------|----------|--------------|
|
|
| **Parser Unit** | `src/parser/declarations/enum_parser.rs::tests` | AST construction, syntax errors, edge cases |
|
|
| **Expander Unit** | `src/macro/enum_expander.rs::tests` | Generated AST structure, expansion errors |
|
|
| **Integration** | `apps/tests/enum/*.hako` | Full pipeline: parse → expand → execute |
|
|
|
|
**Example Integration Test**:
|
|
```hakorune
|
|
// apps/tests/enum/test_enum_basic.hako
|
|
|
|
@enum Result {
|
|
Ok(value: IntegerBox)
|
|
Err(error: StringBox)
|
|
}
|
|
|
|
static box Main {
|
|
main() {
|
|
local r1
|
|
r1 = Result.Ok(42)
|
|
|
|
local r2
|
|
r2 = Result.Err("oops")
|
|
|
|
if r1.is_ok() {
|
|
print("r1 is Ok")
|
|
}
|
|
|
|
if r2.is_err() {
|
|
print("r2 is Err")
|
|
}
|
|
|
|
local val
|
|
val = r1.unwrap()
|
|
print(val) // Should print 42
|
|
|
|
return true
|
|
}
|
|
}
|
|
```
|
|
|
|
**Test Execution**:
|
|
```bash
|
|
# Run parser unit tests
|
|
cargo test --lib enum_parser::tests
|
|
|
|
# Run expander unit tests
|
|
cargo test --lib enum_expander::tests
|
|
|
|
# Run integration tests
|
|
./target/release/hako apps/tests/enum/test_enum_basic.hako
|
|
```
|
|
|
|
---
|
|
|
|
### 3.6 Documentation Placement
|
|
|
|
**User-Facing Documentation**:
|
|
```
|
|
docs/
|
|
└── reference/
|
|
└── language/
|
|
└── enum-syntax.md [NEW - user guide for @enum]
|
|
```
|
|
|
|
**Developer Documentation**:
|
|
```
|
|
docs/
|
|
└── development/
|
|
└── roadmap/phases/phase-19-enum-match/
|
|
├── README.md [EXISTS - project overview]
|
|
├── enum-module-architecture.md [THIS FILE]
|
|
└── enum-implementation-notes.md [NEW - implementation details]
|
|
```
|
|
|
|
**Generated Code Documentation**:
|
|
```
|
|
apps/lib/boxes/
|
|
├── option_generated.hako [OPTION: Generated from @enum Option]
|
|
└── result_generated.hako [OPTION: Generated from @enum Result]
|
|
|
|
# OR: Overwrite existing files
|
|
apps/lib/boxes/
|
|
├── option.hako [REPLACE: Generated by @enum]
|
|
└── result.hako [REPLACE: Generated by @enum]
|
|
```
|
|
|
|
**Decision on Generated Files**:
|
|
- **Day 1-2**: Generate to `apps/lib/boxes/option_generated.hako` (don't overwrite existing)
|
|
- **Day 3+**: Add config to allow overwriting or in-place generation
|
|
|
|
**User Guide Contents** (`enum-syntax.md`):
|
|
- Syntax reference
|
|
- Examples (Option, Result, HttpStatus)
|
|
- Generated methods documentation
|
|
- Best practices
|
|
- Comparison with manual Box implementation
|
|
|
|
---
|
|
|
|
### 3.7 Configuration Updates
|
|
|
|
**hako.toml** (MODIFY if needed):
|
|
```toml
|
|
[modules.overrides]
|
|
# If using generated files in standard library:
|
|
std.option = "apps/lib/boxes/option_generated.hako"
|
|
std.result = "apps/lib/boxes/result_generated.hako"
|
|
|
|
# Or if overwriting:
|
|
std.option = "apps/lib/boxes/option.hako"
|
|
std.result = "apps/lib/boxes/result.hako"
|
|
```
|
|
|
|
**Cargo.toml** (No changes needed):
|
|
- Pure code feature, no new dependencies
|
|
- Uses existing AST/parser/macro infrastructure
|
|
|
|
**Environment Variables** (for testing/debugging):
|
|
```bash
|
|
# Enable enum macro system
|
|
export NYASH_ENUM_ENABLE=1
|
|
|
|
# Trace enum expansion
|
|
export NYASH_ENUM_TRACE=1
|
|
|
|
# Dump generated AST
|
|
export NYASH_ENUM_DUMP_AST=1
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Dependency Management
|
|
|
|
### 4.1 Dependency Graph
|
|
|
|
```
|
|
Tokenizer (existing)
|
|
↓ Token stream
|
|
enum_parser.rs (NEW)
|
|
↓ EnumDeclaration AST
|
|
ast.rs (MODIFY - add structs)
|
|
↓ AST nodes
|
|
enum_expander.rs (NEW)
|
|
↓ BoxDeclaration + StaticBoxDeclaration
|
|
macro/engine.rs (MODIFY - call expander)
|
|
↓ Expanded AST
|
|
parser/mod.rs (integration)
|
|
↓ Complete program AST
|
|
MIR builder (existing)
|
|
↓ MIR
|
|
Backend (existing)
|
|
```
|
|
|
|
### 4.2 Module Dependencies
|
|
|
|
```rust
|
|
// src/parser/declarations/enum_parser.rs
|
|
use crate::ast::{ASTNode, EnumDeclaration, EnumVariant, EnumField, Span};
|
|
use crate::parser::{NyashParser, ParseError};
|
|
use crate::parser::common::ParserUtils;
|
|
use crate::tokenizer::TokenType;
|
|
|
|
// src/macro/enum_expander.rs
|
|
use crate::ast::{ASTNode, EnumDeclaration, EnumVariant, Span};
|
|
use crate::ast::{BoxDeclaration, StaticBoxDeclaration}; // existing
|
|
use std::collections::HashMap;
|
|
|
|
// src/macro/engine.rs (modification)
|
|
use crate::macro::enum_expander;
|
|
```
|
|
|
|
**Circular Dependency Check**: ✅ None
|
|
- Tokenizer → Parser → AST → Expander → Engine (linear dependency chain)
|
|
- No cycles detected
|
|
|
|
---
|
|
|
|
## 5. Naming Conventions
|
|
|
|
### 5.1 File Names
|
|
| Type | Convention | Examples |
|
|
|------|------------|----------|
|
|
| Rust modules | `snake_case.rs` | `enum_parser.rs`, `enum_expander.rs` |
|
|
| Hakorune tests | `test_*.hako` | `test_enum_basic.hako`, `test_enum_option.hako` |
|
|
| Generated boxes | `*_generated.hako` or `*.hako` | `option_generated.hako` or `option.hako` |
|
|
| Documentation | `kebab-case.md` | `enum-syntax.md`, `enum-implementation-notes.md` |
|
|
|
|
### 5.2 Function Names
|
|
| Type | Convention | Examples |
|
|
|------|------------|----------|
|
|
| Parser | `parse_*` | `parse_enum_declaration()`, `parse_enum_variant()` |
|
|
| Expander | `expand_*`, `generate_*` | `expand_enum()`, `generate_box_declaration()` |
|
|
| Helpers | `verb_noun` | `validate_variant_name()`, `collect_fields()` |
|
|
|
|
### 5.3 Type Names
|
|
| Type | Convention | Examples |
|
|
|------|------------|----------|
|
|
| AST structs | `CamelCase` | `EnumDeclaration`, `EnumVariant`, `EnumField` |
|
|
| Error types | `*Error` | `EnumExpansionError`, `ParseError` |
|
|
| Config types | `*Config` | `EnumGeneratorConfig` (future) |
|
|
|
|
### 5.4 Generated Code
|
|
| Item | Naming Pattern | Examples |
|
|
|------|----------------|----------|
|
|
| Input syntax | `@enum Name { ... }` | `@enum Result`, `@enum Option` |
|
|
| Generated box | `{Name}Box<T>` | `ResultBox<T, E>`, `OptionBox<T>` |
|
|
| Static box | `{Name}` | `Result`, `Option` |
|
|
| Variant field | `{variant}_{field}` | `ok_value`, `err_error`, `some_value` |
|
|
| Query method | `is_{variant}()` | `is_ok()`, `is_err()`, `is_some()`, `is_none()` |
|
|
|
|
---
|
|
|
|
## 6. File Creation Checklist
|
|
|
|
### 6.1 Files to CREATE
|
|
|
|
**Parser**:
|
|
- [ ] `src/parser/declarations/enum_parser.rs` (NEW)
|
|
- `parse_enum_declaration()` - main parser function
|
|
- `parse_enum_variant()` - variant parser
|
|
- `parse_enum_field()` - field parser
|
|
- Unit tests in `#[cfg(test)] mod tests`
|
|
|
|
**Macro Expander**:
|
|
- [ ] `src/macro/enum_expander.rs` (NEW)
|
|
- `expand_enum()` - main expander function
|
|
- `generate_box_declaration()` - box AST generator
|
|
- `generate_static_box()` - static box AST generator
|
|
- `EnumExpansionError` enum
|
|
- Unit tests in `#[cfg(test)] mod tests`
|
|
|
|
**Tests**:
|
|
- [ ] `apps/tests/enum/test_enum_basic.hako` (NEW) - 2-variant Result-like
|
|
- [ ] `apps/tests/enum/test_enum_option.hako` (NEW) - 2-variant Option-like
|
|
- [ ] `apps/tests/enum/test_enum_multi.hako` (NEW) - 3+ variants
|
|
|
|
**Documentation**:
|
|
- [ ] `docs/reference/language/enum-syntax.md` (NEW) - user guide
|
|
- [ ] `docs/private/roadmap/phases/phase-19-enum-match/enum-implementation-notes.md` (NEW)
|
|
|
|
### 6.2 Files to MODIFY
|
|
|
|
**AST**:
|
|
- [ ] `src/ast.rs` - Add 3 new structs:
|
|
- `EnumDeclaration` struct
|
|
- `EnumVariant` struct
|
|
- `EnumField` struct
|
|
- Add `ASTNode::EnumDeclaration` variant
|
|
|
|
**Parser Integration**:
|
|
- [ ] `src/parser/declarations/mod.rs` - Export `enum_parser` module
|
|
- [ ] `src/parser/mod.rs` - Integrate enum parsing in `parse_statement()` or equivalent
|
|
|
|
**Tokenizer** (if needed):
|
|
- [ ] `src/tokenizer/mod.rs` - Add `MacroKeyword` token (or reuse existing)
|
|
|
|
**Macro Engine**:
|
|
- [ ] `src/macro/mod.rs` - Export `enum_expander` module
|
|
- [ ] `src/macro/engine.rs` - Call `enum_expander::expand_enum()` for `EnumDeclaration` nodes
|
|
|
|
**Configuration** (optional):
|
|
- [ ] `hako.toml` - Add generated module overrides (if using generated stdlib)
|
|
|
|
### 6.3 Files to READ (Understand Before Coding)
|
|
|
|
**Required Reading**:
|
|
- [ ] `src/parser/mod.rs` - Understand current parser structure
|
|
- [ ] `src/parser/declarations/box_definition.rs` - Learn box parsing patterns
|
|
- [ ] `src/ast.rs` - Understand AST node patterns
|
|
- [ ] `src/macro/engine.rs` - Understand macro expansion flow
|
|
|
|
**Reference Reading**:
|
|
- [ ] `apps/lib/boxes/option.hako` - Current manual Option implementation
|
|
- [ ] `apps/lib/boxes/result.hako` - Current manual Result implementation
|
|
- [ ] `docs/private/roadmap/phases/phase-19-enum-match/README.md` - Project overview
|
|
|
|
---
|
|
|
|
## 7. Directory Tree (Before/After)
|
|
|
|
### 7.1 BEFORE (Current State)
|
|
|
|
```
|
|
hakorune-selfhost/
|
|
├── src/
|
|
│ ├── ast.rs [16,868 bytes]
|
|
│ ├── parser/
|
|
│ │ ├── mod.rs [15,287 bytes]
|
|
│ │ ├── declarations/
|
|
│ │ │ ├── mod.rs
|
|
│ │ │ ├── box_definition.rs
|
|
│ │ │ ├── static_box.rs
|
|
│ │ │ └── ... (6 files total)
|
|
│ │ └── ... (11 modules)
|
|
│ ├── macro/
|
|
│ │ ├── mod.rs
|
|
│ │ ├── engine.rs
|
|
│ │ ├── macro_box.rs
|
|
│ │ └── ... (6 files total)
|
|
│ └── ... (32 modules)
|
|
├── apps/
|
|
│ ├── lib/boxes/
|
|
│ │ ├── option.hako [1,755 bytes - manual impl]
|
|
│ │ ├── result.hako [1,874 bytes - manual impl]
|
|
│ │ └── ... (6 files)
|
|
│ ├── tests/
|
|
│ │ ├── (48 subdirectories)
|
|
│ │ └── ... (many test files)
|
|
│ └── ... (38 directories)
|
|
├── docs/
|
|
│ ├── reference/language/
|
|
│ │ ├── quick-reference.md
|
|
│ │ ├── LANGUAGE_REFERENCE_2025.md
|
|
│ │ └── ... (other docs)
|
|
│ └── development/roadmap/phases/phase-19-enum-match/
|
|
│ └── README.md [13,478 bytes]
|
|
└── ...
|
|
```
|
|
|
|
### 7.2 AFTER (With @enum Implementation)
|
|
|
|
```
|
|
hakorune-selfhost/
|
|
├── src/
|
|
│ ├── ast.rs [+80 lines - 3 new structs]
|
|
│ ├── parser/
|
|
│ │ ├── mod.rs [+10 lines - integration]
|
|
│ │ ├── declarations/
|
|
│ │ │ ├── mod.rs [+1 line - export]
|
|
│ │ │ ├── enum_parser.rs [NEW ~200 lines] ⭐
|
|
│ │ │ └── ... (existing files)
|
|
│ │ └── ... (existing modules)
|
|
│ ├── macro/
|
|
│ │ ├── mod.rs [+1 line - export]
|
|
│ │ ├── engine.rs [+15 lines - enum expansion]
|
|
│ │ ├── enum_expander.rs [NEW ~300 lines] ⭐
|
|
│ │ └── ... (existing files)
|
|
│ └── ... (existing modules)
|
|
├── apps/
|
|
│ ├── lib/boxes/
|
|
│ │ ├── option.hako [unchanged OR regenerated]
|
|
│ │ ├── result.hako [unchanged OR regenerated]
|
|
│ │ ├── option_generated.hako [OPTION: NEW ~150 lines]
|
|
│ │ ├── result_generated.hako [OPTION: NEW ~180 lines]
|
|
│ │ └── ... (existing files)
|
|
│ ├── tests/
|
|
│ │ ├── enum/ [NEW directory] ⭐
|
|
│ │ │ ├── test_enum_basic.hako [NEW ~40 lines]
|
|
│ │ │ ├── test_enum_option.hako [NEW ~30 lines]
|
|
│ │ │ └── test_enum_multi.hako [NEW ~50 lines]
|
|
│ │ └── ... (existing subdirectories)
|
|
│ └── ... (existing directories)
|
|
├── docs/
|
|
│ ├── reference/language/
|
|
│ │ ├── enum-syntax.md [NEW ~300 lines] ⭐
|
|
│ │ └── ... (existing docs)
|
|
│ └── development/roadmap/phases/phase-19-enum-match/
|
|
│ ├── README.md [existing]
|
|
│ ├── enum-module-architecture.md [THIS FILE] ⭐
|
|
│ └── enum-implementation-notes.md [NEW ~200 lines] ⭐
|
|
└── ...
|
|
```
|
|
|
|
**Summary of Changes**:
|
|
- **NEW files**: 8 files (⭐ marked above)
|
|
- **Modified files**: 5 files (`ast.rs`, `parser/mod.rs`, `declarations/mod.rs`, `macro/mod.rs`, `macro/engine.rs`)
|
|
- **Total new code**: ~1,500 lines (parser 200 + expander 300 + tests 120 + docs 500 + AST 80 + integration 50 + notes 200)
|
|
|
|
---
|
|
|
|
## 8. Implementation Phases (Day-by-Day Breakdown)
|
|
|
|
### Day 1: Parser + AST (Foundation)
|
|
|
|
**Goal**: Parse `@enum` syntax into AST nodes
|
|
|
|
**Tasks**:
|
|
1. ✅ Read reference files (parser/mod.rs, box_definition.rs, ast.rs)
|
|
2. ✅ Add AST structs to `src/ast.rs` (EnumDeclaration, EnumVariant, EnumField)
|
|
3. ✅ Create `src/parser/declarations/enum_parser.rs`
|
|
4. ✅ Implement `parse_enum_declaration()`
|
|
5. ✅ Add tokenizer support for `@enum` (if needed)
|
|
6. ✅ Write parser unit tests
|
|
7. ✅ Integration: Export from `declarations/mod.rs`, integrate in `parser/mod.rs`
|
|
|
|
**Success Criteria**:
|
|
- Parser can parse: `@enum Result { Ok(value: T), Err(error: E) }`
|
|
- Parser unit tests pass
|
|
- AST nodes correctly constructed
|
|
|
|
**Estimated Lines**: ~300 lines (200 parser + 80 AST + 20 integration)
|
|
|
|
### Day 2: Macro Expander (Code Generation)
|
|
|
|
**Goal**: Transform `EnumDeclaration` AST into `BoxDeclaration` + `StaticBoxDeclaration`
|
|
|
|
**Tasks**:
|
|
1. ✅ Create `src/macro/enum_expander.rs`
|
|
2. ✅ Implement `expand_enum()` - main expander
|
|
3. ✅ Implement `generate_box_declaration()` - box with variant field
|
|
4. ✅ Implement `generate_static_box()` - constructor methods
|
|
5. ✅ Write expander unit tests (check generated AST structure)
|
|
6. ✅ Integration: Call from `macro/engine.rs`
|
|
|
|
**Success Criteria**:
|
|
- Expander generates correct Box AST (variant field + all variant fields)
|
|
- Expander generates correct StaticBox AST (constructor methods)
|
|
- Expander unit tests pass
|
|
- Generated AST can be converted to MIR (existing pipeline)
|
|
|
|
**Estimated Lines**: ~350 lines (300 expander + 50 integration/tests)
|
|
|
|
### Day 3: Integration Tests + Documentation
|
|
|
|
**Goal**: Full pipeline working, documented, tested
|
|
|
|
**Tasks**:
|
|
1. ✅ Create `apps/tests/enum/` directory
|
|
2. ✅ Write `test_enum_basic.hako` (Result-like)
|
|
3. ✅ Write `test_enum_option.hako` (Option-like)
|
|
4. ✅ Write `test_enum_multi.hako` (3+ variants)
|
|
5. ✅ Run integration tests (parse → expand → execute)
|
|
6. ✅ Write `docs/reference/language/enum-syntax.md` (user guide)
|
|
7. ✅ Write `enum-implementation-notes.md` (developer notes)
|
|
|
|
**Success Criteria**:
|
|
- All integration tests pass
|
|
- Documentation complete and accurate
|
|
- Generated code executes correctly
|
|
|
|
**Estimated Lines**: ~620 lines (120 tests + 500 docs)
|
|
|
|
### Day 4: Polish + Edge Cases (Buffer)
|
|
|
|
**Goal**: Handle edge cases, improve error messages, optimize
|
|
|
|
**Tasks**:
|
|
- Error handling: duplicate variants, invalid names, circular dependencies
|
|
- Error messages: clear, actionable, with suggestions
|
|
- Edge cases: zero-field variants, single variant, generic types
|
|
- Performance: AST generation optimization
|
|
- Code review: refactor, simplify, document
|
|
|
|
**Success Criteria**:
|
|
- All error cases handled gracefully
|
|
- Error messages are helpful
|
|
- Code is clean and maintainable
|
|
|
|
---
|
|
|
|
## 9. Testing Strategy
|
|
|
|
### 9.1 Unit Tests (Rust)
|
|
|
|
**Parser Unit Tests** (`src/parser/declarations/enum_parser.rs::tests`):
|
|
```rust
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_parse_two_variant_enum() {
|
|
// Parse: @enum Result { Ok(value: T), Err(error: E) }
|
|
// Assert: Correct AST structure
|
|
}
|
|
|
|
#[test]
|
|
fn test_parse_zero_field_variant() {
|
|
// Parse: @enum Option { Some(value: T), None() }
|
|
// Assert: None has empty fields vec
|
|
}
|
|
|
|
#[test]
|
|
fn test_duplicate_variant_error() {
|
|
// Parse: @enum Foo { A(), A() }
|
|
// Assert: ParseError with duplicate message
|
|
}
|
|
}
|
|
```
|
|
|
|
**Expander Unit Tests** (`src/macro/enum_expander.rs::tests`):
|
|
```rust
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_expand_result_enum() {
|
|
// Input: EnumDeclaration for Result
|
|
// Assert: Generated BoxDeclaration has variant field
|
|
// Assert: Generated StaticBoxDeclaration has Ok/Err methods
|
|
}
|
|
|
|
#[test]
|
|
fn test_generated_box_fields() {
|
|
// Assert: ok_value and err_error fields exist
|
|
// Assert: Fields are nullable for variants not using them
|
|
}
|
|
}
|
|
```
|
|
|
|
### 9.2 Integration Tests (Hakorune)
|
|
|
|
**Basic Functionality**:
|
|
```hakorune
|
|
// apps/tests/enum/test_enum_basic.hako
|
|
|
|
@enum Result {
|
|
Ok(value: IntegerBox)
|
|
Err(error: StringBox)
|
|
}
|
|
|
|
static box Main {
|
|
main() {
|
|
local r1
|
|
r1 = Result.Ok(42)
|
|
|
|
if r1.is_ok() {
|
|
print("PASS: is_ok works")
|
|
} else {
|
|
print("FAIL: is_ok broken")
|
|
return false
|
|
}
|
|
|
|
local val
|
|
val = r1.unwrap()
|
|
if val == 42 {
|
|
print("PASS: unwrap works")
|
|
} else {
|
|
print("FAIL: unwrap broken")
|
|
return false
|
|
}
|
|
|
|
return true
|
|
}
|
|
}
|
|
```
|
|
|
|
**Test Execution**:
|
|
```bash
|
|
# Run all enum tests
|
|
for test in apps/tests/enum/*.hako; do
|
|
echo "Running $test..."
|
|
./target/release/hako "$test" || echo "FAILED: $test"
|
|
done
|
|
```
|
|
|
|
### 9.3 Smoke Tests
|
|
|
|
**Add to Existing Smoke Test Suite**:
|
|
```bash
|
|
# tools/smokes/v2/profiles/quick/core/enum_basic_vm.sh
|
|
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
source "$SCRIPT_DIR/../../../lib/test_runner.sh"
|
|
|
|
TEST_FILE="apps/tests/enum/test_enum_basic.hako"
|
|
EXPECTED_OUTPUT="PASS: is_ok works
|
|
PASS: unwrap works"
|
|
|
|
run_test_vm "$TEST_FILE" "$EXPECTED_OUTPUT"
|
|
```
|
|
|
|
---
|
|
|
|
## 10. Error Handling Strategy
|
|
|
|
### 10.1 Parser Errors
|
|
|
|
**Error Types**:
|
|
```rust
|
|
pub enum ParseError {
|
|
// Existing errors...
|
|
|
|
// Enum-specific errors
|
|
DuplicateEnumVariant {
|
|
enum_name: String,
|
|
variant_name: String,
|
|
line: usize,
|
|
},
|
|
|
|
DuplicateEnumField {
|
|
variant_name: String,
|
|
field_name: String,
|
|
line: usize,
|
|
},
|
|
|
|
InvalidEnumName {
|
|
name: String,
|
|
reason: String,
|
|
line: usize,
|
|
},
|
|
|
|
InvalidVariantName {
|
|
name: String,
|
|
reason: String,
|
|
line: usize,
|
|
},
|
|
}
|
|
```
|
|
|
|
**Error Messages** (User-Friendly):
|
|
```
|
|
Error: Duplicate variant 'Ok' in enum 'Result' at line 3
|
|
|
|
|
3 | Ok(value: T)
|
|
| ^^ variant already declared at line 2
|
|
|
|
|
Help: Each variant in an enum must have a unique name
|
|
```
|
|
|
|
### 10.2 Expander Errors
|
|
|
|
**Error Types**:
|
|
```rust
|
|
pub enum EnumExpansionError {
|
|
DuplicateVariant(String),
|
|
DuplicateField(String),
|
|
InvalidTypeName(String),
|
|
CircularDependency(String),
|
|
ReservedMethodName { variant: String, method: String },
|
|
}
|
|
```
|
|
|
|
**Example**:
|
|
```
|
|
Error: Reserved method name 'is_ok' conflicts with generated query method
|
|
|
|
|
| @enum Result { Ok(is_ok: IntegerBox) }
|
|
| ^^^^^ field name conflicts with auto-generated method
|
|
|
|
|
Help: Rename field to avoid 'is_*' prefix (e.g., 'ok_status', 'ok_flag')
|
|
```
|
|
|
|
---
|
|
|
|
## 11. Performance Considerations
|
|
|
|
### 11.1 Parser Performance
|
|
- **Minimal lookahead**: Use `peek()` instead of backtracking
|
|
- **Single-pass parsing**: No multiple traversals of token stream
|
|
- **Reuse existing patterns**: Follow `box_definition.rs` structure
|
|
|
|
### 11.2 Expander Performance
|
|
- **Lazy AST construction**: Build nodes only when needed
|
|
- **Avoid cloning**: Use references where possible, clone only final AST
|
|
- **Inline small functions**: Mark helpers as `#[inline]`
|
|
|
|
### 11.3 Generated Code Performance
|
|
- **Minimal runtime overhead**: Generated boxes behave like hand-written boxes
|
|
- **No macro runtime**: All expansion at compile-time (parse-time)
|
|
- **Efficient method calls**: `is_ok()` checks string comparison (variant == "Ok")
|
|
|
|
**Future Optimizations** (Post-Day 1):
|
|
- Enum variant as integer tag (faster than string comparison)
|
|
- Packed field storage (avoid nullable fields for unused variants)
|
|
- Static dispatch for known variant types
|
|
|
|
---
|
|
|
|
## 12. Future Expansion Points
|
|
|
|
### 12.1 Match Expression Integration (Phase 19.2)
|
|
|
|
**Current**: `if result.is_ok() { ... }`
|
|
**Future**: `match result { Ok(v) => ..., Err(e) => ... }`
|
|
|
|
**Architecture Hook**:
|
|
```rust
|
|
// src/parser/expressions.rs (future)
|
|
fn parse_match_expression(&mut self) -> Result<ASTNode, ParseError> {
|
|
// match <scrutinee> { <pattern> => <expr>, ... }
|
|
}
|
|
|
|
// src/macro/enum_expander.rs (future)
|
|
fn expand_match_expression(expr: MatchExpression, enum_decl: &EnumDeclaration)
|
|
-> Result<ASTNode, EnumExpansionError> {
|
|
// Desugar to if-else tree with is_* checks
|
|
}
|
|
```
|
|
|
|
### 12.2 Hakorune Self-Hosting (Phase 19.3)
|
|
|
|
**Current**: Rust implementation (`src/macro/enum_expander.rs`)
|
|
**Future**: Hakorune implementation (`apps/macros/enum_macro.hako`)
|
|
|
|
**Migration Path**:
|
|
1. Keep Rust implementation as reference
|
|
2. Implement in Hakorune (string-based codegen)
|
|
3. Test both implementations in parallel
|
|
4. Switch default to Hakorune version
|
|
5. Archive Rust version for fallback
|
|
|
|
### 12.3 Advanced Features (Phase 19.4+)
|
|
|
|
- **Enum with methods**: `@enum Result { ... } impl { map() { ... } }`
|
|
- **Enum with associated values**: `@enum Color { Red = 1, Green = 2 }`
|
|
- **Enum exhaustiveness checking**: Compile-time warning for missing match arms
|
|
- **Enum pattern matching destructuring**: `match r { Ok(value) if value > 0 => ... }`
|
|
|
|
---
|
|
|
|
## 13. Summary Table
|
|
|
|
| Aspect | Decision | File Location |
|
|
|--------|----------|---------------|
|
|
| **Parser** | New module in existing directory | `src/parser/declarations/enum_parser.rs` |
|
|
| **AST** | Add to existing monolithic file | `src/ast.rs` (3 new structs) |
|
|
| **Expander** | New module in macro system | `src/macro/enum_expander.rs` |
|
|
| **Tests** | Dedicated subdirectory | `apps/tests/enum/` |
|
|
| **Docs** | User guide + developer notes | `docs/reference/language/enum-syntax.md` + `docs/development/.../enum-implementation-notes.md` |
|
|
| **Implementation** | Rust (Day 1-2) → Hakorune (Future) | Rust for MVP, migrate later |
|
|
| **Generated Code** | Option/Result standard library | `apps/lib/boxes/option.hako`, `result.hako` |
|
|
| **Integration** | Minimal modifications | 5 files modified |
|
|
| **Timeline** | 3 days core + 1 day buffer | Day 1: Parser, Day 2: Expander, Day 3: Tests+Docs |
|
|
|
|
---
|
|
|
|
## 14. Rationale Summary
|
|
|
|
**Why this architecture?**
|
|
|
|
1. **Follows Existing Patterns**:
|
|
- Parser in `declarations/` (like `box_definition.rs`)
|
|
- AST in monolithic `ast.rs` (current practice)
|
|
- Macro in `macro/` directory (existing system)
|
|
|
|
2. **Separation of Concerns**:
|
|
- Parser: Token → AST (single responsibility)
|
|
- Expander: AST → AST (single responsibility)
|
|
- Engine: Orchestration (existing)
|
|
|
|
3. **Testability**:
|
|
- Parser unit tests (AST correctness)
|
|
- Expander unit tests (generation correctness)
|
|
- Integration tests (full pipeline)
|
|
|
|
4. **Maintainability**:
|
|
- Clear file structure (predictable locations)
|
|
- Minimal dependencies (linear flow)
|
|
- Comprehensive documentation
|
|
|
|
5. **Future-Proof**:
|
|
- Easy to extend (match expressions, advanced features)
|
|
- Easy to migrate (Rust → Hakorune)
|
|
- Easy to optimize (clear boundaries)
|
|
|
|
---
|
|
|
|
## 15. Risks and Mitigations
|
|
|
|
| Risk | Mitigation |
|
|
|------|------------|
|
|
| **Parser complexity** | Follow existing `box_definition.rs` patterns |
|
|
| **AST size growth** | Monitor file size, split if exceeds 25KB |
|
|
| **Expander bugs** | Comprehensive unit tests for each generated AST node |
|
|
| **Performance regression** | Benchmark generated code vs hand-written |
|
|
| **Breaking changes** | Keep manual implementations working alongside generated |
|
|
| **Documentation drift** | Update docs in same PR as implementation |
|
|
|
|
---
|
|
|
|
## 16. Conclusion
|
|
|
|
This architecture provides a **clean, maintainable, production-ready** foundation for @enum implementation. It follows Hakorune conventions, integrates naturally with existing code, and provides clear extension points for future features.
|
|
|
|
**Next Steps**:
|
|
1. ✅ Review this document with team
|
|
2. ✅ Create implementation checklist from "File Creation Checklist"
|
|
3. ✅ Begin Day 1 implementation (Parser + AST)
|
|
4. ✅ Iterate based on feedback
|
|
|
|
**Questions or Concerns?**
|
|
- Open issue in Phase 19 tracker
|
|
- Discuss in development channel
|
|
- Update this document as architecture evolves
|
|
|
|
---
|
|
|
|
**Document Version**: 1.0
|
|
**Last Updated**: 2025-10-08
|
|
**Author**: Claude Code (Architecture Analysis Agent)
|
|
**Status**: Ready for Implementation ✅
|