501 lines
15 KiB
Markdown
501 lines
15 KiB
Markdown
# Phase 20.23 — Arc/RefCell in Hakorune（メモリ所有権管理）
|
||
|
||
**期間**: 2-3週間
|
||
**ステータス**: 計画更新（20.21 RCテーブル導入と整合）
|
||
**前提条件**: Phase 20.21（Manual Memory Management）完了
|
||
|
||
---
|
||
|
||
## 🎯 Executive Summary
|
||
|
||
**Critical Strategic Decision**: Arc/RefCell を **C ではなく Hakorune** で実装する。
|
||
|
||
### 理由（Why Hakorune, Not C）
|
||
|
||
| 観点 | C実装 | Hakorune実装 |
|
||
|------|-------|-------------|
|
||
| **開発期間** | 2-3ヶ月 | **2-3週間** ⚡ |
|
||
| **複雑性** | ❌ 高（低レベル並行制御） | ✅ 低（ポリシー記述） |
|
||
| **リスク** | ❌ 高（メモリ安全性） | ✅ 低（境界明確） |
|
||
| **保守性** | ❌ C コード保守 | ✅ Hakorune コード保守 |
|
||
| **テスト容易性** | ⚠️ 中 | ✅ 高（スクリプトレベル） |
|
||
|
||
### 実装パターン（GcBox を踏襲）
|
||
|
||
**既存の成功事例**: `lang/src/runtime/gc/gc_box.hako`
|
||
|
||
```hakorune
|
||
// Policy plane (Hakorune) - ポリシー・ロジック層
|
||
static box GcBox {
|
||
stats() {
|
||
return call("env.gc.stats/0") // Data plane呼び出し
|
||
}
|
||
|
||
collect() {
|
||
call("env.gc.collect/0") // Data plane呼び出し
|
||
}
|
||
}
|
||
|
||
// Data plane (C/Rust) - 実際のメモリ操作
|
||
// Rust: fn hako_gc_stats() -> String { /* 実メモリ走査 */ }
|
||
// Rust: fn hako_gc_collect() { /* 実GC実行 */ }
|
||
```
|
||
|
||
**このパターンを Arc/RefCell に適用**:
|
||
- **Policy plane**: ArcBox/RefCellBox (Hakorune) - retain/release ロジック
|
||
- **Data plane**: Phase 20.21 の `hako_alloc/hako_free` (C) - 実メモリ操作
|
||
|
||
### Time Savings（時間短縮効果）
|
||
|
||
```
|
||
Before (C implementation plan):
|
||
Phase 20.23: Arc/RefCell (C) 2-3 months ❌
|
||
|
||
After (Hakorune implementation):
|
||
Phase 20.23: Arc/RefCell (Hako) 2-3 weeks ✅
|
||
|
||
Savings: ~2.5 months = 10-11 weeks ⚡
|
||
```
|
||
|
||
---
|
||
|
||
## 🏗️ アーキテクチャ設計
|
||
|
||
### Policy-Data Plane 分離（現状の足場）
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────┐
|
||
│ Policy Plane (Hakorune) │
|
||
│ │
|
||
│ ┌────────────┐ ┌──────────────┐ │
|
||
│ │ ArcBox │ │ RefCellBox │ │
|
||
│ │ │ │ │ │
|
||
│ │ - retain() │ │ - borrow() │ │
|
||
│ │ - release()│ │ - borrow_mut │ │
|
||
│ │ - clone() │ │ - try_borrow │ │
|
||
│ └──────┬─────┘ └──────┬───────┘ │
|
||
│ │ │ │
|
||
└────────┼────────────────┼────────────────────────┘
|
||
│ │
|
||
│ call("env.arc.retain/1", ptr)
|
||
│ call("env.arc.release/1", ptr)
|
||
│ call("env.refcell.borrow/1", ptr)
|
||
▼ ▼
|
||
┌─────────────────────────────────────────────────┐
|
||
│ Data Plane (C - Phase 20.21 既存API) │
|
||
│ │
|
||
│ void* hako_alloc(size_t size) │
|
||
│ void hako_free(void* ptr) │
|
||
│ │
|
||
│ // Atomic operations (C11 標準) │
|
||
│ atomic_fetch_add(&ref_count, 1) │
|
||
│ atomic_fetch_sub(&ref_count, 1) │
|
||
└─────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### ArcBox API 設計（Hakorune Layer）
|
||
|
||
```hakorune
|
||
// lang/src/runtime/memory/arc_box.hako
|
||
|
||
static box ArcBox {
|
||
// Opaque handle to C memory
|
||
ptr: IntegerBox // 64-bit pointer/handle
|
||
ref_count: IntegerBox // Local cache (optional)
|
||
|
||
// Constructor: Allocate and initialize ref_count=1
|
||
birth(data_ptr) {
|
||
me.ptr = data_ptr
|
||
me.ref_count = 1
|
||
call("env.arc.init/2", me.ptr, 1) // C side初期化
|
||
}
|
||
|
||
// Increment ref count (atomic)
|
||
retain() {
|
||
local new_count = call("env.arc.retain/1", me.ptr)
|
||
me.ref_count = new_count
|
||
return new_count
|
||
}
|
||
|
||
// Decrement ref count, free if 0 (atomic)
|
||
release() {
|
||
local new_count = call("env.arc.release/1", me.ptr)
|
||
me.ref_count = new_count
|
||
|
||
if (new_count == 0) {
|
||
call("env.arc.free/1", me.ptr) // C側でhako_free呼び出し
|
||
me.ptr = 0 // Dangling pointer防止
|
||
}
|
||
|
||
return new_count
|
||
}
|
||
|
||
// Clone (increment ref_count and return new ArcBox)
|
||
clone() {
|
||
me.retain()
|
||
local new_arc = new ArcBox()
|
||
new_arc.ptr = me.ptr
|
||
new_arc.ref_count = me.ref_count
|
||
return new_arc
|
||
}
|
||
|
||
// Get raw pointer (unsafe, for C interop)
|
||
as_ptr() {
|
||
return me.ptr
|
||
}
|
||
}
|
||
```
|
||
|
||
### RefCellBox API 設計（Hakorune Layer）
|
||
|
||
```hakorune
|
||
// lang/src/runtime/memory/refcell_box.hako
|
||
|
||
static box RefCellBox {
|
||
ptr: IntegerBox // Opaque handle
|
||
borrow_state: IntegerBox // 0=free, >0=shared, -1=mutable
|
||
|
||
birth(data_ptr) {
|
||
me.ptr = data_ptr
|
||
me.borrow_state = 0 // Initially free
|
||
call("env.refcell.init/2", me.ptr, 0)
|
||
}
|
||
|
||
// Shared borrow (multiple readers allowed)
|
||
borrow() {
|
||
local state = call("env.refcell.try_borrow/1", me.ptr)
|
||
|
||
if (state < 0) {
|
||
// Already mutably borrowed - panic
|
||
call("env.panic/1", "RefCellBox: already mutably borrowed")
|
||
}
|
||
|
||
me.borrow_state = state + 1 // Increment reader count
|
||
return me.ptr // Return borrowed reference
|
||
}
|
||
|
||
// Mutable borrow (exclusive)
|
||
borrow_mut() {
|
||
local state = call("env.refcell.try_borrow_mut/1", me.ptr)
|
||
|
||
if (state != 0) {
|
||
// Already borrowed - panic
|
||
call("env.panic/1", "RefCellBox: already borrowed")
|
||
}
|
||
|
||
me.borrow_state = -1 // Mark as mutably borrowed
|
||
return me.ptr
|
||
}
|
||
|
||
// Release borrow (caller must specify type)
|
||
release_borrow(is_mutable) {
|
||
if (is_mutable == 1) {
|
||
call("env.refcell.release_mut/1", me.ptr)
|
||
me.borrow_state = 0
|
||
} else {
|
||
call("env.refcell.release_shared/1", me.ptr)
|
||
me.borrow_state = me.borrow_state - 1
|
||
}
|
||
}
|
||
|
||
// Try operations (non-panicking)
|
||
try_borrow() {
|
||
local state = call("env.refcell.try_borrow/1", me.ptr)
|
||
if (state < 0) {
|
||
return null // Failed
|
||
}
|
||
me.borrow_state = state + 1
|
||
return me.ptr // Success
|
||
}
|
||
|
||
try_borrow_mut() {
|
||
local state = call("env.refcell.try_borrow_mut/1", me.ptr)
|
||
if (state != 0) {
|
||
return null // Failed
|
||
}
|
||
me.borrow_state = -1
|
||
return me.ptr // Success
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 C Side Minimal API（Data Plane）
|
||
|
||
Phase 20.21 既存APIに以下を追加（100行程度）:
|
||
（補足）現状、`include/hako_mem.h` と `src/abi/c/mem_libc.c` は RC テーブルへ委譲済み：
|
||
- `src/abi/c/handle_registry.{h,c}` — ハンドルの参照カウントをC11 atomicで管理
|
||
- `hako_retain/hako_release` は未知/二重解放で Fail‑Fast（安定メッセージ）
|
||
この上に Arc/RefCell の data‑plane API を薄く足す。
|
||
|
||
```c
|
||
// lang/src/runtime/memory/arc.c (NEW, ~50 lines)
|
||
|
||
#include <stdatomic.h>
|
||
#include <stdint.h>
|
||
#include "hakmem.h" // Phase 20.21
|
||
|
||
// Arc metadata (prepended to allocation)
|
||
typedef struct {
|
||
atomic_uint_fast64_t ref_count;
|
||
size_t size;
|
||
} ArcHeader;
|
||
|
||
// Initialize Arc (called from ArcBox.birth)
|
||
void hako_arc_init(void* ptr, uint64_t initial_count) {
|
||
ArcHeader* hdr = (ArcHeader*)ptr;
|
||
atomic_init(&hdr->ref_count, initial_count);
|
||
}
|
||
|
||
// Atomic retain (returns new count)
|
||
uint64_t hako_arc_retain(void* ptr) {
|
||
ArcHeader* hdr = (ArcHeader*)ptr;
|
||
return atomic_fetch_add(&hdr->ref_count, 1) + 1;
|
||
}
|
||
|
||
// Atomic release (returns new count)
|
||
uint64_t hako_arc_release(void* ptr) {
|
||
ArcHeader* hdr = (ArcHeader*)ptr;
|
||
uint64_t old = atomic_fetch_sub(&hdr->ref_count, 1);
|
||
return old - 1; // New count
|
||
}
|
||
|
||
// Free Arc (called when ref_count=0)
|
||
void hako_arc_free(void* ptr) {
|
||
hako_free(ptr); // Phase 20.21 API
|
||
}
|
||
```
|
||
|
||
```c
|
||
// lang/src/runtime/memory/refcell.c (NEW, ~50 lines)
|
||
|
||
#include <stdatomic.h>
|
||
|
||
// RefCell metadata
|
||
typedef struct {
|
||
atomic_int_fast32_t borrow_state; // 0=free, >0=shared, -1=mutable
|
||
} RefCellHeader;
|
||
|
||
void hako_refcell_init(void* ptr, int32_t initial_state) {
|
||
RefCellHeader* hdr = (RefCellHeader*)ptr;
|
||
atomic_init(&hdr->borrow_state, initial_state);
|
||
}
|
||
|
||
// Try shared borrow (returns current state or -1 if failed)
|
||
int32_t hako_refcell_try_borrow(void* ptr) {
|
||
RefCellHeader* hdr = (RefCellHeader*)ptr;
|
||
int32_t state = atomic_load(&hdr->borrow_state);
|
||
|
||
if (state < 0) return -1; // Already mutably borrowed
|
||
|
||
atomic_fetch_add(&hdr->borrow_state, 1);
|
||
return state + 1;
|
||
}
|
||
|
||
// Try mutable borrow
|
||
int32_t hako_refcell_try_borrow_mut(void* ptr) {
|
||
RefCellHeader* hdr = (RefCellHeader*)ptr;
|
||
|
||
int32_t expected = 0;
|
||
if (atomic_compare_exchange_strong(&hdr->borrow_state, &expected, -1)) {
|
||
return 0; // Success
|
||
}
|
||
return expected; // Failed (return current state)
|
||
}
|
||
|
||
// Release shared borrow
|
||
void hako_refcell_release_shared(void* ptr) {
|
||
RefCellHeader* hdr = (RefCellHeader*)ptr;
|
||
atomic_fetch_sub(&hdr->borrow_state, 1);
|
||
}
|
||
|
||
// Release mutable borrow
|
||
void hako_refcell_release_mut(void* ptr) {
|
||
RefCellHeader* hdr = (RefCellHeader*)ptr;
|
||
atomic_store(&hdr->borrow_state, 0);
|
||
}
|
||
```
|
||
|
||
**Total C code added**: ~100 lines（Phase 20.21 の 5%）
|
||
|
||
---
|
||
|
||
## 📋 Week-by-Week Implementation Plan
|
||
|
||
### Week 1: ArcBox Implementation（RCテーブル上に実装）
|
||
|
||
**Day 1-2: C Data Plane**
|
||
- [ ] `arc.c/arc.h` 実装（50行）
|
||
- [ ] Atomic operations テスト（retain/release）
|
||
- [ ] Memory leak テスト（valgrind）
|
||
|
||
**Day 3-4: Hakorune Policy Plane**
|
||
- [ ] `lang/src/runtime/memory/arc_box.hako` 実装
|
||
- [ ] `birth/retain/release/clone` メソッド
|
||
- [ ] `call("env.arc.*")` extern binding
|
||
|
||
**Day 5: Integration Testing**
|
||
- [ ] スモークテスト追加（`tools/smokes/v2/suites/memory/arc_*.sh`）
|
||
- [ ] Multi-threaded stress test（4 threads, 10000 ops）
|
||
- [ ] Memory leak verification（valgrind clean）
|
||
|
||
### Week 2: RefCellBox Implementation
|
||
|
||
**Day 1-2: C Data Plane**
|
||
- [ ] `refcell.c/refcell.h` 実装（50行）
|
||
- [ ] Borrow state transitions テスト
|
||
- [ ] Panic case validation（double mut borrow）
|
||
|
||
**Day 3-4: Hakorune Policy Plane**
|
||
- [ ] `lang/src/runtime/memory/refcell_box.hako` 実装
|
||
- [ ] `borrow/borrow_mut/release_borrow` メソッド
|
||
- [ ] `try_borrow/try_borrow_mut` 実装
|
||
|
||
**Day 5: Integration Testing**
|
||
- [ ] Borrow check テスト（panic cases）
|
||
- [ ] Nested borrow テスト
|
||
- [ ] Shared + mutable conflict テスト
|
||
|
||
### Week 3: Integration & Documentation
|
||
|
||
**Day 1-2: Integration with Box System**
|
||
- [ ] StringBox + Arc 統合テスト
|
||
- [ ] ArrayBox + RefCell 統合テスト
|
||
- [ ] MapBox + Arc/RefCell 統合テスト
|
||
|
||
**Day 3: Performance Benchmarking**
|
||
- [ ] Arc clone overhead（vs Rust `Arc::clone`）
|
||
- [ ] RefCell borrow overhead（vs Rust `RefCell::borrow`）
|
||
- [ ] Multi-threaded throughput
|
||
|
||
**Day 4-5: Documentation**
|
||
- [ ] API documentation（ArcBox/RefCellBox）
|
||
- [ ] Migration guide（Rust Arc/RefCell → Hakorune）
|
||
- [ ] Best practices（when to use Arc vs RefCell）
|
||
- [ ] Failure modes & error handling
|
||
|
||
---
|
||
|
||
## ✅ 受け入れ基準（Acceptance Criteria）
|
||
|
||
### Functional Requirements
|
||
|
||
- [ ] **ArcBox**: retain/release/clone 正常動作
|
||
- [ ] **RefCellBox**: borrow/borrow_mut 正常動作
|
||
- [ ] **Panic cases**: 不正なborrow操作で適切にpanic
|
||
- [ ] **Memory safety**: valgrind clean（leak なし）
|
||
- [ ] **Thread safety**: 4 threads concurrent access で破損なし
|
||
|
||
### Performance Requirements
|
||
|
||
- [ ] **Arc clone**: < 50ns per operation（mimalloc程度）
|
||
- [ ] **RefCell borrow**: < 10ns per operation（ほぼゼロコスト）
|
||
- [ ] **Overhead**: Rust Arc/RefCell の 2倍以内
|
||
|
||
### Testing Requirements
|
||
|
||
- [ ] **Unit tests**: 20+ tests（C layer + Hakorune layer）
|
||
- [ ] **Integration tests**: 10+ Box system 統合テスト
|
||
- [ ] **Smoke tests**: `tools/smokes/v2/profiles/quick/` 追加
|
||
- [ ] **Stress tests**: 10000 ops, 4 threads, no crashes
|
||
|
||
### Documentation Requirements
|
||
|
||
- [ ] **API docs**: ArcBox/RefCellBox 完全仕様
|
||
- [ ] **Implementation docs**: Policy-Data plane 設計図
|
||
- [ ] **Migration guide**: Rust → Hakorune 移行手順
|
||
- [ ] **Troubleshooting**: よくあるエラーと対処法
|
||
|
||
---
|
||
|
||
## 🚨 リスク分析と軽減策
|
||
|
||
### Risk Matrix
|
||
|
||
| リスク | 確率 | 影響 | 軽減策 |
|
||
|--------|------|------|--------|
|
||
| **Atomic operations バグ** | 低 | 高 | C11 標準 atomic 使用、既存実装参照 |
|
||
| **Memory leak** | 中 | 高 | valgrind 継続検証、ref_count audit |
|
||
| **Race condition** | 低 | 高 | TSan（Thread Sanitizer）使用 |
|
||
| **Borrow check 不完全** | 中 | 中 | Rust RefCell テストケース移植 |
|
||
| **Performance 劣化** | 低 | 中 | Benchmark driven development |
|
||
|
||
### Mitigation Strategies
|
||
|
||
1. **Continuous Testing**: 毎日 valgrind + TSan 実行
|
||
2. **Reference Implementation**: Rust std::sync::Arc/RefCell をリファレンス
|
||
3. **Incremental Integration**: Week 1 で Arc 完成後、Week 2 で RefCell
|
||
4. **Early Performance Validation**: Week 1 で overhead 確認、問題なければ継続
|
||
5. **Rollback Plan**: Phase 20.21 API は変更なし、いつでも Arc/RefCell 削除可能
|
||
|
||
---
|
||
|
||
## 📊 Timeline Comparison（再掲）
|
||
|
||
### Before (C Implementation - Original Plan)
|
||
|
||
```
|
||
Phase 20.23: Arc/RefCell (C) 2-3 months ❌
|
||
├─ Week 1-4: Arc implementation (C)
|
||
├─ Week 5-8: RefCell implementation (C)
|
||
└─ Week 9-12: Integration & testing
|
||
|
||
Risk: HIGH ❌
|
||
Complexity: HIGH ❌
|
||
```
|
||
|
||
### After (Hakorune Implementation - This Plan)
|
||
|
||
```
|
||
Phase 20.23: Arc/RefCell (Hako) 2-3 weeks ✅
|
||
├─ Week 1: ArcBox (Hakorune + 50 lines C)
|
||
├─ Week 2: RefCellBox (Hakorune + 50 lines C)
|
||
└─ Week 3: Integration & docs
|
||
|
||
Risk: LOW ✅
|
||
Complexity: LOW ✅
|
||
```
|
||
|
||
**Savings**: ~2.5 months (10-11 weeks) ⚡
|
||
|
||
---
|
||
|
||
## 🔗 関連ドキュメント
|
||
|
||
- **Phase 20.21**: [Manual Memory Management](../phase-20.21/README.md) - C API 基盤
|
||
- **Phase 20.22**: [String/Array C-ABI](../phase-20.22/README.md) - Box system 連携
|
||
- **Phase 20.24**: [Parser削除](../phase-20.24/README.md) - 次のフェーズ
|
||
- **GcBox Pattern**: `lang/src/runtime/gc/gc_box.hako` - 実装パターン参照
|
||
|
||
---
|
||
|
||
## 💡 Key Insights（まとめ）
|
||
|
||
### Why This Approach Works
|
||
|
||
1. **Policy-Data Separation**: Hakorune はロジック、C は実メモリ操作
|
||
2. **Minimal C Code**: 100行のみ追加（Phase 20.21 の 5%）
|
||
3. **Proven Pattern**: GcBox で既に検証済み
|
||
4. **Low Risk**: Hakorune レイヤーなので変更容易
|
||
5. **Fast Development**: スクリプトレベル実装は C より 10倍速い
|
||
|
||
### Strategic Advantage
|
||
|
||
**User の洞察**: "Arc/RefCell はHakoruneスクリプトで実装するやつじゃない？"
|
||
|
||
→ **完全に正しい！** ✅
|
||
|
||
これにより:
|
||
- **2.5ヶ月の時間短縮** ⚡
|
||
- **リスク大幅削減**（C バグ vs Hakorune バグ）
|
||
- **保守性向上**（Hakorune コードは読みやすい）
|
||
- **テスト容易性向上**（スクリプトレベルでテスト）
|
||
|
||
---
|
||
|
||
**ステータス**: 未開始
|
||
**開始可能条件**: Phase 20.21 完了
|
||
**期間**: 2-3週間
|
||
**次フェーズ**: Phase 20.24（Parser削除）
|