a9ddb52ad4
Phase 1 完了：環境変数整理 + fprintf デバッグガード ENV変数削除（BG/HotMag系）: - core/hakmem_tiny_init.inc: HotMag ENV 削除 (~131 lines) - core/hakmem_tiny_bg_spill.c: BG spill ENV 削除 - core/tiny_refill.h: BG remote 固定値化 - core/hakmem_tiny_slow.inc: BG refs 削除 fprintf Debug Guards (#if !HAKMEM_BUILD_RELEASE): - core/hakmem_shared_pool.c: Lock stats (~18 fprintf) - core/page_arena.c: Init/Shutdown/Stats (~27 fprintf) - core/hakmem.c: SIGSEGV init message ドキュメント整理: - 328 markdown files 削除（旧レポート・重複docs） 性能確認: - Larson: 52.35M ops/s (前回52.8M、安定動作✅) - ENV整理による機能影響なし - Debug出力は一部残存（次phase で対応） 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
163 lines
8.1 KiB
Markdown
163 lines
8.1 KiB
Markdown
# HAKMEM Design Flaws - Quick Reference
|
||
|
||
**Date**: 2025-11-08
|
||
**Key Insight**: "キャッシュ層って足らなくなったら動的拡張するものではないですかにゃ？" ← **100% CORRECT**
|
||
|
||
## Visual Summary
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────┐
|
||
│ HAKMEM Resource Management │
|
||
│ Fixed vs Dynamic Analysis │
|
||
└─────────────────────────────────────────────────────────────────┘
|
||
|
||
Component │ Type │ Capacity │ Expansion │ Priority
|
||
───────────────────┼────────────────┼───────────────┼──────────────┼──────────
|
||
SuperSlab │ Fixed Array │ 32 slabs │ ❌ None │ 🔴 CRITICAL
|
||
└─ slabs[] │ │ COMPILE-TIME │ │ 4T OOM!
|
||
│ │ │ │
|
||
TLS Cache │ Fixed Cap │ 256-768 slots │ ❌ None │ 🟡 HIGH
|
||
└─ g_tls_sll_* │ │ ENV override │ │ No adapt
|
||
│ │ │ │
|
||
BigCache │ Fixed 2D Array │ 256×8 = 2048 │ ❌ Eviction │ 🟡 MEDIUM
|
||
└─ g_cache[][] │ │ COMPILE-TIME │ │ Hash coll
|
||
│ │ │ │
|
||
L2.5 Pool │ Fixed Shards │ 64 shards │ ❌ None │ 🟡 MEDIUM
|
||
└─ freelist[][] │ │ COMPILE-TIME │ │ Contention
|
||
│ │ │ │
|
||
Mid Registry │ Dynamic Array │ 64 → 2x │ ✅ Grows │ ✅ GOOD
|
||
└─ entries │ │ RUNTIME mmap │ │ Correct!
|
||
│ │ │ │
|
||
Mid TLS Ring │ Fixed Array │ 48 slots │ ❌ Overflow │ 🟢 LOW
|
||
└─ items[] │ │ to LIFO │ │ Minor
|
||
```
|
||
|
||
## Problem: SuperSlab Fixed 32 Slabs (CRITICAL)
|
||
|
||
```
|
||
Current Design (BROKEN):
|
||
┌────────────────────────────────────────────┐
|
||
│ SuperSlab (2MB) │
|
||
│ ┌────────────────────────────────────────┐ │
|
||
│ │ slabs[32] ← FIXED ARRAY! │ │
|
||
│ │ [0][1][2]...[31] ← Cannot grow! │ │
|
||
│ └────────────────────────────────────────┘ │
|
||
│ │
|
||
│ 4T high-contention: │
|
||
│ Thread 1: slabs[0-7] ← all busy │
|
||
│ Thread 2: slabs[8-15] ← all busy │
|
||
│ Thread 3: slabs[16-23] ← all busy │
|
||
│ Thread 4: slabs[24-31] ← all busy │
|
||
│ → OOM! No more slabs! │
|
||
└────────────────────────────────────────────┘
|
||
|
||
Proposed Fix (Mimalloc-style):
|
||
┌────────────────────────────────────────────┐
|
||
│ SuperSlabChunk (2MB) │
|
||
│ ┌────────────────────────────────────────┐ │
|
||
│ │ slabs[32] (initial) │ │
|
||
│ └────────────────────────────────────────┘ │
|
||
│ ↓ link on overflow │
|
||
│ ┌────────────────────────────────────────┐ │
|
||
│ │ slabs[32] (expansion chunk) │ │
|
||
│ └────────────────────────────────────────┘ │
|
||
│ ↓ can continue growing │
|
||
│ ... │
|
||
│ │
|
||
│ 4T high-contention: │
|
||
│ Chunk 1: slabs[0-31] ← full │
|
||
│ → Allocate Chunk 2 │
|
||
│ Chunk 2: slabs[32-63] ← expand! │
|
||
│ → No OOM! │
|
||
└────────────────────────────────────────────┘
|
||
```
|
||
|
||
## Comparison: HAKMEM vs Other Allocators
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────┐
|
||
│ Dynamic Expansion │
|
||
└─────────────────────────────────────────────────────────────────┘
|
||
|
||
mimalloc:
|
||
Segment → Pages → Blocks
|
||
✅ Variable segment size
|
||
✅ Dynamic page allocation
|
||
✅ Adaptive thread cache
|
||
|
||
jemalloc:
|
||
Chunk → Runs → Regions
|
||
✅ Variable chunk size
|
||
✅ Dynamic run creation
|
||
✅ Adaptive tcache
|
||
|
||
HAKMEM:
|
||
SuperSlab → Slabs → Blocks
|
||
❌ Fixed 2MB SuperSlab size
|
||
❌ Fixed 32 slabs per SuperSlab ← PROBLEM!
|
||
❌ Fixed TLS cache capacity
|
||
✅ Dynamic Mid Registry (only this!)
|
||
```
|
||
|
||
## Fix Priority Matrix
|
||
|
||
```
|
||
High Impact
|
||
▲
|
||
│
|
||
┌────────────┼────────────┐
|
||
│ SuperSlab │ │
|
||
│ (32 slabs) │ TLS Cache │
|
||
│ 🔴 CRITICAL│ (256-768) │
|
||
│ 7-10 days │ 🟡 HIGH │
|
||
│ │ 3-5 days │
|
||
├────────────┼────────────┤
|
||
│ BigCache │ L2.5 Pool │
|
||
│ (256×8) │ (64 shards)│
|
||
│ 🟡 MEDIUM │ 🟡 MEDIUM │
|
||
│ 1-2 days │ 2-3 days │
|
||
└────────────┼────────────┘
|
||
│
|
||
▼
|
||
Low Impact
|
||
◄────────────┼────────────►
|
||
Low Effort High Effort
|
||
```
|
||
|
||
## Quick Stats
|
||
|
||
```
|
||
Total Components Analyzed: 6
|
||
├─ CRITICAL issues: 1 (SuperSlab)
|
||
├─ HIGH issues: 1 (TLS Cache)
|
||
├─ MEDIUM issues: 2 (BigCache, L2.5)
|
||
├─ LOW issues: 1 (Mid TLS Ring)
|
||
└─ GOOD examples: 1 (Mid Registry) ✅
|
||
|
||
Estimated Fix Effort: 13-20 days
|
||
├─ Phase 2a (SuperSlab): 7-10 days
|
||
├─ Phase 2b (TLS Cache): 3-5 days
|
||
└─ Phase 2c (Others): 3-5 days
|
||
|
||
Expected Outcomes:
|
||
✅ 4T stable operation (no OOM)
|
||
✅ Adaptive performance (hot classes get more cache)
|
||
✅ Better memory efficiency (no over-provisioning)
|
||
```
|
||
|
||
## Key Takeaways
|
||
|
||
1. **User is 100% correct**: Cache layers should expand dynamically.
|
||
|
||
2. **Root cause of 4T crashes**: SuperSlab fixed 32-slab array.
|
||
|
||
3. **Mid Registry is the gold standard**: Use its pattern for other components.
|
||
|
||
4. **Design principle**: "Resources should expand on-demand, not be pre-allocated."
|
||
|
||
5. **Fix order**: SuperSlab → TLS Cache → BigCache → L2.5 Pool.
|
||
|
||
---
|
||
|
||
**Full Analysis**: See [`DESIGN_FLAWS_ANALYSIS.md`](DESIGN_FLAWS_ANALYSIS.md) (11 chapters, detailed roadmap)
|