hakmem/docs/analysis/DESIGN_FLAWS_ANALYSIS.md

HAKMEM Design Flaws Analysis - Dynamic Scaling Investigation

Date: 2025-11-08 Investigator: Claude Task Agent (Ultrathink Mode) Trigger: User insight - "キャッシュ層って足らなくなったら動的拡張するものではないですかにゃ？"

Executive Summary

User is 100% correct. Fixed-size caches are a fundamental design flaw.

HAKMEM suffers from multiple fixed-capacity bottlenecks that prevent dynamic scaling under high load. While some components (Mid Registry) correctly implement dynamic expansion, most critical components use fixed-size arrays that cannot grow when capacity is exhausted.

Critical Finding: SuperSlab uses a fixed 32-slab array, causing 4T high-contention OOM crashes. This is the root cause of the observed failures.

---

1. SuperSlab Fixed Size (CRITICAL 🔴)

Problem

File: /mnt/workdisk/public_share/hakmem/core/superslab/superslab_types.h:82

typedef struct SuperSlab {
    // ...
    TinySlabMeta slabs[SLABS_PER_SUPERSLAB_MAX];  // ← FIXED 32 slabs!
    _Atomic(uintptr_t) remote_heads[SLABS_PER_SUPERSLAB_MAX];
    _Atomic(uint32_t)  remote_counts[SLABS_PER_SUPERSLAB_MAX];
    atomic_uint slab_listed[SLABS_PER_SUPERSLAB_MAX];
} SuperSlab;

Impact:

• 4T high-contention: Each SuperSlab has only 32 slabs, leading to contention and OOM
• No dynamic expansion: When all 32 slabs are active, the only option is to allocate a new SuperSlab (expensive 2MB mmap)
• Memory fragmentation: Multiple partially-used SuperSlabs waste memory

Why this is wrong:

• SuperSlab itself is dynamically allocated (via ss_os_acquire() → mmap)
• Registry supports unlimited SuperSlabs (dynamic array, see below)
• BUT: Each SuperSlab is capped at 32 slabs (fixed array)

Comparison with other allocators:

Allocator	Structure	Capacity	Dynamic Expansion
mimalloc	Segment	Variable pages	✅ On-demand page allocation
jemalloc	Chunk	Variable runs	✅ Dynamic run creation
HAKMEM	SuperSlab	Fixed 32 slabs	❌ Must allocate new SuperSlab

Root cause: Fixed-size array prevents per-SuperSlab scaling.

Evidence

Allocation (hakmem_tiny_superslab.c:321-485):

SuperSlab* superslab_allocate(uint8_t size_class) {
    // ... environment parsing ...
    ptr = ss_os_acquire(size_class, ss_size, ss_mask, populate);  // mmap 2MB
    // ... initialize header ...
    int max_slabs = (int)(ss_size / SLAB_SIZE);  // max_slabs = 32 for 2MB
    for (int i = 0; i < max_slabs; i++) {
        ss->slabs[i].freelist = NULL;  // Initialize fixed 32 slabs
        // ...
    }
}

Problem: slabs[SLABS_PER_SUPERSLAB_MAX] is a compile-time fixed array, not a dynamic allocation.

Fix Difficulty

Difficulty: HIGH (7-10 days)

Why:

1. ABI change: All SuperSlab pointers would need to carry size info
2. Alignment requirements: SuperSlab must remain 2MB-aligned for fast ptr & ~MASK lookup
3. Registry refactoring: Need to store per-SuperSlab capacity in registry
4. Atomic operations: All slab access needs bounds checking

Proposed Fix (Phase 2a):

// Option A: Variable-length array (requires allocation refactoring)
typedef struct SuperSlab {
    uint64_t magic;
    uint8_t  size_class;
    uint8_t  active_slabs;
    uint8_t  lg_size;
    uint8_t  max_slabs;  // NEW: actual capacity (16-32)
    // ...
    TinySlabMeta slabs[];  // Flexible array member
} SuperSlab;

// Option B: Two-tier structure (easier, mimalloc-style)
typedef struct SuperSlabChunk {
    SuperSlabHeader header;
    TinySlabMeta slabs[32];  // First chunk
    SuperSlabChunk* next;     // Link to additional chunks (if needed)
} SuperSlabChunk;

Recommendation: Option B (mimalloc-style linked chunks) for easier migration.

---

2. TLS Cache Fixed Capacity (HIGH 🟡)

Problem

File: /mnt/workdisk/public_share/hakmem/core/hakmem_tiny.c:1752-1762

static inline int ultra_sll_cap_for_class(int class_idx) {
    int ov = g_ultra_sll_cap_override[class_idx];
    if (ov > 0) return ov;
    switch (class_idx) {
        case 0: return 256;   // 8B   ← FIXED CAPACITY
        case 1: return 384;   // 16B  ← FIXED CAPACITY
        case 2: return 384;   // 32B
        case 3: return 768;   // 64B
        case 4: return 256;   // 128B
        default: return 128;
    }
}

Impact:

• Fixed capacity per class: 256-768 blocks
• Overflow behavior: Spill to Magazine (HKP_TINY_SPILL), which also has fixed capacity
• No learning: Cannot adapt to workload (hot classes stuck at fixed cap)

Evidence (hakmem_tiny_free.inc:269-299):

uint32_t sll_cap = sll_cap_for_class(class_idx, (uint32_t)TINY_TLS_MAG_CAP);
if ((int)g_tls_sll_count[class_idx] < (int)sll_cap) {
    // Push to TLS cache
    *(void**)ptr = g_tls_sll_head[class_idx];
    g_tls_sll_head[class_idx] = ptr;
    g_tls_sll_count[class_idx]++;
} else {
    // Overflow: spill to Magazine (also fixed capacity!)
    // ...
}

Comparison with other allocators:

Allocator	TLS Cache	Capacity	Dynamic Adjustment
mimalloc	Thread-local free list	Variable	✅ Adapts to workload
jemalloc	tcache	Variable	✅ Dynamic sizing based on usage
HAKMEM	g_tls_sll	Fixed 256-768	❌ Override via env var only

Fix Difficulty

Difficulty: MEDIUM (3-5 days)

Proposed Fix (Phase 2b):

// Per-class dynamic capacity
static __thread struct {
    void* head;
    uint32_t count;
    uint32_t capacity;     // NEW: dynamic capacity
    uint32_t high_water;   // Track peak usage
} g_tls_sll_dynamic[TINY_NUM_CLASSES];

// Adaptive resizing
if (high_water > capacity * 0.9) {
    capacity = min(capacity * 2, MAX_CAP);  // Grow by 2x
}
if (high_water < capacity * 0.3) {
    capacity = max(capacity / 2, MIN_CAP);  // Shrink by 2x
}

---

3. BigCache Fixed Size (MEDIUM 🟡)

Problem

File: /mnt/workdisk/public_share/hakmem/core/hakmem_bigcache.c:29

// Fixed 2D array: 256 sites × 8 classes = 2048 slots
static BigCacheSlot g_cache[BIGCACHE_MAX_SITES][BIGCACHE_NUM_CLASSES];

Impact:

• Fixed 256 sites: Hash collision causes eviction, not expansion
• Fixed 8 classes: Cannot add new size classes
• LFU eviction: Old entries are evicted instead of expanding cache

Eviction logic (hakmem_bigcache.c:106-118):

static inline void evict_slot(BigCacheSlot* slot) {
    if (!slot->valid) return;
    if (g_free_callback) {
        g_free_callback(slot->ptr, slot->actual_bytes);  // Free evicted block
    }
    slot->valid = 0;
    g_stats.evictions++;
}

Problem: When cache is full, blocks are freed instead of expanding cache.

Fix Difficulty

Difficulty: LOW (1-2 days)

Proposed Fix (Phase 2c):

// Hash table with chaining (mimalloc pattern)
typedef struct BigCacheEntry {
    void* ptr;
    size_t actual_bytes;
    size_t class_bytes;
    uintptr_t site;
    struct BigCacheEntry* next;  // Chaining for collisions
} BigCacheEntry;

static BigCacheEntry* g_cache_buckets[BIGCACHE_BUCKETS];  // Hash table
static size_t g_cache_count = 0;
static size_t g_cache_capacity = INITIAL_CAPACITY;

// Dynamic expansion
if (g_cache_count > g_cache_capacity * 0.75) {
    rehash(g_cache_capacity * 2);  // Grow and rehash
}

---

4. L2.5 Pool Fixed Shards (MEDIUM 🟡)

Problem

File: /mnt/workdisk/public_share/hakmem/core/hakmem_l25_pool.c:92-100

static struct {
    L25Block* freelist[L25_NUM_CLASSES][L25_NUM_SHARDS];  // Fixed 5×64 = 320 lists
    PaddedMutex freelist_locks[L25_NUM_CLASSES][L25_NUM_SHARDS];
    atomic_uint_fast64_t nonempty_mask[L25_NUM_CLASSES];
    // ...
} g_l25_pool;

Impact:

• Fixed 64 shards: Cannot add more shards under high contention
• Fixed 5 classes: Cannot add new size classes
• Soft CAP: bundles_by_class[] limits total allocations per class (not clear what happens on overflow)

Evidence (hakmem_l25_pool.c:108-112):

// Per-class bundle accounting (for Soft CAP guidance)
uint64_t bundles_by_class[L25_NUM_CLASSES] __attribute__((aligned(64)));

Question: What happens when Soft CAP is reached? (Needs code inspection)

Fix Difficulty

Difficulty: LOW-MEDIUM (2-3 days)

Proposed Fix: Dynamic shard allocation (jemalloc pattern)

---

5. Mid Pool TLS Ring Fixed Size (LOW 🟢)

Problem

File: /mnt/workdisk/public_share/hakmem/core/box/pool_tls_types.inc.h:15-18

#ifndef POOL_L2_RING_CAP
#define POOL_L2_RING_CAP 48  // Fixed 48 slots
#endif
typedef struct { PoolBlock* items[POOL_L2_RING_CAP]; int top; } PoolTLSRing;

Impact:

• Fixed 48 slots per TLS ring: Overflow goes to lo_head LIFO (unbounded)
• Minor issue: LIFO is unbounded, so this is less critical

Fix Difficulty

Difficulty: LOW (1 day)

Proposed Fix: Dynamic ring size based on usage.

---

6. Mid Registry (GOOD ✅)

Correct Implementation

File: /mnt/workdisk/public_share/hakmem/core/hakmem_mid_mt.c:78-114

static void registry_add(void* base, size_t block_size, int class_idx) {
    pthread_mutex_lock(&g_mid_registry.lock);

    // ✅ DYNAMIC EXPANSION!
    if (g_mid_registry.count >= g_mid_registry.capacity) {
        uint32_t new_capacity = g_mid_registry.capacity == 0
            ? MID_REGISTRY_INITIAL_CAPACITY  // Start at 64
            : g_mid_registry.capacity * 2;   // Double on overflow

        size_t new_size = new_capacity * sizeof(MidSegmentRegistry);
        MidSegmentRegistry* new_entries = mmap(
            NULL, new_size,
            PROT_READ | PROT_WRITE,
            MAP_PRIVATE | MAP_ANONYMOUS,
            -1, 0
        );

        if (new_entries != MAP_FAILED) {
            memcpy(new_entries, g_mid_registry.entries,
                   g_mid_registry.count * sizeof(MidSegmentRegistry));
            g_mid_registry.entries = new_entries;
            g_mid_registry.capacity = new_capacity;
        }
    }
    // ...
}

Why this is correct:

1. Initial capacity: 64 entries
2. Exponential growth: 2x on overflow
3. mmap instead of realloc: Avoids deadlock (malloc → mid_mt → registry_add)
4. Lazy cleanup: Old mappings not freed (simple, avoids complexity)

This is the pattern that should be applied to other components.

---

7. System malloc/mimalloc Comparison

mimalloc Dynamic Expansion Pattern

Segment allocation:

// mimalloc segments are allocated on-demand
mi_segment_t* mi_segment_alloc(size_t required) {
    size_t segment_size = _mi_segment_size(required);  // Variable size!
    void* p = _mi_os_alloc(segment_size);
    // Initialize segment with variable page count
    mi_segment_t* segment = (mi_segment_t*)p;
    segment->page_count = segment_size / MI_PAGE_SIZE;  // Dynamic!
    return segment;
}

Key differences:

• Variable segment size: Not fixed at 2MB
• Variable page count: Adapts to allocation size
• Thread cache adapts: mi_page_free_collect() grows/shrinks based on usage

jemalloc Dynamic Expansion Pattern

Chunk allocation:

// jemalloc chunks are allocated with variable run sizes
chunk_t* chunk_alloc(size_t size, size_t alignment) {
    void* ret = pages_map(NULL, size);  // Variable size
    chunk_register(ret, size);  // Register in dynamic registry
    return ret;
}

Key differences:

• Variable chunk size: Not fixed
• Dynamic run creation: Runs are created as needed within chunks
• tcache adapts: Thread cache grows/shrinks based on miss rate

HAKMEM vs. Others

Feature	mimalloc	jemalloc	HAKMEM
Segment/Chunk Size	Variable	Variable	Fixed 2MB
Slabs/Pages/Runs	Dynamic	Dynamic	Fixed 32
Registry	Dynamic	Dynamic	✅ Dynamic
Thread Cache	Adaptive	Adaptive	Fixed cap
BigCache	N/A	N/A	Fixed 2D array

Conclusion: HAKMEM has multiple fixed-capacity bottlenecks that other allocators avoid.

---

8. Priority-Ranked Fix List

CRITICAL (Immediate Action Required)

1. SuperSlab Dynamic Slabs (CRITICAL 🔴)

• Problem: Fixed 32 slabs per SuperSlab → 4T OOM
• Impact: Allocator crashes under high contention
• Effort: 7-10 days
• Approach: Mimalloc-style linked chunks
• Files: superslab/superslab_types.h, hakmem_tiny_superslab.c

HIGH (Performance/Stability Impact)

2. TLS Cache Dynamic Capacity (HIGH 🟡)

• Problem: Fixed 256-768 capacity → cannot adapt to hot classes
• Impact: Performance degradation on skewed workloads
• Effort: 3-5 days
• Approach: Adaptive resizing based on high-water mark
• Files: hakmem_tiny.c, hakmem_tiny_free.inc

3. Magazine Dynamic Capacity (HIGH 🟡)

• Problem: Fixed capacity (not investigated in detail)
• Impact: Spill behavior under load
• Effort: 2-3 days
• Approach: Link to TLS Cache dynamic sizing

MEDIUM (Memory Efficiency Impact)

4. BigCache Hash Table (MEDIUM 🟡)

• Problem: Fixed 256 sites × 8 classes → eviction instead of expansion
• Impact: Cache miss rate increases with site count
• Effort: 1-2 days
• Approach: Hash table with chaining
• Files: hakmem_bigcache.c

5. L2.5 Pool Dynamic Shards (MEDIUM 🟡)

• Problem: Fixed 64 shards → contention under high load
• Impact: Lock contention on popular shards
• Effort: 2-3 days
• Approach: Dynamic shard allocation
• Files: hakmem_l25_pool.c

LOW (Edge Cases)

6. Mid Pool TLS Ring (LOW 🟢)

• Problem: Fixed 48 slots → minor overflow to LIFO
• Impact: Minimal (LIFO is unbounded)
• Effort: 1 day
• Approach: Dynamic ring size
• Files: box/pool_tls_types.inc.h

---

9. Implementation Roadmap

Phase 2a: SuperSlab Dynamic Expansion (7-10 days)

Goal: Allow SuperSlab to grow beyond 32 slabs under high contention.

Approach: Mimalloc-style linked chunks

Steps:

1. Refactor SuperSlab structure (2 days)

   • Add max_slabs field
   • Add next_chunk pointer for expansion
   • Update all slab access to use max_slabs

2. Implement chunk allocation (2 days)

   • superslab_expand_chunk() - allocate additional 32-slab chunk
   • Link new chunk to existing SuperSlab
   • Update active_slabs and max_slabs

3. Update refill logic (2 days)

   • superslab_refill() - check if expansion is cheaper than new SuperSlab
   • Expand existing SuperSlab if active_slabs < max_slabs

4. Update registry (1 day)

   • Store max_slabs in registry for lookup bounds checking

5. Testing (2 days)

   • 4T Larson stress test
   • Valgrind memory leak check
   • Performance regression testing

Success Metric: 4T Larson runs without OOM.

Phase 2b: TLS Cache Adaptive Sizing (3-5 days)

Goal: Dynamically adjust TLS cache capacity based on workload.

Approach: High-water mark tracking + exponential growth/shrink

Steps:

1. Add dynamic capacity tracking (1 day)

   • Per-class capacity and high_water fields
   • Update g_tls_sll_count checks to use dynamic capacity

2. Implement resize logic (2 days)

   • Grow: capacity *= 2 when high_water > capacity * 0.9
   • Shrink: capacity /= 2 when high_water < capacity * 0.3
   • Clamp: MIN_CAP = 64, MAX_CAP = 4096

3. Testing (1-2 days)

   • Larson with skewed size distribution
   • Memory footprint measurement

Success Metric: Adaptive capacity matches workload, no fixed limits.

Phase 2c: BigCache Hash Table (1-2 days)

Goal: Replace fixed 2D array with dynamic hash table.

Approach: Chaining for collision resolution + rehashing on 75% load

Steps:

1. Refactor to hash table (1 day)

   • Replace g_cache[][] with g_cache_buckets[]
   • Implement chaining for collisions

2. Implement rehashing (1 day)

   • Trigger: count > capacity * 0.75
   • Double bucket count and rehash

Success Metric: No evictions due to hash collisions.

---

10. Recommendations

Immediate Actions

1. Fix SuperSlab fixed-size bottleneck (CRITICAL)

   • This is the root cause of 4T crashes
   • Implement mimalloc-style chunk linking
   • Target: Complete within 2 weeks

2. Audit all fixed-size arrays

   • Search codebase for [CONSTANT] array declarations
   • Flag all non-dynamic structures
   • Prioritize by impact

3. Implement dynamic sizing as default pattern

   • All new components should use dynamic allocation
   • Document pattern in CONTRIBUTING.md

Long-Term Strategy

Adopt mimalloc/jemalloc patterns:

• Variable-size segments/chunks
• Adaptive thread caches
• Dynamic registry/metadata structures

Design principle: "Resources should expand on-demand, not be pre-allocated."

---

11. Conclusion

User's insight is 100% correct: Cache layers should expand dynamically when capacity is insufficient.

HAKMEM has multiple fixed-capacity bottlenecks:

• SuperSlab: Fixed 32 slabs (CRITICAL)
• TLS Cache: Fixed 256-768 capacity (HIGH)
• BigCache: Fixed 256×8 array (MEDIUM)
• L2.5 Pool: Fixed 64 shards (MEDIUM)

Mid Registry is the exception - it correctly implements dynamic expansion via exponential growth and mmap.

Fix priority:

1. SuperSlab dynamic slabs (7-10 days) → Fixes 4T crashes
2. TLS Cache adaptive sizing (3-5 days) → Improves performance
3. BigCache hash table (1-2 days) → Reduces cache misses
4. L2.5 dynamic shards (2-3 days) → Reduces contention

Estimated total effort: 13-20 days for all critical fixes.

Expected outcome:

• 4T stable operation (no OOM)
• Adaptive performance (hot classes get more cache)
• Better memory efficiency (no over-provisioning)

---

Files for reference:

• SuperSlab: /mnt/workdisk/public_share/hakmem/core/superslab/superslab_types.h:82
• TLS Cache: /mnt/workdisk/public_share/hakmem/core/hakmem_tiny.c:1752
• BigCache: /mnt/workdisk/public_share/hakmem/core/hakmem_bigcache.c:29
• L2.5 Pool: /mnt/workdisk/public_share/hakmem/core/hakmem_l25_pool.c:92
• Mid Registry (GOOD): /mnt/workdisk/public_share/hakmem/core/hakmem_mid_mt.c:78