hakmem/docs/status/PHASE7_CRITICAL_FINDINGS_SUMMARY.md

Phase 7 Critical Findings - Executive Summary

Date: 2025-11-09 Status: 🚨 CRITICAL PERFORMANCE ISSUE IDENTIFIED

---

TL;DR

Previous Report: 17M ops/s (3-4x slower than System) Actual Reality: 4.5M ops/s (16x slower than System) 💀💀💀

Root Cause: Phase 7 header-based fast free is NOT working (100% of frees use slow SuperSlab lookup)

---

Actual Measured Performance

Size	HAKMEM	System	Gap
128B	4.53M ops/s	81.78M ops/s	18.1x slower
256B	4.76M ops/s	79.29M ops/s	16.7x slower
512B	4.80M ops/s	73.24M ops/s	15.3x slower
1024B	4.78M ops/s	69.63M ops/s	14.6x slower

Average: 16.2x slower than System malloc

---

Critical Issue: Phase 7 Header Free NOT Working

Expected Behavior (Phase 7)

void free(ptr) {
    uint8_t cls = *((uint8_t*)ptr - 1);  // Read 1-byte header (5-10 cycles)
    *(void**)ptr = g_tls_head[cls];       // Push to TLS (2-3 cycles)
    g_tls_head[cls] = ptr;
}

Expected: 5-10 cycles

Actual Behavior (Observed)

void free(ptr) {
    SuperSlab* ss = hak_super_lookup(ptr);  // Hash + linear probing (100+ cycles!)
    hak_tiny_free_superslab(ptr, ss);
}

Actual: 100+ cycles ❌

Evidence

$ HAKMEM_FREE_ROUTE_TRACE=1 ./bench_random_mixed_hakmem 100 128 42
[FREE_ROUTE] ss_hit ptr=0x79796a810040
[FREE_ROUTE] ss_hit ptr=0x79796ac10000
[FREE_ROUTE] ss_hit ptr=0x79796ac10020
...

100% ss_hit (SuperSlab lookup), 0% header_fast

---

Top 3 Bottlenecks (Priority Order)

1. SuperSlab Lookup in Free Path 🔥🔥🔥

Current: 100+ cycles per free Expected (Phase 7): 5-10 cycles per free Potential Gain: +400-800% (biggest win!)

Action: Debug why hak_tiny_free_fast_v2() returns 0 (failure)

---

2. Wrapper Overhead 🔥

Current: 20-30 cycles per malloc/free Expected: 5-10 cycles Potential Gain: +30-50%

Issues:

• LD_PRELOAD checks (every call)
• Initialization guards (every call)
• TLS depth tracking (every call)

Action: Eliminate unnecessary checks in direct-link builds

---

3. Front Gate Complexity 🟡

Current: 30+ instructions per allocation Expected: 10-15 instructions Potential Gain: +10-20%

Issues:

• SFC/SLL split (2 layers instead of 1)
• Corruption checks (even in release!)
• Hit counters (every allocation)

Action: Simplify to single TLS freelist

---

Cycle Count Analysis

Operation	System malloc	HAKMEM Phase 7	Ratio
malloc()	10-15 cycles	100-150 cycles	10-15x
free()	8-12 cycles	150-250 cycles	18-31x
Combined	18-27 cycles	250-400 cycles	14-22x 🔥

Measured 16.2x gap ✅ matches theoretical 14-22x estimate!

---

Immediate Action Items

This Week: Fix Phase 7 Header Free (CRITICAL!)

Investigation Steps:

1. Verify headers are written on allocation

   • Add debug log to tiny_region_id_write_header()
   • Confirm magic byte 0xa0 is written

2. Find why free path fails header check

   • Add debug log to hak_tiny_free_fast_v2()
   • Check why it returns 0

3. Check dispatch priority

   • Is Pool TLS checked before Tiny?
   • Is magic validation correct? (0xa0 vs 0xb0)

4. Fix root cause

   • Ensure headers are written
   • Fix dispatch logic
   • Prioritize header path over SuperSlab

Expected Result: 4.5M → 18-25M ops/s (+400-550%)

---

Next Week: Eliminate Wrapper Overhead

Changes:

1. Skip LD_PRELOAD checks in direct-link builds
2. Use one-time initialization flag
3. Replace TLS depth with atomic recursion guard
4. Move force_libc to compile-time

Expected Result: 18-25M → 28-35M ops/s (+55-75%)

---

Week 3: Simplify + Polish

Changes:

1. Single TLS freelist (remove SFC/SLL split)
2. Remove corruption checks in release
3. Remove debug counters
4. Final validation

Expected Result: 28-35M → 35-45M ops/s (+25-30%)

---

Target Performance

Current: 4.5M ops/s (5.5% of System) After Fix 1: 18-25M ops/s (25-30% of System) After Fix 2: 28-35M ops/s (40-50% of System) After Fix 3: 35-45M ops/s (50-60% of System) ✅ Acceptable!

Final Gap: 50-60% of System malloc (acceptable for learning allocator with advanced features)

---

What Went Wrong

1. Previous performance reports used wrong measurements

   • Possibly stale binary or cached results
   • Need strict build verification

2. Phase 7 implementation is correct but NOT activated

   • Header write/read logic exists
   • Dispatch logic prefers SuperSlab over header
   • Needs debugging to find why

3. Wrapper overhead accumulated unnoticed

   • Each guard adds 2-5 cycles
   • 5-10 guards = 20-30 cycles
   • System malloc has ~0 wrapper overhead

---

Confidence Level

Measurements: ✅ High (3 runs each, consistent results) Analysis: ✅ High (code inspection + theory matches reality) Fixes: ⚠️ Medium (need to debug Phase 7 header issue)

Projected Gain: 7-10x improvement possible (to 35-45M ops/s)

---

Full Report

See: PHASE7_PERFORMANCE_INVESTIGATION_REPORT.md

---

Prepared by: Claude Task Agent Investigation Mode: Ultrathink (measurement-based, no speculation) Status: Ready for immediate action