hakmem/docs/analysis/PHASE7_CRITICAL_FINDINGS_SUMMARY.md

# Phase 7 Critical Findings - Executive Summary

**Date:** 2025-11-09
**Status:** 🚨 **CRITICAL PERFORMANCE ISSUE IDENTIFIED**

---

## TL;DR

**Previous Report:** 17M ops/s (3-4x slower than System)
**Actual Reality:** **4.5M ops/s (16x slower than System)** 💀💀💀

**Root Cause:** Phase 7 header-based fast free **is NOT working** (100% of frees use slow SuperSlab lookup)

---

## Actual Measured Performance

| Size | HAKMEM | System | Gap |
|------|--------|--------|-----|
| 128B | 4.53M ops/s | 81.78M ops/s | **18.1x slower** |
| 256B | 4.76M ops/s | 79.29M ops/s | **16.7x slower** |
| 512B | 4.80M ops/s | 73.24M ops/s | **15.3x slower** |
| 1024B | 4.78M ops/s | 69.63M ops/s | **14.6x slower** |

**Average: 16.2x slower than System malloc**

---

## Critical Issue: Phase 7 Header Free NOT Working

### Expected Behavior (Phase 7)

```c
void free(ptr) {
    uint8_t cls = *((uint8_t*)ptr - 1);  // Read 1-byte header (5-10 cycles)
    *(void**)ptr = g_tls_head[cls];       // Push to TLS (2-3 cycles)
    g_tls_head[cls] = ptr;
}
```

**Expected: 5-10 cycles**

### Actual Behavior (Observed)

```c
void free(ptr) {
    SuperSlab* ss = hak_super_lookup(ptr);  // Hash + linear probing (100+ cycles!)
    hak_tiny_free_superslab(ptr, ss);
}
```

**Actual: 100+ cycles** ❌

### Evidence

```bash
$ HAKMEM_FREE_ROUTE_TRACE=1 ./bench_random_mixed_hakmem 100 128 42
[FREE_ROUTE] ss_hit ptr=0x79796a810040
[FREE_ROUTE] ss_hit ptr=0x79796ac10000
[FREE_ROUTE] ss_hit ptr=0x79796ac10020
...
```

**100% ss_hit (SuperSlab lookup), 0% header_fast**

---

## Top 3 Bottlenecks (Priority Order)

### 1. SuperSlab Lookup in Free Path 🔥🔥🔥

**Current:** 100+ cycles per free
**Expected (Phase 7):** 5-10 cycles per free
**Potential Gain:** **+400-800%** (biggest win!)

**Action:** Debug why `hak_tiny_free_fast_v2()` returns 0 (failure)

---

### 2. Wrapper Overhead 🔥

**Current:** 20-30 cycles per malloc/free
**Expected:** 5-10 cycles
**Potential Gain:** **+30-50%**

**Issues:**
- LD_PRELOAD checks (every call)
- Initialization guards (every call)
- TLS depth tracking (every call)

**Action:** Eliminate unnecessary checks in direct-link builds

---

### 3. Front Gate Complexity 🟡

**Current:** 30+ instructions per allocation
**Expected:** 10-15 instructions
**Potential Gain:** **+10-20%**

**Issues:**
- SFC/SLL split (2 layers instead of 1)
- Corruption checks (even in release!)
- Hit counters (every allocation)

**Action:** Simplify to single TLS freelist

---

## Cycle Count Analysis

| Operation | System malloc | HAKMEM Phase 7 | Ratio |
|-----------|--------------|----------------|-------|
| malloc() | 10-15 cycles | 100-150 cycles | **10-15x** |
| free() | 8-12 cycles | 150-250 cycles | **18-31x** |
| **Combined** | **18-27 cycles** | **250-400 cycles** | **14-22x** 🔥 |

**Measured 16.2x gap ✅ matches theoretical 14-22x estimate!**

---

## Immediate Action Items

### This Week: Fix Phase 7 Header Free (CRITICAL!)

**Investigation Steps:**

1. **Verify headers are written on allocation**
   - Add debug log to `tiny_region_id_write_header()`
   - Confirm magic byte 0xa0 is written

2. **Find why free path fails header check**
   - Add debug log to `hak_tiny_free_fast_v2()`
   - Check why it returns 0

3. **Check dispatch priority**
   - Is Pool TLS checked before Tiny?
   - Is magic validation correct? (0xa0 vs 0xb0)

4. **Fix root cause**
   - Ensure headers are written
   - Fix dispatch logic
   - Prioritize header path over SuperSlab

**Expected Result:** 4.5M → 18-25M ops/s (+400-550%)

---

### Next Week: Eliminate Wrapper Overhead

**Changes:**

1. Skip LD_PRELOAD checks in direct-link builds
2. Use one-time initialization flag
3. Replace TLS depth with atomic recursion guard
4. Move force_libc to compile-time

**Expected Result:** 18-25M → 28-35M ops/s (+55-75%)

---

### Week 3: Simplify + Polish

**Changes:**

1. Single TLS freelist (remove SFC/SLL split)
2. Remove corruption checks in release
3. Remove debug counters
4. Final validation

**Expected Result:** 28-35M → 35-45M ops/s (+25-30%)

---

## Target Performance

**Current:** 4.5M ops/s (5.5% of System)
**After Fix 1:** 18-25M ops/s (25-30% of System)
**After Fix 2:** 28-35M ops/s (40-50% of System)
**After Fix 3:** **35-45M ops/s (50-60% of System)** ✅ Acceptable!

**Final Gap:** 50-60% of System malloc (acceptable for learning allocator with advanced features)

---

## What Went Wrong

1. **Previous performance reports used wrong measurements**
   - Possibly stale binary or cached results
   - Need strict build verification

2. **Phase 7 implementation is correct but NOT activated**
   - Header write/read logic exists
   - Dispatch logic prefers SuperSlab over header
   - Needs debugging to find why

3. **Wrapper overhead accumulated unnoticed**
   - Each guard adds 2-5 cycles
   - 5-10 guards = 20-30 cycles
   - System malloc has ~0 wrapper overhead

---

## Confidence Level

**Measurements:** ✅ High (3 runs each, consistent results)
**Analysis:** ✅ High (code inspection + theory matches reality)
**Fixes:** ⚠️ Medium (need to debug Phase 7 header issue)

**Projected Gain:** 7-10x improvement possible (to 35-45M ops/s)

---

## Full Report

See: `PHASE7_PERFORMANCE_INVESTIGATION_REPORT.md`

---

**Prepared by:** Claude Task Agent
**Investigation Mode:** Ultrathink (measurement-based, no speculation)
**Status:** Ready for immediate action