hakmem/docs/specs/ENV_VARS_COMPLETE.md

# HAKMEM Environment Variables Complete Reference

**Total Variables**: 83 environment variables + multiple compile-time flags
**Last Updated**: 2025-11-01
**Purpose**: Complete reference for diagnosing memory issues and configuration

---

## CRITICAL DISCOVERY: Statistics Disabled by Default

### The Problem
**Tiny Pool statistics are DISABLED** unless you build with `-DHAKMEM_ENABLE_STATS`:
- Current behavior: `alloc=0, free=0, slab=0` (statistics not collected)
- Impact: Memory diagnostics are blind
- Root cause: Build-time flag NOT set in Makefile

### How to Enable Statistics

**Option 1: Build with statistics** (RECOMMENDED for debugging)
```bash
make clean
make CFLAGS="-DHAKMEM_ENABLE_STATS" bench_fragment_stress_hakmem
```

**Option 2: Edit Makefile** (add to line 18)
```makefile
CFLAGS = -O3 ... -DHAKMEM_ENABLE_STATS ...
```

### Why Statistics are Disabled
From `/mnt/workdisk/public_share/hakmem/core/hakmem_tiny_stats.h`:
```c
// Purpose: Zero-overhead production builds by disabling stats collection
// Usage:   Build with -DHAKMEM_ENABLE_STATS to enable (default: disabled)
// Impact:  3-5% speedup when disabled (removes 0.5ns TLS increment)
//
// Default: DISABLED (production performance)
// Enable:  make CFLAGS=-DHAKMEM_ENABLE_STATS
```

**When DISABLED**: All `stats_record_alloc()` and `stats_record_free()` become no-ops
**When ENABLED**: Batched TLS counters track exact allocation/free counts

---

## Environment Variable Categories

### 1. Tiny Pool Core (Critical)

#### HAKMEM_WRAP_TINY
- **Default**: 1 (enabled)
- **Purpose**: Enable Tiny Pool fast-path (bypasses wrapper guard)
- **Impact**: Controls whether malloc/free use Tiny Pool for ≤1KB allocations
- **Usage**: `export HAKMEM_WRAP_TINY=1` (already default since Phase 7.4)
- **Location**: `/mnt/workdisk/public_share/hakmem/core/hakmem_tiny_init.inc:25`
- **Notes**: Without this, Tiny Pool returns NULL and falls back to L2/L25

#### HAKMEM_WRAP_TINY_REFILL
- **Default**: 0 (disabled)
- **Purpose**: Allow trylock-based magazine refill during wrapper calls
- **Impact**: Enables limited refill under trylock (no blocking)
- **Usage**: `export HAKMEM_WRAP_TINY_REFILL=1`
- **Safety**: OFF by default (avoids deadlock risk in recursive malloc)

#### HAKMEM_TINY_USE_SUPERSLAB
- **Default**: 1 (enabled)
- **Purpose**: Enable SuperSlab allocator for Tiny Pool slabs
- **Impact**: When OFF, Tiny Pool cannot allocate new slabs
- **Critical**: Must be ON for Tiny Pool to work

---

### 2. Tiny Pool TLS Caching (Performance Critical)

#### HAKMEM_TINY_MAG_CAP
- **Default**: Per-class (typically 512-2048)
- **Purpose**: Global TLS magazine capacity override
- **Impact**: Larger = fewer refills, more memory
- **Usage**: `export HAKMEM_TINY_MAG_CAP=1024`

#### HAKMEM_TINY_MAG_CAP_C{0..7}
- **Default**: None (uses class defaults)
- **Purpose**: Per-class magazine capacity override
- **Example**: `HAKMEM_TINY_MAG_CAP_C3=512` (64B class)
- **Classes**: C0=8B, C1=16B, C2=32B, C3=64B, C4=128B, C5=256B, C6=512B, C7=1KB

#### HAKMEM_TINY_TLS_SLL
- **Default**: 1 (enabled)
- **Purpose**: Enable TLS Single-Linked-List cache layer
- **Impact**: Fast-path cache before magazine
- **Performance**: Critical for tiny allocations (8-64B)

#### HAKMEM_SLL_MULTIPLIER
- **Default**: 2
- **Purpose**: SLL capacity = MAG_CAP × multiplier for small classes (0-3)
- **Range**: 1..16
- **Impact**: Higher = more TLS memory, fewer refills

#### HAKMEM_TINY_REFILL_MAX
- **Default**: 64
- **Purpose**: Magazine refill batch size (normal classes)
- **Impact**: Larger = fewer refills, more memory spike

#### HAKMEM_TINY_REFILL_MAX_HOT
- **Default**: 192
- **Purpose**: Magazine refill batch size for hot classes (≤64B)
- **Impact**: Larger batches for frequently used sizes

#### HAKMEM_TINY_REFILL_MAX_C{0..7}
- **Default**: None
- **Purpose**: Per-class refill batch override
- **Example**: `HAKMEM_TINY_REFILL_MAX_C2=96` (32B class)

#### HAKMEM_TINY_REFILL_MAX_HOT_C{0..7}
- **Default**: None
- **Purpose**: Per-class hot refill override (classes 0-3)
- **Priority**: Overrides HAKMEM_TINY_REFILL_MAX_HOT

---

### 3. SuperSlab Configuration

#### HAKMEM_TINY_SS_MAX_MB
- **Default**: Unlimited
- **Purpose**: Maximum SuperSlab memory per class (MB)
- **Impact**: Caps total slab allocation
- **Usage**: `export HAKMEM_TINY_SS_MAX_MB=512`

#### HAKMEM_TINY_SS_MIN_MB
- **Default**: 0
- **Purpose**: Minimum SuperSlab reservation per class (MB)
- **Impact**: Pre-allocates memory at startup

#### HAKMEM_TINY_SS_RESERVE
- **Default**: 0
- **Purpose**: Reserve SuperSlab memory at init
- **Impact**: Prevents initial allocation delays

#### HAKMEM_TINY_TRIM_SS
- **Default**: 0
- **Purpose**: Enable SuperSlab trimming/deallocation
- **Impact**: Returns memory to OS when idle

#### HAKMEM_TINY_SS_PARTIAL
- **Default**: 0
- **Purpose**: Enable partial slab reclamation
- **Impact**: Free partially-used slabs

#### HAKMEM_TINY_SS_PARTIAL_INTERVAL
- **Default**: 1000000 (1M allocations)
- **Purpose**: Interval between partial slab checks
- **Impact**: Lower = more aggressive trimming

#### HAKMEM_TINY_SS_CACHE
- **Default**: 0 (disabled)
- **Purpose**: Per-class SuperSlab cache capacity
- **Impact**: Limits how many freed SuperSlabs are kept in LRU cache before munmap

#### HAKMEM_TINY_SS_CACHE_C{0..7}
- **Default**: unset (inherits `HAKMEM_TINY_SS_CACHE`)
- **Purpose**: Per-class overrides for cache capacity
- **Impact**: Fine-grained control of cache size per Tiny class

#### HAKMEM_TINY_SS_PRECHARGE
- **Default**: 0
- **Purpose**: Precharge (pre-allocate) SuperSlabs into cache at startup/runtime
- **Impact**: Reduces first-use page faults by having warm SuperSlabs ready

#### HAKMEM_TINY_SS_PRECHARGE_C{0..7}
- **Default**: unset (inherits `HAKMEM_TINY_SS_PRECHARGE`)
- **Purpose**: Per-class precharge targets
- **Impact**: e.g., `HAKMEM_TINY_SS_PRECHARGE_C0=4` precharges 4 SuperSlabs for class 0

#### HAKMEM_TINY_SS_POPULATE_ONCE
- **Default**: 0
- **Purpose**: Use `MAP_POPULATE` for the next SuperSlab allocation only
- **Impact**: One-shot prefault for A/B testing; superseded by `HAKMEM_SS_PREFAULT` for常時運用

#### HAKMEM_SS_PREFAULT
- **Default**: `0` (OFF, safety-first default)
- **Type**: integer (0–3)
- **Purpose**: Control SuperSlab prefault strategy to reduce kernel page fault overhead (enabled explicitly when tuning).
- **Values**:
  - `0` = OFF — legacy behavior, only `HAKMEM_TINY_SS_POPULATE_ONCE` may trigger a one-shot `MAP_POPULATE`（現状の安全デフォルト）。
  - `1` = POPULATE — always pass `populate=1` to `ss_os_acquire()` (use `MAP_POPULATE` for every new SuperSlab). **要 perf 確認。**
  - `2` = TOUCH — POPULATE + `ss_prefault_region()` touches each page once (4KB stride) after `mmap`（実験用）。
  - `3` = ASYNC — reserved for future background-prefault implementation (currently treated as TOUCH).
- **Implementation**:
  - Policy Box: `core/box/ss_prefault_box.h`
  - Integration: `core/box/ss_allocation_box.c` calls `ss_prefault_policy()` to set `populate` and `ss_prefault_region()` immediately after `ss_os_acquire()`.

---

### 4. Remote Free & Background Processing

#### HAKMEM_TINY_REMOTE_DRAIN_THRESHOLD
- **Default**: 32
- **Purpose**: Trigger remote free drain when count exceeds threshold
- **Impact**: Controls when to process cross-thread frees
- **Per-class**: ACE can tune this per-class

#### HAKMEM_TINY_REMOTE_DRAIN_TRYRATE
- **Default**: 16
- **Purpose**: Probability (1/N) of attempting trylock drain
- **Impact**: Lower = more aggressive draining

### 5. Statistics & Profiling

#### HAKMEM_ENABLE_STATS (BUILD-TIME)
- **Default**: UNDEFINED (statistics DISABLED)
- **Purpose**: Enable batched TLS statistics collection
- **Build**: `make CFLAGS=-DHAKMEM_ENABLE_STATS`
- **Impact**: 0.5ns overhead per alloc/free when enabled
- **Critical**: Must be defined to see any statistics

#### HAKMEM_TINY_STAT_RATE_LG
- **Default**: 0 (no sampling)
- **Purpose**: Sample statistics at 1/2^N rate
- **Example**: `HAKMEM_TINY_STAT_RATE_LG=4` → sample 1/16 allocs
- **Requires**: HAKMEM_ENABLE_STATS + HAKMEM_TINY_STAT_SAMPLING build flags

#### HAKMEM_TINY_COUNT_SAMPLE
- **Default**: 8
- **Purpose**: Legacy sampling exponent (deprecated)
- **Note**: Replaced by batched stats in Phase 3

#### HAKMEM_TINY_PATH_DEBUG
- **Default**: 0
- **Purpose**: Enable allocation path debugging counters
- **Requires**: HAKMEM_DEBUG_COUNTERS=1 build flag
- **Output**: atexit() dump of path hit counts

---

### 6. ACE Learning System (Adaptive Control Engine)

#### HAKMEM_ACE_ENABLED
- **Default**: 0
- **Purpose**: Enable ACE learning system
- **Impact**: Adaptive tuning of Tiny Pool parameters
- **Note**: Already integrated but can be disabled

#### HAKMEM_ACE_OBSERVE
- **Default**: 0
- **Purpose**: Enable ACE observation logging
- **Impact**: Verbose output of ACE decisions

#### HAKMEM_ACE_DEBUG
- **Default**: 0
- **Purpose**: Enable ACE debug logging
- **Impact**: Detailed ACE internal state

#### HAKMEM_ACE_SAMPLE
- **Default**: Undefined (no sampling)
- **Purpose**: Sample ACE events at given rate
- **Impact**: Reduces ACE overhead

#### HAKMEM_ACE_LOG_LEVEL
- **Default**: 0
- **Purpose**: ACE logging verbosity (0-3)
- **Levels**: 0=off, 1=errors, 2=info, 3=debug

#### HAKMEM_ACE_FAST_INTERVAL_MS
- **Default**: 100ms
- **Purpose**: Fast ACE update interval
- **Impact**: How often ACE checks metrics

#### HAKMEM_ACE_SLOW_INTERVAL_MS
- **Default**: 1000ms
- **Purpose**: Slow ACE update interval
- **Impact**: Background tuning frequency

---

### 7. Intelligence Engine (INT)

#### HAKMEM_INT_ENGINE
- **Default**: 0
- **Purpose**: Enable background intelligence/adaptation engine
- **Impact**: Deferred event processing + adaptive tuning
- **Pairs with**: HAKMEM_TINY_FRONTEND

#### HAKMEM_INT_ADAPT_REFILL
- **Default**: 1 (when INT enabled)
- **Purpose**: Adapt REFILL_MAX dynamically (±16)
- **Impact**: Tunes refill sizes based on miss rate

#### HAKMEM_INT_ADAPT_CAPS
- **Default**: 1 (when INT enabled)
- **Purpose**: Adapt MAG/SLL capacities (±16/±32)
- **Impact**: Grows hot classes, shrinks cold ones

#### HAKMEM_INT_EVENT_TS
- **Default**: 0
- **Purpose**: Include timestamps in INT events
- **Impact**: Adds clock_gettime() overhead

#### HAKMEM_INT_SAMPLE
- **Default**: Undefined (no sampling)
- **Purpose**: Sample INT events at 1/2^N rate
- **Impact**: Reduces INT overhead on hot path

---

### 8. Frontend & Experimental Features

#### HAKMEM_TINY_FRONTEND
- **Default**: 0
- **Purpose**: Enable mimalloc-style frontend cache
- **Impact**: Adds FastCache layer before backend
- **Experimental**: A/B testing only

#### HAKMEM_TINY_FASTCACHE
- **Default**: 0
- **Purpose**: Low-level FastCache toggle
- **Impact**: Internal A/B switch

#### HAKMEM_TINY_QUICK
- **Default**: 0
- **Purpose**: Enable TinyQuickSlot (6-item single-cacheline stack)
- **Impact**: Ultra-fast path for ≤64B
- **Experimental**: Bench-only optimization

#### HAKMEM_TINY_HOTMAG (削除済み)
- 2025-12 cleanup: HotMag runtime ENVトグルは削除。HotMagはデフォルトOFF固定、ENVでの調整不可。

#### HAKMEM_TINY_HOTMAG_CAP (削除済み)
- 2025-12 cleanup: HotMag容量ENVを削除（固定値128）。

#### HAKMEM_TINY_HOTMAG_REFILL (削除済み)
- 2025-12 cleanup: HotMag refillバッチENVを削除（固定値32）。

#### HAKMEM_TINY_HOTMAG_C{0..7} (削除済み)
- 2025-12 cleanup: クラス別HotMag有効/無効ENVを削除（全クラス固定OFF）。

---

### 9. Tiny Front Routing

#### HAKMEM_TINY_PROFILE
- **Default**: `"conservative"`
- **Type**: string
- **Purpose**: Control Tiny Front (TLS SLL / FastCache) vs Pool/backend routing per Tiny class via a simple profile.
- **Profiles**:
  - `"conservative"`:
    - All classes (C0–C7) use `TINY_FIRST`: try Tiny Front first, then fallback to Pool/backend on miss.
  - `"hot"`:
    - C0–C3: `TINY_ONLY`  (small classes use Tiny exclusively via front gate)
    - C4–C6: `TINY_FIRST`
    - C7:    `POOL_ONLY`  (1KB headerless class uses Pool/backend)
  - `"off"`:
    - All classes `POOL_ONLY` (Tiny Front is fully disabled, Pool-only allocator behaviour).
  - `"full"`:
    - All classes `TINY_ONLY` (microbench-style, front gate always routes via Tiny).
- **Implementation**:
  - Box: `core/box/tiny_route_box.h` / `tiny_route_box.c` (per-class `g_tiny_route[8]` table).
  - Gate: `tiny_alloc_gate_fast()` reads `TinyRoutePolicy` and decides Tiny vs Pool on each allocation.

---

### 10. Superslab Tiering & Registry Control

#### HAKMEM_SS_TIER_DOWN_THRESHOLD
- **Default**: `0.25`
- **Range**: 0.0–1.0
- **Purpose**: SuperSlab 利用率がこの値以下になったときに、Tier を `HOT → DRAINING` に遷移させる下限。
- **Impact**:
  - DRAINING Tier の SuperSlab は新規割り当ての対象外となり、drain/解放候補として扱われる。
  - 利用率が低い SuperSlab への新規割り当てを避け、活発な SuperSlab に負荷を集中させる。

#### HAKMEM_SS_TIER_UP_THRESHOLD
- **Default**: `0.50`
- **Range**: 0.0–1.0
- **Purpose**: DRAINING Tier の SuperSlab 利用率がこの値以上になったときに `DRAINING → HOT` に戻す上限（ヒステリシス）。
- **Impact**:
  - Down/Up 閾値にギャップを持たせることで、Tier が HOT と DRAINING の間で頻繁に振動するのを防ぐ。
  - Sustained な利用増加が観測された SuperSlab のみ HOT に復帰させる。

---

### 11. Memory Efficiency & RSS Control

#### HAKMEM_TINY_RSS_BUDGET_KB
- **Default**: Unlimited
- **Purpose**: Total RSS budget for Tiny Pool (kB)
- **Impact**: When exceeded, shrinks MAG/SLL capacities
- **INT interaction**: Requires HAKMEM_INT_ENGINE=1

#### HAKMEM_TINY_INT_TIGHT
- **Default**: 0
- **Purpose**: Bias INT toward memory reduction
- **Impact**: Higher shrink thresholds, lower floor values

#### HAKMEM_TINY_DIET_STEP
- **Default**: 16
- **Purpose**: Capacity reduction step when over budget
- **Impact**: MAG -= step, SLL -= step×2

#### HAKMEM_TINY_CAP_FLOOR_C{0..7}
- **Default**: None (no floor)
- **Purpose**: Minimum MAG capacity per class
- **Example**: `HAKMEM_TINY_CAP_FLOOR_C0=64` (8B class min)
- **Impact**: Prevents INT from shrinking below floor

#### HAKMEM_TINY_MEM_DIET
- **Default**: 0
- **Purpose**: Enable memory diet mode (aggressive trimming)
- **Impact**: Reduces memory footprint at cost of performance

#### HAKMEM_TINY_SPILL_HYST
- **Default**: 0
- **Purpose**: Magazine spill hysteresis (avoid thrashing)
- **Impact**: Keep N extra items before spilling

---

### 11. Policy & Learning Parameters

#### HAKMEM_LEARN
- **Default**: 0 (OFF, unless HAKMEM_MODE=learning/research)
- **Purpose**: Legacy global learning toggle (CAP/WMAX Learner thread)
- **Impact**:
  - HAKMEM_LEARN が明示的に設定されている場合:
    - `0` → Learner 無効
    - `!=0` → Learner 有効
  - 未設定の場合:
    - `HAKMEM_MODE=learning` / `research` のときだけ Learner 有効
    - それ以外のモードでは Learner 無効（balanced/fast/minimal）
  - 実装: `core/box/learner_env_box.h`（学習レイヤ用 ENV Box）

#### HAKMEM_WMAX_MID
- **Default**: 256KB
- **Purpose**: Mid-size allocation working set max
- **Impact**: Pool cache size for mid-tier

#### HAKMEM_WMAX_LARGE
- **Default**: 2MB
- **Purpose**: Large allocation working set max
- **Impact**: Pool cache size for large-tier

#### HAKMEM_CAP_MID
- **Default**: Unlimited
- **Purpose**: Mid-tier pool capacity cap
- **Impact**: Maximum mid-tier pool size

#### HAKMEM_CAP_LARGE
- **Default**: Unlimited
- **Purpose**: Large-tier pool capacity cap
- **Impact**: Maximum large-tier pool size

#### HAKMEM_WMAX_LEARN
- **Default**: 0
- **Purpose**: Enable working set max learning
- **Impact**: Adaptively tune WMAX based on hit rate

#### HAKMEM_WMAX_CANDIDATES_MID
- **Default**: "128,256,512,1024"
- **Purpose**: Candidate WMAX values for mid-tier learning
- **Format**: Comma-separated KB values

#### HAKMEM_WMAX_CANDIDATES_LARGE
- **Default**: "1024,2048,4096,8192"
- **Purpose**: Candidate WMAX values for large-tier learning
- **Format**: Comma-separated KB values

#### HAKMEM_WMAX_ADOPT_PCT
- **Default**: 0.01 (1%)
- **Purpose**: Adoption threshold for WMAX candidates
- **Impact**: How much better to switch candidates

#### HAKMEM_TARGET_HIT_MID
- **Default**: 0.65 (65%)
- **Purpose**: Target hit rate for mid-tier
- **Impact**: Learning objective

#### HAKMEM_TARGET_HIT_LARGE
- **Default**: 0.55 (55%)
- **Purpose**: Target hit rate for large-tier
- **Impact**: Learning objective

#### HAKMEM_GAIN_W_MISS
- **Default**: 1.0
- **Purpose**: Learning gain weight for misses
- **Impact**: How much to penalize misses

---

### 11. THP (Transparent Huge Pages)

#### HAKMEM_THP
- **Default**: "auto"
- **Purpose**: THP policy (off/auto/on)
- **Values**:
  - "off" = MADV_NOHUGEPAGE for all
  - "auto" = ≥2MB → MADV_HUGEPAGE
  - "on" = MADV_HUGEPAGE for all ≥1MB

#### HAKMEM_THP_LEARN
- **Default**: 0
- **Purpose**: Enable THP policy learning
- **Impact**: Adaptively choose THP policy

#### HAKMEM_THP_CANDIDATES
- **Default**: "off,auto,on"
- **Purpose**: THP candidate policies for learning
- **Format**: Comma-separated

#### HAKMEM_THP_ADOPT_PCT
- **Default**: 0.015 (1.5%)
- **Purpose**: Adoption threshold for THP switch
- **Impact**: How much better to switch

---

### 12. L2/L25 Pool Configuration

#### HAKMEM_WRAP_L2
- **Default**: 0
- **Purpose**: Enable L2 pool wrapper bypass
- **Impact**: Allow L2 during wrapper calls

#### HAKMEM_WRAP_L25
- **Default**: 0
- **Purpose**: Enable L25 pool wrapper bypass
- **Impact**: Allow L25 during wrapper calls

#### HAKMEM_POOL_TLS_FREE
- **Default**: 1
- **Purpose**: Enable TLS-local free for L2 pool
- **Impact**: Lock-free fast path

#### HAKMEM_POOL_TLS_RING
- **Default**: 1
- **Purpose**: Enable TLS ring buffer for pool
- **Impact**: Batched cross-thread returns

#### HAKMEM_POOL_MIN_BUNDLE
- **Default**: 4
- **Purpose**: Minimum bundle size for L2 pool
- **Impact**: Batch refill size

#### HAKMEM_L25_MIN_BUNDLE
- **Default**: 4
- **Purpose**: Minimum bundle size for L25 pool
- **Impact**: Batch refill size

#### HAKMEM_L25_DZ
- **Default**: "64,256"
- **Purpose**: L25 size zones (comma-separated)
- **Format**: "size1,size2,..."

#### HAKMEM_L25_RUN_BLOCKS
- **Default**: 16
- **Purpose**: Run blocks per L25 slab
- **Impact**: Slab structure

#### HAKMEM_L25_RUN_FACTOR
- **Default**: 2
- **Purpose**: Run factor multiplier
- **Impact**: Slab allocation strategy

---

### 13. Debugging & Observability

#### HAKMEM_VERBOSE
- **Default**: 0
- **Purpose**: Enable verbose logging
- **Impact**: Detailed allocation logs

#### HAKMEM_QUIET
- **Default**: 0
- **Purpose**: Suppress all logging
- **Impact**: Overrides HAKMEM_VERBOSE

#### HAKMEM_TIMING
- **Default**: 0
- **Purpose**: Enable timing measurements
- **Impact**: Track allocation latency

#### HAKMEM_HIST_SAMPLE
- **Default**: 0
- **Purpose**: Size histogram sampling rate
- **Impact**: Track size distribution

#### HAKMEM_PROF
- **Default**: 0
- **Purpose**: Enable profiling mode
- **Impact**: Detailed performance tracking

#### HAKMEM_LOG_FILE
- **Default**: stderr
- **Purpose**: Redirect logs to file
- **Impact**: File path for logging output

---

### 14. Mode Presets

#### HAKMEM_MODE
- **Default**: "balanced"
- **Purpose**: High-level configuration preset
- **Values**:
  - "minimal" = malloc/mmap only
  - "fast" = pool fast-path + frozen learning
  - "balanced" = BigCache + ELO + Batch (default)
  - "learning" = ELO LEARN + adaptive
  - "research" = all features + verbose

#### HAKMEM_PRESET
- **Default**: None
- **Purpose**: Evolution preset (from PRESETS.md)
- **Impact**: Load predefined parameter set

#### HAKMEM_FREE_POLICY
- **Default**: "batch"
- **Purpose**: Free path policy
- **Values**: "batch", "keep", "adaptive"

---

### 15. Build-Time Flags (Not Environment Variables)

#### HAKMEM_ENABLE_STATS
- **Type**: Compiler flag (`-DHAKMEM_ENABLE_STATS`)
- **Default**: NOT DEFINED
- **Impact**: Completely disables statistics when absent
- **Critical**: Must be set to collect any statistics

#### HAKMEM_BUILD_RELEASE
- **Type**: Compiler flag
- **Default**: NOT DEFINED (= 0)
- **Impact**: When undefined, enables debug paths
- **Check**: `#if !HAKMEM_BUILD_RELEASE` = true when not set

#### HAKMEM_BUILD_DEBUG
- **Type**: Compiler flag
- **Default**: NOT DEFINED (= 0)
- **Impact**: Enables debug counters and logging

#### HAKMEM_DEBUG_COUNTERS
- **Type**: Compiler flag
- **Default**: 0
- **Impact**: Include path debug counters in build

#### HAKMEM_TINY_MINIMAL_FRONT
- **Type**: Compiler flag
- **Default**: 0
- **Impact**: Strip optional front-end layers (bench only)

#### HAKMEM_TINY_BENCH_FASTPATH
- **Type**: Compiler flag
- **Default**: 0
- **Impact**: Enable benchmark-optimized fast path

#### HAKMEM_TINY_BENCH_SLL_ONLY
- **Type**: Compiler flag
- **Default**: 0
- **Impact**: SLL-only mode (no magazines)

#### HAKMEM_USDT
- **Type**: Compiler flag
- **Default**: 0
- **Impact**: Enable USDT tracepoints for perf
- **Requires**: `<sys/sdt.h>` (systemtap-sdt-dev)

---

## NULL Return Path Analysis

### Why hak_tiny_alloc() Returns NULL

The Tiny Pool allocator returns NULL in these cases:

1. **Size > 1KB** (line 97)
   ```c
   if (class_idx < 0) return NULL;  // >1KB
   ```

2. **Wrapper Guard Active** (lines 88-91, only when `!HAKMEM_BUILD_RELEASE`)
   ```c
   #if !HAKMEM_BUILD_RELEASE
   if (!g_wrap_tiny_enabled && g_tls_in_wrapper != 0) return NULL;
   #endif
   ```
   **Note**: `HAKMEM_BUILD_RELEASE` is NOT defined by default!
   This guard is ACTIVE in your build and returns NULL during malloc recursion.

3. **Wrapper Context Empty** (line 73)
   ```c
   return NULL;  // empty → fallback to next allocator tier
   ```
   Called from `hak_tiny_alloc_wrapper()` when magazine is empty.

4. **Slow Path Exhaustion**
   When all of these fail in `hak_tiny_alloc_slow()`:
   - HotMag refill fails
   - TLS list empty
   - TLS slab refill fails
   - `hak_tiny_alloc_superslab()` returns NULL

### When Tiny Pool is Bypassed

Given `HAKMEM_WRAP_TINY=1` (default), Tiny Pool is still bypassed when:

1. **During wrapper recursion** (if `HAKMEM_BUILD_RELEASE` not set)
   - malloc() calls getenv()
   - getenv() calls malloc()
   - Guard returns NULL → falls back to L2/L25

2. **Size > 1KB**
   - Always falls through to L2 pool (1KB-32KB)

3. **All caches empty + SuperSlab allocation fails**
   - Magazine empty
   - SLL empty
   - Active slabs full
   - SuperSlab cannot allocate new slab
   - Falls back to L2/L25

---

## Memory Issue Diagnosis: 9GB Usage

### Current Symptoms
- bench_fragment_stress_long_hakmem: **9GB RSS**
- System allocator: **1.6MB RSS**
- Tiny Pool stats: `alloc=0, free=0, slab=0` (ZERO activity)

### Root Cause Analysis

#### Hypothesis #1: Statistics Disabled (CONFIRMED)
**Probability**: 100%

**Evidence**:
- `HAKMEM_ENABLE_STATS` not defined in Makefile
- All stats show 0 (no data collection)
- Code in `hakmem_tiny_stats.h:243-275` shows no-op when disabled

**Impact**:
- Cannot see if Tiny Pool is being used
- Cannot diagnose allocation patterns
- Blind to memory leaks

**Fix**:
```bash
make clean
make CFLAGS="-DHAKMEM_ENABLE_STATS" bench_fragment_stress_hakmem
```

#### Hypothesis #2: Wrapper Guard Blocking Tiny Pool
**Probability**: 90%

**Evidence**:
- `HAKMEM_BUILD_RELEASE` not defined → guard is ACTIVE
- Wrapper guard code at `hakmem_tiny_alloc.inc:86-92`
- During benchmark, many allocations may trigger wrapper context

**Mechanism**:
```c
#if !HAKMEM_BUILD_RELEASE  // This is TRUE (not defined)
if (!g_wrap_tiny_enabled && g_tls_in_wrapper != 0)
    return NULL;  // Bypass Tiny Pool!
#endif
```

**Result**:
- Tiny Pool returns NULL
- Falls back to L2/L25 pools
- L2/L25 may be leaking or over-allocating

**Fix**:
```bash
make CFLAGS="-DHAKMEM_BUILD_RELEASE=1"
```

#### Hypothesis #3: L2/L25 Pool Leak or Over-Retention
**Probability**: 75%

**Evidence**:
- If Tiny Pool is bypassed → L2/L25 handles ≤1KB allocations
- L2/L25 may have less aggressive trimming
- Fragment stress workload may trigger worst-case pooling

**Verification**:
1. Enable L2/L25 statistics
2. Check pool sizes: `g_pool_*` counters
3. Look for unbounded pool growth

**Fix**: Tune L2/L25 parameters:
```bash
export HAKMEM_POOL_TLS_FREE=1
export HAKMEM_CAP_MID=256  # Cap mid-tier pool at 256 blocks
```

---

## Recommended Diagnostic Steps

### Step 1: Enable Statistics
```bash
make clean
make CFLAGS="-DHAKMEM_ENABLE_STATS -DHAKMEM_BUILD_RELEASE=1" bench_fragment_stress_hakmem
```

### Step 2: Run with Diagnostics
```bash
export HAKMEM_WRAP_TINY=1
export HAKMEM_VERBOSE=1
./bench_fragment_stress_hakmem
```

### Step 3: Check Statistics
```bash
# In benchmark output, look for:
# - Tiny Pool stats (should be non-zero now)
# - L2/L25 pool stats
# - Cache hit rates
# - RSS growth pattern
```

### Step 4: Profile Memory
```bash
# Option A: Valgrind massif
valgrind --tool=massif --massif-out-file=massif.out ./bench_fragment_stress_hakmem
ms_print massif.out

# Option B: HAKMEM internal profiling
export HAKMEM_PROF=1
export HAKMEM_PROF_SAMPLE=100
./bench_fragment_stress_hakmem
```

### Step 5: Compare Allocator Tiers
```bash
# Force Tiny-only (disable L2/L25 fallback)
export HAKMEM_TINY_USE_SUPERSLAB=1
export HAKMEM_CAP_MID=0      # Disable mid-tier
export HAKMEM_CAP_LARGE=0    # Disable large-tier
./bench_fragment_stress_hakmem

# Check if RSS improves → L2/L25 is the problem
```

---

## Quick Reference: Must-Set Variables for Debugging

```bash
# Enable everything for debugging
export HAKMEM_WRAP_TINY=1              # Use Tiny Pool
export HAKMEM_VERBOSE=1                # See what's happening
export HAKMEM_ACE_DEBUG=1              # ACE diagnostics
export HAKMEM_TINY_PATH_DEBUG=1        # Path counters (if built with HAKMEM_DEBUG_COUNTERS)

# Build with statistics
make clean
make CFLAGS="-DHAKMEM_ENABLE_STATS -DHAKMEM_BUILD_RELEASE=1 -DHAKMEM_DEBUG_COUNTERS=1"
```

---

## Summary: Critical Variables for Your Issue

| Variable | Current | Should Be | Impact |
|----------|---------|-----------|--------|
| HAKMEM_ENABLE_STATS | undefined | `-DHAKMEM_ENABLE_STATS` | Enable statistics collection |
| HAKMEM_BUILD_RELEASE | undefined (=0) | `-DHAKMEM_BUILD_RELEASE=1` | Disable wrapper guard |
| HAKMEM_WRAP_TINY | 1 ✓ | 1 | Already correct |
| HAKMEM_VERBOSE | 0 | 1 | See allocation logs |

**Action**: Rebuild with both flags, then re-run benchmark to see real statistics.