hakmem/docs/analysis/PHASE26_HOT_PATH_ATOMIC_AUDIT.md

Phase 26: Hot Path Atomic Telemetry Prune - Audit & Plan

Date: 2025-12-16 Purpose: Identify and compile-out telemetry-only atomics in hot alloc/free paths Pattern: Follow Phase 24 (tiny_class_stats) + Phase 25 (g_free_ss_enter) Expected Gain: +2-3% cumulative improvement

---

Executive Summary

Goal: Remove all telemetry-only atomic_fetch_add/sub from hot paths (alloc/free direct paths).

Methodology:

1. Audit all atomics in core/ directory
2. Classify: CORRECTNESS (keep) vs TELEMETRY (compile-out)
3. Prioritize: HOT (direct alloc/free) > WARM (refill/spill) > COLD (init/shutdown)
4. Implement compile gates following Phase 24+25 pattern
5. A/B test each candidate independently

Status: Phase 25 complete (+1.07% GO). Starting Phase 26.

---

Classification Criteria

CORRECTNESS (Do NOT touch)

• Remote queue management: remote_count, remote_head, remote_tail
• Refcount/ownership: refcount, owner, in_use, active
• Lock/synchronization: lock, mutex, head, tail (queue atomics)
• Metadata: meta->used, meta->active, meta->tls_cached

TELEMETRY (Candidate for compile-out)

• Stats counters: *_stats, *_count, *_calls
• Diagnostics: *_trace, *_debug, *_diag, *_log
• Observability: *_enter, *_exit, *_hit, *_miss, *_attempt, *_success
• Metrics: g_metric_*, g_dbg_*, g_rel_*

---

Phase 26 Candidates: HOT PATH TELEMETRY ATOMICS

Priority A: Direct Free Path (tiny_superslab_free.inc.h)

1. g_free_ss_enter - ALREADY DONE (Phase 25)

• Status: GO (+1.07%)
• Location: core/tiny_superslab_free.inc.h:22
• Gate: HAKMEM_TINY_FREE_STATS_COMPILED
• Verdict: Keep compiled-out (default: 0)

2. c7_free_count - NEW CANDIDATE

• Location: core/tiny_superslab_free.inc.h:51
• Code: atomic_fetch_add_explicit(&c7_free_count, 1, memory_order_relaxed);
• Purpose: Debug counter for C7 free path diagnostics
• Path: HOT (free superslab fast path)
• Expected Gain: +0.3-0.8%
• Priority: HIGH
• Action: Create Phase 26A

3. g_hdr_mismatch_log - NEW CANDIDATE

• Location: core/tiny_superslab_free.inc.h:147
• Code: atomic_fetch_add_explicit(&g_hdr_mismatch_log, 1, memory_order_relaxed);
• Purpose: Log header validation mismatches (debug only)
• Path: HOT (free path validation)
• Expected Gain: +0.2-0.5%
• Priority: HIGH
• Action: Create Phase 26B

4. g_hdr_meta_mismatch - NEW CANDIDATE

• Location: core/tiny_superslab_free.inc.h:182
• Code: atomic_fetch_add_explicit(&g_hdr_meta_mismatch, 1, memory_order_relaxed);
• Purpose: Log metadata validation failures (debug only)
• Path: HOT (free path validation)
• Expected Gain: +0.2-0.5%
• Priority: HIGH
• Action: Create Phase 26C

---

Priority B: Direct Alloc Path

5. g_metric_bad_class_once - NEW CANDIDATE

• Location: core/hakmem_tiny_alloc.inc:22
• Code: atomic_fetch_add_explicit(&g_metric_bad_class_once, 1, memory_order_relaxed)
• Purpose: One-shot metric for bad class index (safety check)
• Path: HOT (alloc entry gate)
• Expected Gain: +0.1-0.3%
• Priority: MEDIUM
• Action: Create Phase 26D

6. g_hdr_meta_fast - NEW CANDIDATE

• Location: core/tiny_free_fast_v2.inc.h:181
• Code: atomic_fetch_add_explicit(&g_hdr_meta_fast, 1, memory_order_relaxed);
• Purpose: Fast-path header metadata hit counter (telemetry)
• Path: HOT (free_fast_v2 path)
• Expected Gain: +0.3-0.7%
• Priority: HIGH
• Action: Create Phase 26E

---

Priority C: Warm Path (Refill/Spill)

7. g_bg_spill_len - BORDERLINE

• Location: core/hakmem_tiny_bg_spill.h:32,44
• Code: atomic_fetch_add_explicit(&g_bg_spill_len[class_idx], ...)
• Purpose: Background spill queue length tracking
• Path: WARM (spill path)
• Expected Gain: +0.1-0.2%
• Priority: MEDIUM
• Note: May be CORRECTNESS if queue length is used for flow control
• Action: Review code, then decide (Phase 27+)

8. Unified Cache Stats - MULTIPLE ATOMICS

• Location: core/front/tiny_unified_cache.c (multiple lines)
• Variables: g_unified_cache_hits_global, g_unified_cache_misses_global, etc.
• Purpose: Unified cache hit/miss telemetry
• Path: WARM (cache layer)
• Expected Gain: +0.2-0.4%
• Priority: MEDIUM
• Action: Group into single Phase 27+ candidate

---

Phase 26 Implementation Plan

Phase 26A: c7_free_count Atomic Prune

Target: core/tiny_superslab_free.inc.h:51

Step 1: Add Build Flag

// core/hakmem_build_flags.h (after line 290)

// ------------------------------------------------------------
// Phase 26A: C7 Free Count Atomic Prune (Compile-out c7_free_count)
// ------------------------------------------------------------
// C7 Free Count: Compile gate (default OFF = compile-out)
// Set to 1 for research builds that need C7 free path diagnostics
// Target: c7_free_count atomic in core/tiny_superslab_free.inc.h:51
#ifndef HAKMEM_C7_FREE_COUNT_COMPILED
#  define HAKMEM_C7_FREE_COUNT_COMPILED 0
#endif

Step 2: Wrap Atomic with Compile Gate

// core/tiny_superslab_free.inc.h:51
#if HAKMEM_C7_FREE_COUNT_COMPILED
    extern _Atomic int c7_free_count;
    int count = atomic_fetch_add_explicit(&c7_free_count, 1, memory_order_relaxed);
#else
    int count = 0;  // No-op when compiled out
    (void)count;    // Suppress unused warning
#endif

Step 3: A/B Test (Build-Level)

# Baseline (compiled-out, default)
make clean && make -j bench_random_mixed_hakmem
./bench_random_mixed_hakmem > baseline_26a.txt

# Compiled-in (for comparison)
make clean && make -j EXTRA_CFLAGS='-DHAKMEM_C7_FREE_COUNT_COMPILED=1' bench_random_mixed_hakmem
./bench_random_mixed_hakmem > compiled_in_26a.txt

# Run full bench suite
./scripts/run_mixed_10_cleanenv.sh > bench_26a_baseline.txt
make clean && make -j EXTRA_CFLAGS='-DHAKMEM_C7_FREE_COUNT_COMPILED=1' bench_random_mixed_hakmem
./scripts/run_mixed_10_cleanenv.sh > bench_26a_compiled.txt

Step 4: Verdict

• GO: +0.5% or more → keep compiled-out (default: 0)
• NEUTRAL: ±0.5% → document, keep compiled-out for cleanliness
• NO-GO: -0.5% or worse → revert change

---

Phase 26B-E: Repeat Pattern

Follow same pattern for:

• 26B: g_hdr_mismatch_log (tiny_superslab_free.inc.h:147)
• 26C: g_hdr_meta_mismatch (tiny_superslab_free.inc.h:182)
• 26D: g_metric_bad_class_once (hakmem_tiny_alloc.inc:22)
• 26E: g_hdr_meta_fast (tiny_free_fast_v2.inc.h:181)

Each Phase:

1. Add HAKMEM_[NAME]_COMPILED flag to hakmem_build_flags.h
2. Wrap atomic with #if HAKMEM_[NAME]_COMPILED
3. Run A/B test (baseline vs compiled-in)
4. Measure improvement
5. Document verdict

---

Expected Cumulative Impact

Phase	Target Atomic	File	Expected Gain	Status
24	g_tiny_class_stats_*	tiny_class_stats_box.h	+0.93%	GO ✅
25	g_free_ss_enter	tiny_superslab_free.inc.h:22	+1.07%	GO ✅
26A	c7_free_count	tiny_superslab_free.inc.h:51	+0.3-0.8%	TBD
26B	g_hdr_mismatch_log	tiny_superslab_free.inc.h:147	+0.2-0.5%	TBD
26C	g_hdr_meta_mismatch	tiny_superslab_free.inc.h:182	+0.2-0.5%	TBD
26D	g_metric_bad_class_once	hakmem_tiny_alloc.inc:22	+0.1-0.3%	TBD
26E	g_hdr_meta_fast	tiny_free_fast_v2.inc.h:181	+0.3-0.7%	TBD
Total (24-26E)	-	-	+2.93-4.83%	-

Conservative Estimate: +3.0% cumulative improvement from hot-path atomic prune.

---

Next Steps

1. ✅ Audit complete (this document)
2. ⏳ Implement Phase 26A (c7_free_count)
3. ⏳ Run A/B test (baseline vs compiled-in)
4. ⏳ Document results in PHASE26A_C7_FREE_COUNT_RESULTS.md
5. ⏳ Repeat for 26B-E
6. ⏳ Create cumulative report

---

References

• Phase 24 Pattern: core/box/tiny_class_stats_box.h
• Phase 25 Pattern: core/tiny_superslab_free.inc.h:20-25
• Build Flags: core/hakmem_build_flags.h:274-290
• Mimalloc Principle: No atomics/observe in hot path

---

Notes

• DO NOT touch correctness atomics (remote_count, refcount, meta->used, etc.)
• ALWAYS A/B test each candidate independently (no batching)
• ALWAYS use build-level flags (compile-time, not runtime)
• FOLLOW Phase 24+25 pattern (#if COMPILED with default: 0)
• DOCUMENT all verdicts (GO/NEUTRAL/NO-GO)

mimalloc Gap Analysis: This work closes the "hot path atomic tax" gap identified in optimization roadmap.