Phase 4-Step2: Add Hot/Cold Path Box (+7.3% performance)

Implemented Hot/Cold Path separation using Box pattern for Tiny allocations:

Performance Improvement (without PGO):
- Baseline (Phase 26-A):     53.3 M ops/s
- Hot/Cold Box (Phase 4-Step2): 57.2 M ops/s
- Gain: +7.3% (+3.9 M ops/s)

Implementation:
1. core/box/tiny_front_hot_box.h - Ultra-fast hot path (1 branch)
   - Removed range check (caller guarantees valid class_idx)
   - Inline cache hit path with branch prediction hints
   - Debug metrics with zero overhead in Release builds

2. core/box/tiny_front_cold_box.h - Slow cold path (noinline, cold)
   - Refill logic (batch allocation from SuperSlab)
   - Drain logic (batch free to SuperSlab)
   - Error reporting and diagnostics

3. core/front/malloc_tiny_fast.h - Updated to use Hot/Cold Boxes
   - Hot path: tiny_hot_alloc_fast() (1 branch: cache empty check)
   - Cold path: tiny_cold_refill_and_alloc() (noinline, cold attribute)
   - Clear separation improves i-cache locality

Branch Analysis:
- Baseline: 4-5 branches in hot path (range check + cache check + refill logic mixed)
- Hot/Cold Box: 1 branch in hot path (cache empty check only)
- Reduction: 3-4 branches eliminated from hot path

Design Principles (Box Pattern):
 Single Responsibility: Hot path = cache hit only, Cold path = refill/errors
 Clear Contract: Hot returns NULL on miss, Cold handles miss
 Observable: Debug metrics (TINY_HOT_METRICS_*) gated by NDEBUG
 Safe: Branch prediction hints (TINY_HOT_LIKELY/UNLIKELY)
 Testable: Isolated hot/cold paths, easy A/B testing

PGO Status:
- Temporarily disabled (build issues with __gcov_merge_time_profile)
- Will re-enable PGO in future commit after resolving gcc/lto issues
- Current benchmarks are without PGO (fair A/B comparison)

Other Changes:
- .gitignore: Added *.d files (dependency files, auto-generated)
- Makefile: PGO targets temporarily disabled (show informational message)
- build_pgo.sh: Temporarily disabled (show "PGO paused" message)

Next: Phase 4-Step3 (Front Config Box, target +5-8%)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-11-29 11:58:37 +09:00
parent 24fad8f72f
commit 04186341c1
6 changed files with 449 additions and 201 deletions

View File

@ -34,6 +34,8 @@
#include "tiny_unified_cache.h" // For unified_cache_pop_or_refill
#include "../tiny_region_id.h" // For tiny_region_id_write_header
#include "../hakmem_tiny.h" // For hak_tiny_size_to_class
#include "../box/tiny_front_hot_box.h" // Phase 4-Step2: Hot Path Box
#include "../box/tiny_front_cold_box.h" // Phase 4-Step2: Cold Path Box
// Helper: current thread id (low 32 bits) for owner check
#ifndef TINY_SELF_U32_LOCAL_DEFINED
@ -64,42 +66,49 @@ static inline int front_gate_unified_enabled(void) {
}
// ============================================================================
// Phase 26-A: malloc_tiny_fast() - Ultra-thin Tiny allocation
// Phase 4-Step2: malloc_tiny_fast() - Hot/Cold Path Box (ACTIVE)
// ============================================================================
// Single-layer Tiny allocation (bypasses hak_alloc_at + wrapper + diagnostics)
// Ultra-thin Tiny allocation using Hot/Cold Path Box (Phase 4-Step2)
//
// IMPROVEMENTS over Phase 26-A:
// - Branch reduction: Hot path has only 1 branch (cache empty check)
// - Branch hints: TINY_HOT_LIKELY/UNLIKELY for better CPU prediction
// - Hot/Cold separation: Keeps hot path small (better i-cache locality)
// - Explicit fallback: Clear hot→cold transition
//
// PERFORMANCE:
// - Baseline (Phase 26-A, no PGO): 53.3 M ops/s
// - Hot/Cold Box (no PGO): 57.2 M ops/s (+7.3%)
//
// DESIGN:
// 1. size → class_idx (same as Phase 26-A)
// 2. Hot path: tiny_hot_alloc_fast() - cache hit (1 branch)
// 3. Cold path: tiny_cold_refill_and_alloc() - cache miss (noinline, cold)
//
// Preconditions:
// - Called AFTER malloc() safety checks (lock depth, initializing, LD_SAFE)
// - size <= tiny_get_max_size() (caller verified)
// Returns:
// - USER pointer on success
// - NULL on Unified Cache miss (caller falls back to normal path)
// - NULL on failure (caller falls back to normal path)
//
__attribute__((always_inline))
static inline void* malloc_tiny_fast(size_t size) {
// 1. size → class_idx (inline table lookup, 1-2 instructions)
int class_idx = hak_tiny_size_to_class(size);
if (__builtin_expect(class_idx < 0 || class_idx >= TINY_NUM_CLASSES, 0)) {
return NULL; // Out of range (should not happen if caller checked tiny_get_max_size())
// 2. Phase 4-Step2: Hot/Cold Path Box
// Try hot path first (cache hit, 1 branch)
void* ptr = tiny_hot_alloc_fast(class_idx);
if (TINY_HOT_LIKELY(ptr != NULL)) {
// Hot path: Cache hit → return USER pointer
return ptr;
}
// 2. Phase 23: Unified Cache pop-or-refill (tcache-style, 2-3 cache misses)
// This internally handles:
// - Cache hit: direct pop (fast path)
// - Cache miss: batch refill from SuperSlab (slow path)
void* base = unified_cache_pop_or_refill(class_idx);
if (__builtin_expect(base == NULL, 0)) {
// Unified Cache disabled OR refill failed
// Fall back to normal path (caller handles via hak_alloc_at)
return NULL;
}
// 3. Write header + return USER pointer (2-3 instructions)
#ifdef HAKMEM_TINY_HEADER_CLASSIDX
tiny_region_id_write_header(base, class_idx); // Write 1-byte header (BASE first!)
return (void*)((char*)base + 1); // Return USER pointer
#else
return base; // No header mode - return BASE directly
#endif
// 3. Cold path: Cache miss → refill + alloc
// noinline, cold attribute keeps this code out of hot path
return tiny_cold_refill_and_alloc(class_idx);
}
// ============================================================================