Files
hakmem/core/box/front_metrics_box.c

124 lines
5.1 KiB
C
Raw Normal View History

Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
// front_metrics_box.c - Box FrontMetrics Implementation
// Purpose: Collect and report frontend layer hit rates
#include "front_metrics_box.h"
#include "../hakmem_tiny_stats_api.h"
P0 Optimization: Shared Pool fast path with O(1) metadata lookup Performance Results: - Throughput: 2.66M ops/s → 3.8M ops/s (+43% improvement) - sp_meta_find_or_create: O(N) linear scan → O(1) direct pointer - Stage 2 metadata scan: 100% → 10-20% (80-90% reduction via hints) Core Optimizations: 1. O(1) Metadata Lookup (superslab_types.h) - Added `shared_meta` pointer field to SuperSlab struct - Eliminates O(N) linear search through ss_metadata[] array - First access: O(N) scan + cache | Subsequent: O(1) direct return 2. sp_meta_find_or_create Fast Path (hakmem_shared_pool.c) - Check cached ss->shared_meta first before linear scan - Cache pointer after successful linear scan for future lookups - Reduces 7.8% CPU hotspot to near-zero for hot paths 3. Stage 2 Class Hints Fast Path (hakmem_shared_pool_acquire.c) - Try class_hints[class_idx] FIRST before full metadata scan - Uses O(1) ss->shared_meta lookup for hint validation - __builtin_expect() for branch prediction optimization - 80-90% of acquire calls now skip full metadata scan 4. Proper Initialization (ss_allocation_box.c) - Initialize shared_meta = NULL in superslab_allocate() - Ensures correct NULL-check semantics for new SuperSlabs Additional Improvements: - Updated ptr_trace and debug ring for release build efficiency - Enhanced ENV variable documentation and analysis - Added learner_env_box.h for configuration management - Various Box optimizations for reduced overhead Thread Safety: - All atomic operations use correct memory ordering - shared_meta cached under mutex protection - Lock-free Stage 2 uses proper CAS with acquire/release semantics Testing: - Benchmark: 1M iterations, 3.8M ops/s stable - Build: Clean compile RELEASE=0 and RELEASE=1 - No crashes, memory leaks, or correctness issues Next Optimization Candidates: - P1: Per-SuperSlab free slot bitmap for O(1) slot claiming - P2: Reduce Stage 2 critical section size - P3: Page pre-faulting (MAP_POPULATE) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 16:21:54 +09:00
#include "../hakmem_stats_master.h" // Phase 4d: Master stats control
Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// ============================================================================
// Per-thread counters (NEW - declared in header, defined here)
// ============================================================================
__thread uint64_t g_front_ultrahot_hit[TINY_NUM_CLASSES] = {0};
__thread uint64_t g_front_ultrahot_miss[TINY_NUM_CLASSES] = {0};
__thread uint64_t g_front_heapv2_hit[TINY_NUM_CLASSES] = {0};
__thread uint64_t g_front_heapv2_miss[TINY_NUM_CLASSES] = {0};
__thread uint64_t g_front_class5_hit[TINY_NUM_CLASSES] = {0};
__thread uint64_t g_front_class5_miss[TINY_NUM_CLASSES] = {0};
// ============================================================================
// Existing counters (defined in hakmem_tiny.c, extern here for reading)
// ============================================================================
extern unsigned long long g_front_fc_hit[TINY_NUM_CLASSES];
extern unsigned long long g_front_fc_miss[TINY_NUM_CLASSES];
extern unsigned long long g_front_sfc_hit[TINY_NUM_CLASSES];
extern unsigned long long g_front_sll_hit[TINY_NUM_CLASSES];
// ============================================================================
// Enable flag (cached)
// ============================================================================
int front_metrics_enabled(void) {
static int g_enabled = -1;
if (__builtin_expect(g_enabled == -1, 0)) {
ENV cleanup: Add RELEASE guards to DEBUG ENV variables (14 vars) Added compile-time guards (#if HAKMEM_BUILD_RELEASE) to eliminate DEBUG ENV variable overhead in RELEASE builds. Variables guarded (14 total): - HAKMEM_TINY_TRACE_RING, HAKMEM_TINY_DUMP_RING_ATEXIT - HAKMEM_TINY_RF_TRACE, HAKMEM_TINY_MAILBOX_TRACE - HAKMEM_TINY_MAILBOX_TRACE_LIMIT, HAKMEM_TINY_MAILBOX_SLOWDISC - HAKMEM_TINY_MAILBOX_SLOWDISC_PERIOD - HAKMEM_SS_PREWARM_DEBUG, HAKMEM_SS_FREE_DEBUG - HAKMEM_TINY_FRONT_METRICS, HAKMEM_TINY_FRONT_DUMP - HAKMEM_TINY_COUNTERS_DUMP, HAKMEM_TINY_REFILL_DUMP - HAKMEM_PTR_TRACE_DUMP, HAKMEM_PTR_TRACE_VERBOSE Files modified (9 core files): - core/tiny_debug_ring.c (ring trace/dump) - core/box/mailbox_box.c (mailbox trace + slowdisc) - core/tiny_refill.h (refill trace) - core/hakmem_tiny_superslab.c (superslab debug) - core/box/ss_allocation_box.c (allocation debug) - core/tiny_superslab_free.inc.h (free debug) - core/box/front_metrics_box.c (frontend metrics) - core/hakmem_tiny_stats.c (stats dump) - core/ptr_trace.h (pointer trace) Bug fixes during implementation: 1. mailbox_box.c - Fixed variable scope (moved 'used' outside guard) 2. hakmem_tiny_stats.c - Fixed incomplete declarations (on1, on2) Impact: - Binary size: -85KB total - bench_random_mixed_hakmem: 319K → 305K (-14K, -4.4%) - larson_hakmem: 380K → 309K (-71K, -18.7%) - Performance: No regression (16.9-17.9M ops/s maintained) - Functional: All tests pass (Random Mixed + Larson) - Behavior: DEBUG ENV vars correctly ignored in RELEASE builds Testing: - Build: Clean compilation (warnings only, pre-existing) - 100K Random Mixed: 16.9-17.9M ops/s (PASS) - 10K Larson: 25.9M ops/s (PASS) - DEBUG ENV verification: Correctly ignored (PASS) Result: 14 DEBUG ENV variables now have zero overhead in RELEASE builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 03:41:07 +09:00
#if HAKMEM_BUILD_RELEASE
g_enabled = 0;
#else
Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
const char* env = getenv("HAKMEM_TINY_FRONT_METRICS");
g_enabled = (env && *env && *env != '0') ? 1 : 0;
ENV cleanup: Add RELEASE guards to DEBUG ENV variables (14 vars) Added compile-time guards (#if HAKMEM_BUILD_RELEASE) to eliminate DEBUG ENV variable overhead in RELEASE builds. Variables guarded (14 total): - HAKMEM_TINY_TRACE_RING, HAKMEM_TINY_DUMP_RING_ATEXIT - HAKMEM_TINY_RF_TRACE, HAKMEM_TINY_MAILBOX_TRACE - HAKMEM_TINY_MAILBOX_TRACE_LIMIT, HAKMEM_TINY_MAILBOX_SLOWDISC - HAKMEM_TINY_MAILBOX_SLOWDISC_PERIOD - HAKMEM_SS_PREWARM_DEBUG, HAKMEM_SS_FREE_DEBUG - HAKMEM_TINY_FRONT_METRICS, HAKMEM_TINY_FRONT_DUMP - HAKMEM_TINY_COUNTERS_DUMP, HAKMEM_TINY_REFILL_DUMP - HAKMEM_PTR_TRACE_DUMP, HAKMEM_PTR_TRACE_VERBOSE Files modified (9 core files): - core/tiny_debug_ring.c (ring trace/dump) - core/box/mailbox_box.c (mailbox trace + slowdisc) - core/tiny_refill.h (refill trace) - core/hakmem_tiny_superslab.c (superslab debug) - core/box/ss_allocation_box.c (allocation debug) - core/tiny_superslab_free.inc.h (free debug) - core/box/front_metrics_box.c (frontend metrics) - core/hakmem_tiny_stats.c (stats dump) - core/ptr_trace.h (pointer trace) Bug fixes during implementation: 1. mailbox_box.c - Fixed variable scope (moved 'used' outside guard) 2. hakmem_tiny_stats.c - Fixed incomplete declarations (on1, on2) Impact: - Binary size: -85KB total - bench_random_mixed_hakmem: 319K → 305K (-14K, -4.4%) - larson_hakmem: 380K → 309K (-71K, -18.7%) - Performance: No regression (16.9-17.9M ops/s maintained) - Functional: All tests pass (Random Mixed + Larson) - Behavior: DEBUG ENV vars correctly ignored in RELEASE builds Testing: - Build: Clean compilation (warnings only, pre-existing) - 100K Random Mixed: 16.9-17.9M ops/s (PASS) - 10K Larson: 25.9M ops/s (PASS) - DEBUG ENV verification: Correctly ignored (PASS) Result: 14 DEBUG ENV variables now have zero overhead in RELEASE builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 03:41:07 +09:00
#endif
Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
}
return g_enabled;
}
// ============================================================================
// Dump frontend metrics (CSV format)
// ============================================================================
void hak_tiny_front_metrics_dump(void) {
if (!front_metrics_enabled()) {
return;
}
P0 Optimization: Shared Pool fast path with O(1) metadata lookup Performance Results: - Throughput: 2.66M ops/s → 3.8M ops/s (+43% improvement) - sp_meta_find_or_create: O(N) linear scan → O(1) direct pointer - Stage 2 metadata scan: 100% → 10-20% (80-90% reduction via hints) Core Optimizations: 1. O(1) Metadata Lookup (superslab_types.h) - Added `shared_meta` pointer field to SuperSlab struct - Eliminates O(N) linear search through ss_metadata[] array - First access: O(N) scan + cache | Subsequent: O(1) direct return 2. sp_meta_find_or_create Fast Path (hakmem_shared_pool.c) - Check cached ss->shared_meta first before linear scan - Cache pointer after successful linear scan for future lookups - Reduces 7.8% CPU hotspot to near-zero for hot paths 3. Stage 2 Class Hints Fast Path (hakmem_shared_pool_acquire.c) - Try class_hints[class_idx] FIRST before full metadata scan - Uses O(1) ss->shared_meta lookup for hint validation - __builtin_expect() for branch prediction optimization - 80-90% of acquire calls now skip full metadata scan 4. Proper Initialization (ss_allocation_box.c) - Initialize shared_meta = NULL in superslab_allocate() - Ensures correct NULL-check semantics for new SuperSlabs Additional Improvements: - Updated ptr_trace and debug ring for release build efficiency - Enhanced ENV variable documentation and analysis - Added learner_env_box.h for configuration management - Various Box optimizations for reduced overhead Thread Safety: - All atomic operations use correct memory ordering - shared_meta cached under mutex protection - Lock-free Stage 2 uses proper CAS with acquire/release semantics Testing: - Benchmark: 1M iterations, 3.8M ops/s stable - Build: Clean compile RELEASE=0 and RELEASE=1 - No crashes, memory leaks, or correctness issues Next Optimization Candidates: - P1: Per-SuperSlab free slot bitmap for O(1) slot claiming - P2: Reduce Stage 2 critical section size - P3: Page pre-faulting (MAP_POPULATE) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 16:21:54 +09:00
#if !HAKMEM_BUILD_RELEASE
if (!hak_stats_check("HAKMEM_TINY_FRONT_DUMP", "front")) {
Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
return;
}
ENV cleanup: Add RELEASE guards to DEBUG ENV variables (14 vars) Added compile-time guards (#if HAKMEM_BUILD_RELEASE) to eliminate DEBUG ENV variable overhead in RELEASE builds. Variables guarded (14 total): - HAKMEM_TINY_TRACE_RING, HAKMEM_TINY_DUMP_RING_ATEXIT - HAKMEM_TINY_RF_TRACE, HAKMEM_TINY_MAILBOX_TRACE - HAKMEM_TINY_MAILBOX_TRACE_LIMIT, HAKMEM_TINY_MAILBOX_SLOWDISC - HAKMEM_TINY_MAILBOX_SLOWDISC_PERIOD - HAKMEM_SS_PREWARM_DEBUG, HAKMEM_SS_FREE_DEBUG - HAKMEM_TINY_FRONT_METRICS, HAKMEM_TINY_FRONT_DUMP - HAKMEM_TINY_COUNTERS_DUMP, HAKMEM_TINY_REFILL_DUMP - HAKMEM_PTR_TRACE_DUMP, HAKMEM_PTR_TRACE_VERBOSE Files modified (9 core files): - core/tiny_debug_ring.c (ring trace/dump) - core/box/mailbox_box.c (mailbox trace + slowdisc) - core/tiny_refill.h (refill trace) - core/hakmem_tiny_superslab.c (superslab debug) - core/box/ss_allocation_box.c (allocation debug) - core/tiny_superslab_free.inc.h (free debug) - core/box/front_metrics_box.c (frontend metrics) - core/hakmem_tiny_stats.c (stats dump) - core/ptr_trace.h (pointer trace) Bug fixes during implementation: 1. mailbox_box.c - Fixed variable scope (moved 'used' outside guard) 2. hakmem_tiny_stats.c - Fixed incomplete declarations (on1, on2) Impact: - Binary size: -85KB total - bench_random_mixed_hakmem: 319K → 305K (-14K, -4.4%) - larson_hakmem: 380K → 309K (-71K, -18.7%) - Performance: No regression (16.9-17.9M ops/s maintained) - Functional: All tests pass (Random Mixed + Larson) - Behavior: DEBUG ENV vars correctly ignored in RELEASE builds Testing: - Build: Clean compilation (warnings only, pre-existing) - 100K Random Mixed: 16.9-17.9M ops/s (PASS) - 10K Larson: 25.9M ops/s (PASS) - DEBUG ENV verification: Correctly ignored (PASS) Result: 14 DEBUG ENV variables now have zero overhead in RELEASE builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-27 03:41:07 +09:00
#endif
Phase 19 & 20-1: Frontend optimization + TLS cache prewarm (+16.2% total) Phase 19: Box FrontMetrics & Box FrontPrune (A/B testing framework) ======================================================================== - Box FrontMetrics: Per-class hit rate measurement for all frontend layers - Implementation: core/box/front_metrics_box.{h,c} - ENV: HAKMEM_TINY_FRONT_METRICS=1, HAKMEM_TINY_FRONT_DUMP=1 - Output: CSV format per-class hit rate report - A/B Test Results (Random Mixed 16-1040B, 500K iterations): | Config | Throughput | vs Baseline | C2/C3 Hit Rate | |--------|-----------|-------------|----------------| | Baseline (UH+HV2) | 10.1M ops/s | - | UH=11.7%, HV2=88.3% | | HeapV2 only | 11.4M ops/s | +12.9% ⭐ | HV2=99.3%, SLL=0.7% | | UltraHot only | 6.6M ops/s | -34.4% ❌ | UH=96.4%, SLL=94.2% | - Key Finding: UltraHot removal improves performance by +12.9% - Root cause: Branch prediction miss cost > UltraHot hit rate benefit - UltraHot check: 88.3% cases = wasted branch → CPU confusion - HeapV2 alone: more predictable → better pipeline efficiency - Default Setting Change: UltraHot default OFF - Production: UltraHot OFF (fastest) - Research: HAKMEM_TINY_FRONT_ENABLE_ULTRAHOT=1 to enable - Code preserved (not deleted) for research/debug use Phase 20-1: Box SS-HotPrewarm (TLS cache prewarming, +3.3%) ======================================================================== - Box SS-HotPrewarm: ENV-controlled per-class TLS cache prewarm - Implementation: core/box/ss_hot_prewarm_box.{h,c} - Default targets: C2/C3=128, C4/C5=64 (aggressive prewarm) - ENV: HAKMEM_TINY_PREWARM_C2, _C3, _C4, _C5, _ALL - Total: 384 blocks pre-allocated - Benchmark Results (Random Mixed 256B, 500K iterations): | Config | Page Faults | Throughput | vs Baseline | |--------|-------------|------------|-------------| | Baseline (Prewarm OFF) | 10,399 | 15.7M ops/s | - | | Phase 20-1 (Prewarm ON) | 10,342 | 16.2M ops/s | +3.3% ⭐ | - Page fault reduction: 0.55% (expected: 50-66%, reality: minimal) - Performance gain: +3.3% (15.7M → 16.2M ops/s) - Analysis: ❌ Page fault reduction failed: - User page-derived faults dominate (benchmark initialization) - 384 blocks prewarm = minimal impact on 10K+ total faults - Kernel-side cost (asm_exc_page_fault) uncontrollable from userspace ✅ Cache warming effect succeeded: - TLS SLL pre-filled → reduced initial refill cost - CPU cycle savings → +3.3% performance gain - Stability improvement: warm state from first allocation - Decision: Keep as "light +3% box" - Prewarm valid: 384 blocks (C2/C3=128, C4/C5=64) preserved - No further aggressive scaling: RSS cost vs page fault reduction unbalanced - Next phase: BenchFast mode for structural upper limit measurement Combined Performance Impact: ======================================================================== Phase 19 (HeapV2 only): +12.9% (10.1M → 11.4M ops/s) Phase 20-1 (Prewarm ON): +3.3% (15.7M → 16.2M ops/s) Total improvement: +16.2% vs original baseline Files Changed: ======================================================================== Phase 19: - core/box/front_metrics_box.{h,c} - NEW - core/tiny_alloc_fast.inc.h - metrics + ENV gating - PHASE19_AB_TEST_RESULTS.md - NEW (detailed A/B test report) - PHASE19_FRONTEND_METRICS_FINDINGS.md - NEW (findings report) Phase 20-1: - core/box/ss_hot_prewarm_box.{h,c} - NEW - core/box/hak_core_init.inc.h - prewarm call integration - Makefile - ss_hot_prewarm_box.o added - CURRENT_TASK.md - Phase 19 & 20-1 results documented 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 05:48:59 +09:00
fprintf(stderr, "\n========== Box FrontMetrics: Layer Hit Rates ==========\n");
fprintf(stderr, "Purpose: Identify which frontend layers are doing real work\n");
fprintf(stderr, "Legend: UH=UltraHot, HV2=HeapV2, C5=Class5, FC=FastCache, SFC=SuperFrontCache, SLL=TLS_SLL\n\n");
fprintf(stderr, "%-5s %10s %10s %10s %10s %10s %10s %12s | %6s %6s %6s %6s %6s %6s\n",
"Class", "UH_hit", "HV2_hit", "C5_hit", "FC_hit", "SFC_hit", "SLL_hit", "Total",
"UH%", "HV2%", "C5%", "FC%", "SFC%", "SLL%");
fprintf(stderr, "------|----------|----------|----------|----------|----------|----------|-------------|");
fprintf(stderr, "-------|-------|-------|-------|-------|-------\n");
for (int cls = 0; cls < TINY_NUM_CLASSES; cls++) {
uint64_t uh_hit = g_front_ultrahot_hit[cls];
uint64_t hv2_hit = g_front_heapv2_hit[cls];
uint64_t c5_hit = g_front_class5_hit[cls];
uint64_t fc_hit = g_front_fc_hit[cls];
uint64_t sfc_hit = g_front_sfc_hit[cls];
uint64_t sll_hit = g_front_sll_hit[cls];
uint64_t total = uh_hit + hv2_hit + c5_hit + fc_hit + sfc_hit + sll_hit;
if (total == 0) {
fprintf(stderr, "C%-4d %10s %10s %10s %10s %10s %10s %12s | %6s %6s %6s %6s %6s %6s\n",
cls, "-", "-", "-", "-", "-", "-", "-", "-", "-", "-", "-", "-", "-");
continue;
}
double uh_pct = (double)uh_hit / total * 100.0;
double hv2_pct = (double)hv2_hit / total * 100.0;
double c5_pct = (double)c5_hit / total * 100.0;
double fc_pct = (double)fc_hit / total * 100.0;
double sfc_pct = (double)sfc_hit / total * 100.0;
double sll_pct = (double)sll_hit / total * 100.0;
fprintf(stderr, "C%-4d %10lu %10lu %10lu %10lu %10lu %10lu %12lu | %5.1f%% %5.1f%% %5.1f%% %5.1f%% %5.1f%% %5.1f%%\n",
cls,
(unsigned long)uh_hit,
(unsigned long)hv2_hit,
(unsigned long)c5_hit,
(unsigned long)fc_hit,
(unsigned long)sfc_hit,
(unsigned long)sll_hit,
(unsigned long)total,
uh_pct, hv2_pct, c5_pct, fc_pct, sfc_pct, sll_pct);
}
fprintf(stderr, "=======================================================\n\n");
// Analysis recommendations
fprintf(stderr, "Analysis Recommendations:\n");
fprintf(stderr, " - Layers with >80%% hit rate: Keep and optimize (hot path)\n");
fprintf(stderr, " - Layers with <5%% hit rate: Consider pruning (dead weight)\n");
fprintf(stderr, " - Multiple layers >20%%: Potential redundancy, test pruning\n\n");
}
// Register dump at shutdown
static void front_metrics_atexit(void) __attribute__((destructor));
static void front_metrics_atexit(void) {
hak_tiny_front_metrics_dump();
}