tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 3c: L1D Prefetch Optimization (+10.4% throughput)

Browse Source 
Added software prefetch directives to reduce L1D cache miss penalty.

Changes:
- Refill path: Prefetch SuperSlab hot fields (slab_bitmap, total_active_blocks)
- Refill path: Prefetch SlabMeta freelist and next freelist entry
- Alloc path: Early prefetch of TLS cache head/count
- Alloc path: Prefetch next pointer after SLL pop

Results (Random Mixed 256B, 1M ops):
- Throughput: 22.7M → 25.05M ops/s (+10.4%)
- Cycles: 189.7M → 182.6M (-3.7%)
- Instructions: 285.0M → 280.4M (-1.6%)
- IPC: 1.50 → 1.54 (+2.7%)
- L1-dcache loads: 116.0M → 109.9M (-5.3%)

Files:
- core/hakmem_tiny_refill_p0.inc.h: 3 prefetch sites
- core/tiny_alloc_fast.inc.h: 3 prefetch sites

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-11-19 23:11:27 +09:00
parent 5b36c1c908
commit 437df708ed
2 changed files with 56 additions and 213 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
core/hakmem_tiny_refill_p0.inc.h
View File

@ -58,6 +58,13 @@ static inline int sll_refill_batch_from_ss(int class_idx, int max_take) {
}
TinyTLSSlab* tls = &g_tls_slabs[class_idx];
// Phase 3c L1D Opt: Prefetch SuperSlab hot fields early
if (tls->ss) {
__builtin_prefetch(&tls->ss->slab_bitmap, 0, 3);
__builtin_prefetch(&tls->ss->total_active_blocks, 0, 3);
}
uint32_t active_before = 0;
if (tls->ss) {
active_before = atomic_load_explicit(&tls->ss->total_active_blocks, memory_order_relaxed);
@ -77,6 +84,9 @@ static inline int sll_refill_batch_from_ss(int class_idx, int max_take) {
return 0;
}
// Phase 3c L1D Opt: Prefetch SlabMeta hot fields (freelist, used, capacity)
__builtin_prefetch(&meta->freelist, 0, 3);
#if HAKMEM_INTEGRITY_LEVEL >= 4
uint8_t* initial_slab_base =
tls->slab_base ? tls->slab_base : tiny_slab_base_for(tls->ss, tls->slab_idx);
@ -224,6 +234,12 @@ static inline int sll_refill_batch_from_ss(int class_idx, int max_take) {
&g_tls_sll_count[class_idx]);
ss_active_add(tls->ss, from_freelist);
meta->used = (uint16_t)((uint32_t)meta->used + from_freelist);
// Phase 3c L1D Opt: Prefetch next freelist entry after refill
if (meta->freelist) {
__builtin_prefetch(meta->freelist, 0, 3);
}
#if HAKMEM_DEBUG_COUNTERS
extern unsigned long long g_rf_freelist_items[];
g_rf_freelist_items[class_idx] += from_freelist;