Add SuperSlab Prefault Box with 4MB MAP_POPULATE bug fix
New Feature: ss_prefault_box.h - Box for controlling SuperSlab page prefaulting policy - ENV: HAKMEM_SS_PREFAULT (0=OFF, 1=POPULATE, 2=TOUCH) - Default: OFF (safe mode until further optimization) Bug Fix: 4MB MAP_POPULATE regression - Problem: Fallback path allocated 4MB (2x size for alignment) with MAP_POPULATE causing 52x slower mmap (0.585ms → 30.6ms) and 35% throughput regression - Solution: Remove MAP_POPULATE from 4MB allocation, apply madvise(MADV_WILLNEED) only to the aligned 2MB region after trimming prefix/suffix Changes: - core/box/ss_prefault_box.h: New prefault policy box (header-only) - core/box/ss_allocation_box.c: Integrate prefault box, call ss_prefault_region() - core/superslab_cache.c: Fix fallback path - no MAP_POPULATE on 4MB, always munmap prefix/suffix, use MADV_WILLNEED for 2MB only - docs/specs/ENV_VARS*.md: Document HAKMEM_SS_PREFAULT Performance: - bench_random_mixed: 4.32M ops/s (regression fixed, slight improvement) - bench_tiny_hot: 157M ops/s with prefault=1 (no crash) Box Theory: - OS layer (ss_os_acquire): "how to mmap" - Prefault Box: "when to page-in" - Allocation Box: "when to call prefault" 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@ -3,6 +3,7 @@
|
||||
|
||||
#include "ss_allocation_box.h"
|
||||
#include "ss_os_acquire_box.h"
|
||||
#include "ss_prefault_box.h"
|
||||
#include "ss_cache_box.h"
|
||||
#include "ss_stats_box.h"
|
||||
#include "ss_ace_box.h"
|
||||
@ -167,11 +168,28 @@ SuperSlab* superslab_allocate(uint8_t size_class) {
|
||||
}
|
||||
|
||||
if (!ptr) {
|
||||
int populate = atomic_exchange_explicit(&g_ss_populate_once, 0, memory_order_acq_rel);
|
||||
// Prefault policy: decide MAP_POPULATE and optional manual touch
|
||||
SSPrefaultPolicy pf_policy = ss_prefault_policy();
|
||||
int populate = 0;
|
||||
|
||||
if (pf_policy == SS_PREFAULT_POPULATE ||
|
||||
pf_policy == SS_PREFAULT_TOUCH ||
|
||||
pf_policy == SS_PREFAULT_ASYNC) {
|
||||
// 常時 MAP_POPULATE(+必要なら手動 touch)
|
||||
populate = 1;
|
||||
} else {
|
||||
// OFF の場合のみ、従来の「ワンショット populate」挙動を温存
|
||||
populate = atomic_exchange_explicit(&g_ss_populate_once, 0, memory_order_acq_rel);
|
||||
}
|
||||
|
||||
ptr = ss_os_acquire(size_class, ss_size, ss_mask, populate);
|
||||
if (!ptr) {
|
||||
return NULL;
|
||||
}
|
||||
// 手動 prefault が要求されている場合は、mmap 直後に SuperSlab 全域を page-in
|
||||
if (pf_policy == SS_PREFAULT_TOUCH || pf_policy == SS_PREFAULT_ASYNC) {
|
||||
ss_prefault_region(ptr, ss_size);
|
||||
}
|
||||
// Debug logging for REFILL with new allocation
|
||||
if (dbg == 1) {
|
||||
fprintf(stderr, "[REFILL] class=%d new_alloc=1 ss=%p\n",
|
||||
|
||||
Reference in New Issue
Block a user