Add SuperSlab Prefault Box with 4MB MAP_POPULATE bug fix

New Feature: ss_prefault_box.h
- Box for controlling SuperSlab page prefaulting policy
- ENV: HAKMEM_SS_PREFAULT (0=OFF, 1=POPULATE, 2=TOUCH)
- Default: OFF (safe mode until further optimization)

Bug Fix: 4MB MAP_POPULATE regression
- Problem: Fallback path allocated 4MB (2x size for alignment) with MAP_POPULATE
  causing 52x slower mmap (0.585ms → 30.6ms) and 35% throughput regression
- Solution: Remove MAP_POPULATE from 4MB allocation, apply madvise(MADV_WILLNEED)
  only to the aligned 2MB region after trimming prefix/suffix

Changes:
- core/box/ss_prefault_box.h: New prefault policy box (header-only)
- core/box/ss_allocation_box.c: Integrate prefault box, call ss_prefault_region()
- core/superslab_cache.c: Fix fallback path - no MAP_POPULATE on 4MB,
  always munmap prefix/suffix, use MADV_WILLNEED for 2MB only
- docs/specs/ENV_VARS*.md: Document HAKMEM_SS_PREFAULT

Performance:
- bench_random_mixed: 4.32M ops/s (regression fixed, slight improvement)
- bench_tiny_hot: 157M ops/s with prefault=1 (no crash)

Box Theory:
- OS layer (ss_os_acquire): "how to mmap"
- Prefault Box: "when to page-in"
- Allocation Box: "when to call prefault"

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-12-04 20:11:24 +09:00
parent a32d0fafd4
commit cba6f785a1
5 changed files with 153 additions and 12 deletions

View File

@ -76,11 +76,6 @@ void* ss_os_acquire(uint8_t size_class, size_t ss_size, uintptr_t ss_mask, int p
size_t alloc_size = ss_size * 2;
int flags = MAP_PRIVATE | MAP_ANONYMOUS;
#ifdef MAP_POPULATE
if (populate) {
flags |= MAP_POPULATE;
}
#endif
void* raw = mmap(NULL, alloc_size,
PROT_READ | PROT_WRITE,
flags,
@ -110,13 +105,15 @@ void* ss_os_acquire(uint8_t size_class, size_t ss_size, uintptr_t ss_mask, int p
}
size_t suffix_size = alloc_size - prefix_size - ss_size;
if (suffix_size > 0) {
if (populate) {
#ifdef MADV_DONTNEED
madvise((char*)ptr + ss_size, suffix_size, MADV_DONTNEED);
// 余剰領域は常に munmap して、実際に使用する SuperSlab サイズだけを残す。
munmap((char*)ptr + ss_size, suffix_size);
}
// populate が要求されている場合は、実際に使う SuperSlab 領域だけを事前 fault-in する。
if (populate) {
#ifdef MADV_WILLNEED
madvise(ptr, ss_size, MADV_WILLNEED);
#endif
} else {
munmap((char*)ptr + ss_size, suffix_size);
}
}
ss_stats_os_alloc(size_class, ss_size);