Files
hakmem/scripts/soak_mixed_single_process.sh

84 lines
2.7 KiB
Bash
Raw Normal View History

Phase 54-60: Memory-Lean mode, Balanced mode stabilization, M1 (50%) achievement ## Summary Completed Phase 54-60 optimization work: **Phase 54-56: Memory-Lean mode (LEAN+OFF prewarm suppression)** - Implemented ss_mem_lean_env_box.h with ENV gates - Balanced mode (LEAN+OFF) promoted as production default - Result: +1.2% throughput, better stability, zero syscall overhead - Added to bench_profile.h: MIXED_TINYV3_C7_BALANCED preset **Phase 57: 60-min soak finalization** - Balanced mode: 60-min soak, RSS drift 0%, CV 5.38% - Speed-first mode: 60-min soak, RSS drift 0%, CV 1.58% - Syscall budget: 1.25e-7/op (800× under target) - Status: PRODUCTION-READY **Phase 59: 50% recovery baseline rebase** - hakmem FAST (Balanced): 59.184M ops/s, CV 1.31% - mimalloc: 120.466M ops/s, CV 3.50% - Ratio: 49.13% (M1 ACHIEVED within statistical noise) - Superior stability: 2.68× better CV than mimalloc **Phase 60: Alloc pass-down SSOT (NO-GO)** - Implemented alloc_passdown_ssot_env_box.h - Modified malloc_tiny_fast.h for SSOT pattern - Result: -0.46% (NO-GO) - Key lesson: SSOT not applicable where early-exit already optimized ## Key Metrics - Performance: 49.13% of mimalloc (M1 effectively achieved) - Stability: CV 1.31% (superior to mimalloc 3.50%) - Syscall budget: 1.25e-7/op (excellent) - RSS: 33MB stable, 0% drift over 60 minutes ## Files Added/Modified New boxes: - core/box/ss_mem_lean_env_box.h - core/box/ss_release_policy_box.{h,c} - core/box/alloc_passdown_ssot_env_box.h Scripts: - scripts/soak_mixed_single_process.sh - scripts/analyze_epoch_tail_csv.py - scripts/soak_mixed_rss.sh - scripts/calculate_percentiles.py - scripts/analyze_soak.py Documentation: Phase 40-60 analysis documents ## Design Decisions 1. Profile separation (core/bench_profile.h): - MIXED_TINYV3_C7_SAFE: Speed-first (no LEAN) - MIXED_TINYV3_C7_BALANCED: Balanced mode (LEAN+OFF) 2. Box Theory compliance: - All ENV gates reversible (HAKMEM_SS_MEM_LEAN, HAKMEM_ALLOC_PASSDOWN_SSOT) - Single conversion points maintained - No physical deletions (compile-out only) 3. Lessons learned: - SSOT effective only where redundancy exists (Phase 60 showed limits) - Branch prediction extremely effective (~0 cycles for well-predicted branches) - Early-exit pattern valuable even when seemingly redundant 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-17 06:24:01 +09:00
#!/usr/bin/env bash
set -euo pipefail
# Single-process soak using bench epoch mode (Phase 51).
# - Keeps allocator state within one process.
# - Emits per-epoch throughput + current RSS (from /proc/self/statm).
#
# Output CSV columns:
# epoch,iter,throughput_ops_s,rss_mb
#
# Usage:
# make bench_random_mixed_hakmem_minimal
# BENCH_BIN=./bench_random_mixed_hakmem_minimal \
# DURATION_SEC=300 EPOCH_SEC=5 WS=400 \
# ./scripts/soak_mixed_single_process.sh > soak_single.csv
#
# Notes:
# - This script calibrates epoch iters from a short probe run (20M iters).
# - For reproducibility, it forces the same "clean env" knobs as run_mixed_10_cleanenv.sh.
bin=${BENCH_BIN:-./bench_random_mixed_hakmem_minimal}
ws=${WS:-400}
duration_sec=${DURATION_SEC:-300}
epoch_sec=${EPOCH_SEC:-5}
profile=${HAKMEM_PROFILE:-MIXED_TINYV3_C7_SAFE}
# Clean env (same policy as scripts/run_mixed_10_cleanenv.sh).
export HAKMEM_TINY_HEADER_WRITE_ONCE=${HAKMEM_TINY_HEADER_WRITE_ONCE:-0}
export HAKMEM_TINY_C7_PRESERVE_HEADER=${HAKMEM_TINY_C7_PRESERVE_HEADER:-0}
export HAKMEM_TINY_TCACHE=${HAKMEM_TINY_TCACHE:-0}
export HAKMEM_TINY_TCACHE_CAP=${HAKMEM_TINY_TCACHE_CAP:-64}
export HAKMEM_MALLOC_TINY_DIRECT=${HAKMEM_MALLOC_TINY_DIRECT:-0}
export HAKMEM_FRONT_FASTLANE_ALLOC_LEGACY_DIRECT=${HAKMEM_FRONT_FASTLANE_ALLOC_LEGACY_DIRECT:-0}
export HAKMEM_FORCE_LIBC_ALLOC=${HAKMEM_FORCE_LIBC_ALLOC:-0}
export HAKMEM_ENV_SNAPSHOT_SHAPE=${HAKMEM_ENV_SNAPSHOT_SHAPE:-0}
export HAKMEM_FASTLANE_DIRECT=${HAKMEM_FASTLANE_DIRECT:-1}
export HAKMEM_FREE_TINY_FAST_MONO_DUALHOT=${HAKMEM_FREE_TINY_FAST_MONO_DUALHOT:-1}
export HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT=${HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT:-1}
epochs=$((duration_sec / epoch_sec))
if (( epochs < 1 )); then
epochs=1
fi
probe_iters=20000000
probe_out=$(
HAKMEM_PROFILE="${profile}" "${bin}" "${probe_iters}" "${ws}" 1 2>&1 \
| rg -o "Throughput =\\s+[0-9]+\\s+ops/s" -m 1 \
| rg -o "[0-9]+"
)
if [[ -z "${probe_out}" ]]; then
echo "failed to probe throughput" >&2
exit 1
fi
epoch_iters=$(( (probe_out * epoch_sec) ))
if (( epoch_iters < 1 )); then
epoch_iters=1
fi
total_iters=$(( epoch_iters * epochs ))
echo "epoch,iter,throughput_ops_s,rss_mb"
HAKMEM_PROFILE="${profile}" \
HAKMEM_BENCH_EPOCH_ITERS="${epoch_iters}" \
"${bin}" "${total_iters}" "${ws}" 1 2>&1 \
| rg "^\\[EPOCH\\]" \
| awk '
{
# Example:
# [EPOCH] 0 Throughput = 12345678 ops/s [iter=500000000 ws=400] time=8.123s rss_kb=12345
epoch=$2;
thr=$5;
it=$7;
if (substr(it,1,6) == "[iter=") it=substr(it,7);
sub(/[^0-9].*$/, "", it);
rss_kb=$NF;
sub(/^rss_kb=/, "", rss_kb);
rss_mb=rss_kb/1024.0;
printf "%s,%s,%s,%.2f\n", epoch, it, thr, rss_mb;
}'