CRITICAL FIX: TLS 未初期化による 4T SEGV を完全解消

**問題:** - Larson 4T で 100% SEGV (1T は 2.09M ops/s で完走) - System/mimalloc は 4T で 33.52M ops/s 正常動作 - SS OFF + Remote OFF でも 4T で SEGV **根本原因: (Task agent ultrathink 調査結果)** ``` CRASH: mov (%r15),%r13 R15 = 0x6261 ← ASCII "ba" (ゴミ値、未初期化TLS) ``` Worker スレッドの TLS 変数が未初期化: - `__thread void* g_tls_sll_head[TINY_NUM_CLASSES];` ← 初期化なし - pthread_create() で生成されたスレッドでゼロ初期化されない - NULL チェックが通過 (0x6261 != NULL) → dereference → SEGV **修正内容:** 全 TLS 配列に明示的初期化子 `= {0}` を追加: 1. **core/hakmem_tiny.c:** - `g_tls_sll_head[TINY_NUM_CLASSES] = {0}` - `g_tls_sll_count[TINY_NUM_CLASSES] = {0}` - `g_tls_live_ss[TINY_NUM_CLASSES] = {0}` - `g_tls_bcur[TINY_NUM_CLASSES] = {0}` - `g_tls_bend[TINY_NUM_CLASSES] = {0}` 2. **core/tiny_fastcache.c:** - `g_tiny_fast_cache[TINY_FAST_CLASS_COUNT] = {0}` - `g_tiny_fast_count[TINY_FAST_CLASS_COUNT] = {0}` - `g_tiny_fast_free_head[TINY_FAST_CLASS_COUNT] = {0}` - `g_tiny_fast_free_count[TINY_FAST_CLASS_COUNT] = {0}` 3. **core/hakmem_tiny_magazine.c:** - `g_tls_mags[TINY_NUM_CLASSES] = {0}` 4. **core/tiny_sticky.c:** - `g_tls_sticky_ss[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}` - `g_tls_sticky_idx[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}` - `g_tls_sticky_pos[TINY_NUM_CLASSES] = {0}` **効果:** ``` Before: 1T: 2.09M ✅ | 4T: SEGV 💀 After: 1T: 2.41M ✅ | 4T: 4.19M ✅ (+15% 1T, SEGV解消) ``` **テスト:** ```bash # 1 thread: 完走 ./larson_hakmem 2 8 128 1024 1 12345 1 → Throughput = 2,407,597 ops/s ✅ # 4 threads: 完走（以前は SEGV） ./larson_hakmem 2 8 128 1024 1 12345 4 → Throughput = 4,192,155 ops/s ✅ ``` **調査協力:** Task agent (ultrathink mode) による完璧な根本原因特定 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 01:27:04 +09:00
parent f454d35ea4
commit 1da8754d45
110 changed files with 17703 additions and 1693 deletions
--- a/scripts/build_larson_dev.sh
+++ b/scripts/build_larson_dev.sh
@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# build_larson_dev.sh — deterministic dev builds for Larson (Tiny)
+#
+# Usage:
+#  scripts/build_larson_dev.sh [--route] [--frontgate] [--clean]
+#
+# Profiles (defaults):
+#  - NEW_3LAYER_DEFAULT=1 (3-layer front)
+#  - BOX_REFACTOR_DEFAULT=1 (box refactor on)
+#  - USE_LTO=0 OPT_LEVEL=1 (debuggability)
+#  - Adds EXTRA_CFLAGS based on flags:
+#      --route     → -DHAKMEM_ROUTE=1 (alloc/free route fingerprint)
+#      --frontgate → -DHAKMEM_TINY_FRONT_GATE_BOX=1 (Front Gate Box)
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)"
+cd "$ROOT_DIR"
+
+CLEAN=0
+ROUTE=0
+FRONTGATE=0
+for a in "$@"; do
+  case "$a" in
+    --clean) CLEAN=1 ;;
+    --route) ROUTE=1 ;;
+    --frontgate) FRONTGATE=1 ;;
+    *) echo "Unknown arg: $a" >&2; exit 2 ;;
+  esac
+done
+
+[[ $CLEAN -eq 1 ]] && make clean
+
+XCF=()
+[[ $ROUTE -eq 1 ]] && XCF+=(" -DHAKMEM_ROUTE=1")
+[[ $FRONTGATE -eq 1 ]] && XCF+=(" -DHAKMEM_TINY_FRONT_GATE_BOX=1")
+
+echo "[build] NEW_3LAYER_DEFAULT=1 BOX_REFACTOR_DEFAULT=1 USE_LTO=0 OPT_LEVEL=1"
+[[ $ROUTE -eq 1 ]] && echo "[build] EXTRA_CFLAGS+=-DHAKMEM_ROUTE=1"
+[[ $FRONTGATE -eq 1 ]] && echo "[build] EXTRA_CFLAGS+=-DHAKMEM_TINY_FRONT_GATE_BOX=1"
+
+make NEW_3LAYER_DEFAULT=1 BOX_REFACTOR_DEFAULT=1 USE_LTO=0 OPT_LEVEL=1 \
+  EXTRA_CFLAGS+="${XCF[*]}" larson_hakmem
+
+echo ""
+echo "✓ Built ./larson_hakmem (dev config)"
+echo "Quick run (tput mode):"
+echo "  HAKMEM_QUIET=1 HAKMEM_TINY_SUKESUKE=0 HAKMEM_TINY_TRACE_RING=0 \\"
+echo "  HAKMEM_TINY_FREE_TO_SS=0 HAKMEM_TINY_MUST_ADOPT=0 HAKMEM_TINY_REG_SCAN_MAX=64 \\"
+echo "  ./larson_hakmem 10 8 128 1024 1 12345 4"
+echo ""
+echo "Quick run (pf/sys mode):"
+echo "  HAKMEM_QUIET=1 HAKMEM_TINY_SUKESUKE=0 HAKMEM_TINY_TRACE_RING=0 \\"
+echo "  HAKMEM_TINY_FREE_TO_SS=1 HAKMEM_TINY_MUST_ADOPT=1 HAKMEM_TINY_SS_ADOPT_COOLDOWN=64 HAKMEM_TINY_REG_SCAN_MAX=32 \\"
+echo "  ./larson_hakmem 10 8 128 1024 1 12345 4"
+
--- a/scripts/cleanup_workspace.sh
+++ b/scripts/cleanup_workspace.sh
@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# cleanup_workspace.sh — Archive logs and remove build artifacts
+# - Archives logs to archive/cleanup_YYYYmmdd_HHMMSS/{logs}
+# - Runs make clean
+# - Removes re-buildable bench binaries and helper copies
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)"
+cd "$ROOT_DIR"
+
+ts="$(date +%Y%m%d_%H%M%S)"
+DEST="archive/cleanup_${ts}"
+mkdir -p "${DEST}/logs"
+
+log_patterns=(
+  "*out.txt"
+  "*stdout.log"
+  "*stderr.log"
+  "ring*.txt"
+  "asan_*.log"
+  "run_*.log"
+)
+
+echo "[cleanup] Archiving logs to ${DEST}/logs" | tee "${DEST}/CLEANUP_SUMMARY.txt"
+for pat in "${log_patterns[@]}"; do
+  shopt -s nullglob
+  for f in $pat; do
+    if [[ -f "$f" ]]; then
+      echo "log: $f" >> "${DEST}/LOGS_LIST.txt"
+      mv -f "$f" "${DEST}/logs/"
+    fi
+  done
+done
+
+echo "[cleanup] Running make clean" | tee -a "${DEST}/CLEANUP_SUMMARY.txt"
+if command -v make >/dev/null 2>&1; then
+  ( make clean >/dev/null 2>&1 || true )
+  echo "make clean: done" >> "${DEST}/CLEANUP_SUMMARY.txt"
+else
+  echo "make not found, skipping" >> "${DEST}/CLEANUP_SUMMARY.txt"
+fi
+
+# Remove common bench/wrapper binaries (rebuildable)
+echo "[cleanup] Removing rebuildable binaries" | tee -a "${DEST}/CLEANUP_SUMMARY.txt"
+rm -f \
+  larson_hakmem larson_hakmem_asan larson_hakmem_tsan larson_hakmem_ubsan \
+  bench_*_hakmem bench_*_system bench_*_mi \
+  bench_tiny bench_tiny_mt phase6_bench_tiny_simple test_hakmem
+
+# Report large files remaining at top-level
+echo "[cleanup] Large files remaining (top-level, >1MB)" | tee -a "${DEST}/CLEANUP_SUMMARY.txt"
+{ find . -maxdepth 1 -type f -size +1M -printf "%f\t%k KB\n" 2>/dev/null || true; } | tee -a "${DEST}/POST_CLEAN_LARGE_FILES.txt"
+
+echo "[cleanup] Done. Summary at ${DEST}/CLEANUP_SUMMARY.txt"
+
--- a/scripts/run_larson_claude.sh
+++ b/scripts/run_larson_claude.sh
@ -18,8 +18,16 @@ THREADS=${3:-4}
 ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)"
 cd "$ROOT_DIR"

-# Ensure build
-[[ -x ./larson_hakmem ]] || make -s larson_hakmem >/dev/null
+# Ensure build (honor 3-layer/route build knobs)
+# HAKMEM_BUILD_3LAYER=1    → make larson_hakmem_3layer
+# HAKMEM_BUILD_ROUTE=1      → make larson_hakmem_route (implies 3-layer)
+if [[ "${HAKMEM_BUILD_ROUTE:-0}" == "1" ]]; then
+  make -s larson_hakmem_route >/dev/null
+elif [[ "${HAKMEM_BUILD_3LAYER:-0}" == "1" ]]; then
+  make -s larson_hakmem_3layer >/dev/null
+else
+  [[ -x ./larson_hakmem ]] || make -s larson_hakmem >/dev/null
+fi

 # Common Tiny + Larson envs
 export HAKMEM_LARSON_TINY_ONLY=1
@ -45,11 +53,18 @@ case "$MODE" in
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-21}
    export HAKMEM_TINY_SS_CACHE=${HAKMEM_TINY_SS_CACHE:-0}
    export HAKMEM_TINY_SS_PRECHARGE=${HAKMEM_TINY_SS_PRECHARGE:-0}
+    # Opportunistic background remote drain (lightweight)
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-16}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-2}
    ;;
  pf)
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-20}
    export HAKMEM_TINY_SS_CACHE=${HAKMEM_TINY_SS_CACHE:-4}
    export HAKMEM_TINY_SS_PRECHARGE=${HAKMEM_TINY_SS_PRECHARGE:-1}
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-8}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-4}
    ;;
  repro)
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-21}
@ -59,6 +74,9 @@ case "$MODE" in
    export HAKMEM_TINY_SS_ADOPT=1
    # Force notify to surface publish even if slab_listed was missed
    export HAKMEM_TINY_RF_FORCE_NOTIFY=${HAKMEM_TINY_RF_FORCE_NOTIFY:-1}
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-4}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-2}
    ;;
  fast0)
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-21}
@ -68,6 +86,9 @@ case "$MODE" in
    export HAKMEM_TINY_DEBUG_FAST0=1
    export HAKMEM_TINY_SS_ADOPT=1
    export HAKMEM_TINY_RF_FORCE_NOTIFY=${HAKMEM_TINY_RF_FORCE_NOTIFY:-1}
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-4}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-2}
    ;;
  guard)
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-21}
@ -80,6 +101,9 @@ case "$MODE" in
    export HAKMEM_TINY_RF_FORCE_NOTIFY=${HAKMEM_TINY_RF_FORCE_NOTIFY:-1}
    export HAKMEM_SAFE_FREE=${HAKMEM_SAFE_FREE:-1}
    export HAKMEM_SAFE_FREE_STRICT=${HAKMEM_SAFE_FREE_STRICT:-1}
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-4}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-2}
    ;;
  debug)
    export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-21}
@ -90,6 +114,9 @@ case "$MODE" in
    export HAKMEM_TINY_RF_FORCE_NOTIFY=${HAKMEM_TINY_RF_FORCE_NOTIFY:-1}
    export HAKMEM_SAFE_FREE=${HAKMEM_SAFE_FREE:-1}
    export HAKMEM_SAFE_FREE_STRICT=${HAKMEM_SAFE_FREE_STRICT:-1}
+    export HAKMEM_TINY_BG_REMOTE=${HAKMEM_TINY_BG_REMOTE:-1}
+    export HAKMEM_TINY_BG_REMOTE_TRYRATE=${HAKMEM_TINY_BG_REMOTE_TRYRATE:-4}
+    export HAKMEM_TINY_BG_REMOTE_BUDGET=${HAKMEM_TINY_BG_REMOTE_BUDGET:-2}
    ;;
  asan)
    make -s asan-larson >/dev/null || exit 1
--- a/scripts/run_larson_defaults.sh
+++ b/scripts/run_larson_defaults.sh
@ -53,8 +53,10 @@ if [[ "$MODE" == "tput" ]]; then
  export HAKMEM_TINY_DRAIN_THRESHOLD=${HAKMEM_TINY_DRAIN_THRESHOLD:-4}
  # Prefer mmap over adopt for raw tput until publish pipeline is proven
  export HAKMEM_TINY_MUST_ADOPT=${HAKMEM_TINY_MUST_ADOPT:-0}
-  export HAKMEM_TINY_SS_CACHE=${HAKMEM_TINY_SS_CACHE:-0}                # off
-  export HAKMEM_TINY_SS_PRECHARGE=${HAKMEM_TINY_SS_PRECHARGE:-0}        # off
+  # SS cache/precharge ON also for tput（syscall抑制で張り付き解消を狙う）
+  export HAKMEM_TINY_SS_CACHE=${HAKMEM_TINY_SS_CACHE:-8}
+  export HAKMEM_TINY_SS_PRECHARGE=${HAKMEM_TINY_SS_PRECHARGE:-1}
+  export HAKMEM_TINY_TRIM_SS=${HAKMEM_TINY_TRIM_SS:-0}
 else
  # Lower page-fault/sys defaults
  export HAKMEM_TINY_SS_FORCE_LG=${HAKMEM_TINY_SS_FORCE_LG:-20}         # 1MB
--- a/scripts/run_larson_dev.sh
+++ b/scripts/run_larson_dev.sh
@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# run_larson_dev.sh — deterministic run wrapper (avoids perf warm-up issues)
+#
+# Usage:
+#   scripts/run_larson_dev.sh tput 10 4
+#   scripts/run_larson_dev.sh pf   10 4
+#
+# Notes:
+#  - Runs ./larson_hakmem directly and prints the Throughput line.
+#  - Keeps logging quiet and avoids perf warm-ups that sometimes SEGV under A/B.
+
+MODE=${1:-tput}
+DUR=${2:-10}
+THR=${3:-4}
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)"
+cd "$ROOT_DIR"
+
+[[ -x ./larson_hakmem ]] || ./scripts/build_larson_dev.sh
+
+export HAKMEM_QUIET=1
+export HAKMEM_TINY_SUKESUKE=0
+export HAKMEM_TINY_TRACE_RING=0
+export HAKMEM_DISABLE_BATCH=1
+export HAKMEM_WRAP_TINY=1
+export HAKMEM_LARSON_TINY_ONLY=1
+export HAKMEM_TINY_META_ALLOC=1
+export HAKMEM_TINY_META_FREE=1
+export HAKMEM_TINY_USE_SUPERSLAB=1
+
+if [[ "$MODE" == "tput" ]]; then
+  export HAKMEM_TINY_FREE_TO_SS=0
+  export HAKMEM_TINY_MUST_ADOPT=0
+  export HAKMEM_TINY_REG_SCAN_MAX=${HAKMEM_TINY_REG_SCAN_MAX:-64}
+  export HAKMEM_SFC_ENABLE=${HAKMEM_SFC_ENABLE:-1}
+  export HAKMEM_TINY_TLS_LIST=${HAKMEM_TINY_TLS_LIST:-1}
+  export HAKMEM_TINY_TLS_SLL=${HAKMEM_TINY_TLS_SLL:-1}
+else
+  export HAKMEM_TINY_FREE_TO_SS=1
+  export HAKMEM_TINY_MUST_ADOPT=1
+  export HAKMEM_TINY_SS_ADOPT_COOLDOWN=${HAKMEM_TINY_SS_ADOPT_COOLDOWN:-64}
+  export HAKMEM_TINY_REG_SCAN_MAX=${HAKMEM_TINY_REG_SCAN_MAX:-32}
+fi
+
+echo "[run_dev] mode=$MODE dur=$DUR thr=$THR"
+./larson_hakmem "$DUR" 8 128 1024 1 12345 "$THR" | rg "Throughput" -n || true
+
--- a/scripts/run_larson_perf.sh
+++ b/scripts/run_larson_perf.sh
@ -1,6 +1,14 @@
 #!/usr/bin/env bash
 set -euo pipefail

+# Defensive: ensure timeout exists; if not, best-effort shim
+if ! command -v timeout >/dev/null 2>&1; then
+  echo "[warn] 'timeout' not found; runs may hang on bench bugs" >&2
+  TIMEOUT() { "$@"; }
+else
+  TIMEOUT() { timeout --kill-after=2s "$@"; }
+fi
+
 # Perf-annotated Larson runs for system/mimalloc/HAKMEM without LD_PRELOAD.
 # Writes results under scripts/bench_results/larson_perf_*.txt

@ -35,17 +43,43 @@ run_one() {
  local bin=$1; shift
  local thr=$1; shift
  local tag="${name}_${thr}T_${dur}s_${min}-${max}"
-  local outfile="$OUT_DIR/larson_perf_${tag}.txt"
-  echo "== $name threads=$thr ==" | tee "$outfile"
-  # Warm-up quick run (avoid one-time inits skew)
-  "$bin" 1 "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" >/dev/null 2>&1 || true
-  # Throughput (quiet)
-  local tput
-  tput=$("$bin" "$dur" "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" 2>/dev/null | rg "Throughput" -n || true)
-  echo "$tput" | tee -a "$outfile"
-  # perf stat
-  perf stat -o "$outfile" -a -d -d --append -- \
-    "$bin" "$dur" "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" >/dev/null 2>&1 || true
+  local base="$OUT_DIR/larson_${tag}"
+  local outfile="${base}.txt"
+  local outlog="${base}.stdout"
+  local errlog="${base}.stderr"
+  : >"$outfile"; : >"$outlog"; : >"$errlog"
+
+  echo "== $name threads=$thr ==" | tee -a "$outfile"
+
+  # Warm-up quick run (avoid one-time inits skew). Always bounded by timeout.
+  if [[ "$name" != "hakmem" ]]; then
+    TIMEOUT "$((dur+2))"s "$bin" 1 "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" \
+      >>"$outlog" 2>>"$errlog" || true
+  fi
+
+  # Throughput run with timeout; capture both stdout/stderr to logs
+  echo "[cmd] $bin $dur $min $max $chunks $rounds $seed $thr" | tee -a "$outfile"
+  TIMEOUT "$((dur+3))"s "$bin" "$dur" "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" \
+    >>"$outlog" 2>>"$errlog" || true
+  # Extract a single Throughput line from the captured stdout
+  local tput_line
+  if command -v rg >/dev/null 2>&1; then
+    tput_line=$(rg -n "Throughput" -m 1 "$outlog" || true)
+  else
+    tput_line=$(grep -n "Throughput" "$outlog" | head -n1 || true)
+  fi
+  [[ -n "$tput_line" ]] && echo "$tput_line" | tee -a "$outfile" || echo "(no Throughput line)" | tee -a "$outfile"
+
+  # perf stat (optional; if perf not present, skip gracefully)
+  if command -v perf >/dev/null 2>&1; then
+    TIMEOUT "$((dur+3))"s perf stat -o "$outfile" -a -d -d --append -- \
+      "$bin" "$dur" "$min" "$max" "$chunks" "$rounds" "$seed" "$thr" \
+      >>"$outlog" 2>>"$errlog" || true
+  else
+    echo "[warn] perf not found; skipping perf stat" | tee -a "$outfile"
+  fi
+
+  echo "[logs] stdout=$outlog stderr=$errlog" | tee -a "$outfile"
 }

 for t in "${ts[@]}"; do