Tiny: fix header/stride mismatch and harden refill paths

- Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte
  header during allocation, but linear carve/refill and initial slab capacity
  still used bare class block sizes. This mismatch could overrun slab usable
  space and corrupt freelists, causing reproducible SEGV at ~100k iters.

Changes
- Superslab: compute capacity with effective stride (block_size + header for
  classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a
  debug-only bound check in superslab_alloc_from_slab() to fail fast if carve
  would exceed usable bytes.
- Refill (non-P0 and P0): use header-aware stride for all linear carving and
  TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h
  also uses stride, not raw class size.
- Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes
  before splicing into freelist (already present).

Notes
- This unifies the memory layout across alloc/linear-carve/refill with a single
  stride definition and keeps class7 (1024B) headerless as designed.
- Debug builds add fail-fast checks; release builds remain lean.

Next
- Re-run Tiny benches (256/1024B) in debug to confirm stability, then in
  release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0
  to isolate P0 batch carve, and continue reducing branch-miss as planned.
This commit is contained in:
Moe Charm (CI)
2025-11-09 18:55:50 +09:00
parent ab68ee536d
commit 1010a961fb
171 changed files with 10238 additions and 634 deletions

View File

@ -14,6 +14,10 @@
set -euo pipefail
FLAVOR="release"
if [[ $# -gt 0 && ( "$1" == "release" || "$1" == "debug" ) ]]; then
FLAVOR="$1"; shift
fi
TARGET="${1:-bench_mid_large_mt_hakmem}"
usage() {
@ -89,6 +93,7 @@ fi
echo "========================================="
echo " HAKMEM Build Script"
echo " Flavor: ${FLAVOR}"
echo " Target: ${TARGET}"
echo " Flags: POOL_TLS_PHASE1=1 POOL_TLS_PREWARM=1 HEADER_CLASSIDX=1 AGGRESSIVE_INLINE=1 PREWARM_TLS=1 ${EXTRA_MAKEFLAGS:-}"
echo "========================================="
@ -97,18 +102,45 @@ echo "========================================="
make clean >/dev/null 2>&1 || true
# Phase 7 + Pool TLS defaults (pinned) + user extras
make \
MAKE_ARGS=(
BUILD_FLAVOR=${FLAVOR} \
POOL_TLS_PHASE1=1 \
POOL_TLS_PREWARM=1 \
HEADER_CLASSIDX=1 \
AGGRESSIVE_INLINE=1 \
PREWARM_TLS=1 \
${EXTRA_MAKEFLAGS:-} \
"${TARGET}"
)
# Apply debug flavor extras (non-invasive): verbose + safe-free optional
if [[ "${FLAVOR}" == "debug" ]]; then
MAKE_ARGS+=(HAKMEM_DEBUG_VERBOSE=1)
MAKE_ARGS+=(BUILD_RELEASE_DEFAULT=0) # Disable release mode for debug builds
# Uncomment to enable extra safety by default for debug runs (may slow hot path)
# MAKE_ARGS+=(HAKMEM_TINY_SAFE_FREE=1)
else
MAKE_ARGS+=(BUILD_RELEASE_DEFAULT=1) # Enable release mode for release builds
fi
# Append user-provided extras and target
if [[ -n "${EXTRA_MAKEFLAGS:-}" ]]; then
# shellcheck disable=SC2206
MAKE_ARGS+=(${EXTRA_MAKEFLAGS})
fi
MAKE_ARGS+=("${TARGET}")
make "${MAKE_ARGS[@]}"
echo ""
echo "========================================="
echo " ✅ Build successful"
echo " Run: ./${TARGET}"
echo "-----------------------------------------"
# Place artifacts under out/<flavor>/
OUTDIR="out/${FLAVOR}"
mkdir -p "${OUTDIR}"
if [[ -x "./${TARGET}" ]]; then
cp -f "./${TARGET}" "${OUTDIR}/${TARGET}"
echo " Saved: ${OUTDIR}/${TARGET}"
fi
echo " Tip: ./build.sh help # flags, ENV, targets"
echo "========================================="