tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tiny Pool redesign: P0.1, P0.3, P1.1, P1.2 - Out-of-band class_idx lookup

Browse Source 
This commit implements the first phase of Tiny Pool redesign based on
ChatGPT architecture review. The goal is to eliminate Header/Next pointer
conflicts by moving class_idx lookup out-of-band (to SuperSlab metadata).

## P0.1: C0(8B) class upgraded to 16B
- Size table changed: {16,32,64,128,256,512,1024,2048} (8 classes)
- LUT updated: 1..16 → class 0, 17..32 → class 1, etc.
- tiny_next_off: C0 now uses offset 1 (header preserved)
- Eliminates edge cases for 8B allocations

## P0.3: Slab reuse guard Box (tls_slab_reuse_guard_box.h)
- New Box for draining TLS SLL before slab reuse
- ENV gate: HAKMEM_TINY_SLAB_REUSE_GUARD=1
- Prevents stale pointers when slabs are recycled
- Follows Box theory: single responsibility, minimal API

## P1.1: SuperSlab class_map addition
- Added uint8_t class_map[SLABS_PER_SUPERSLAB_MAX] to SuperSlab
- Maps slab_idx → class_idx for out-of-band lookup
- Initialized to 255 (UNASSIGNED) on SuperSlab creation
- Set correctly on slab initialization in all backends

## P1.2: Free fast path uses class_map
- ENV gate: HAKMEM_TINY_USE_CLASS_MAP=1
- Free path can now get class_idx from class_map instead of Header
- Falls back to Header read if class_map returns invalid value
- Fixed Legacy Backend dynamic slab initialization bug

## Documentation added
- HAKMEM_ARCHITECTURE_OVERVIEW.md: 4-layer architecture analysis
- TLS_SLL_ARCHITECTURE_INVESTIGATION.md: Root cause analysis
- PTR_LIFECYCLE_TRACE_AND_ROOT_CAUSE_ANALYSIS.md: Pointer tracking
- TINY_REDESIGN_CHECKLIST.md: Implementation roadmap (P0-P3)

## Test results
- Baseline: 70% success rate (30% crash - pre-existing issue)
- class_map enabled: 70% success rate (same as baseline)
- Performance: ~30.5M ops/s (unchanged)

## Next steps (P1.3, P2, P3)
- P1.3: Add meta->active for accurate TLS/freelist sync
- P2: TLS SLL redesign with Box-based counting
- P3: Complete Header out-of-band migration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-11-28 13:42:39 +09:00
parent 0ce20bb835
commit dc9e650db3
23 changed files with 5338 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1061
HAKMEM_ARCHITECTURE_OVERVIEW.md Normal file
View File

File diff suppressed because it is too large Load Diff

4
Makefile
View File

@ -195,7 +195,7 @@ OBJS = $(OBJS_BASE)
# Shared library # Shared library
SHARED_LIB = libhakmem.so SHARED_LIB = libhakmem.so
SHARED_OBJS = hakmem_shared.o hakmem_config_shared.o hakmem_tiny_config_shared.o hakmem_ucb1_shared.o hakmem_bigcache_shared.o hakmem_pool_shared.o hakmem_l25_pool_shared.o hakmem_site_rules_shared.o hakmem_tiny_shared.o hakmem_tiny_superslab_shared.o hakmem_smallmid_shared.o core/box/superslab_expansion_box_shared.o core/box/integrity_box_shared.o core/box/mailbox_box_shared.o core/box/front_gate_box_shared.o core/box/free_local_box_shared.o core/box/free_remote_box_shared.o core/box/free_publish_box_shared.o core/box/capacity_box_shared.o core/box/carve_push_box_shared.o core/box/prewarm_box_shared.o core/box/bench_fast_box_shared.o core/front/tiny_unified_cache_shared.o tiny_sticky_shared.o tiny_remote_shared.o tiny_publish_shared.o tiny_debug_ring_shared.o hakmem_tiny_magazine_shared.o hakmem_tiny_stats_shared.o hakmem_tiny_sfc_shared.o hakmem_tiny_query_shared.o hakmem_tiny_rss_shared.o hakmem_tiny_registry_shared.o hakmem_tiny_remote_target_shared.o hakmem_tiny_bg_spill_shared.o tiny_adaptive_sizing_shared.o hakmem_mid_mt_shared.o hakmem_super_registry_shared.o hakmem_elo_shared.o hakmem_batch_shared.o hakmem_p2_shared.o hakmem_sizeclass_dist_shared.o hakmem_evo_shared.o hakmem_debug_shared.o hakmem_sys_shared.o hakmem_whale_shared.o hakmem_policy_shared.o hakmem_ace_shared.o hakmem_ace_stats_shared.o hakmem_ace_controller_shared.o hakmem_ace_metrics_shared.o hakmem_ace_ucb1_shared.o hakmem_prof_shared.o hakmem_learner_shared.o hakmem_size_hist_shared.o hakmem_learn_log_shared.o hakmem_syscall_shared.o tiny_fastcache_shared.o SHARED_OBJS = hakmem_shared.o hakmem_config_shared.o hakmem_tiny_config_shared.o hakmem_ucb1_shared.o hakmem_bigcache_shared.o hakmem_pool_shared.o hakmem_l25_pool_shared.o hakmem_site_rules_shared.o hakmem_tiny_shared.o hakmem_tiny_superslab_shared.o hakmem_smallmid_shared.o hakmem_smallmid_superslab_shared.o core/box/superslab_expansion_box_shared.o core/box/integrity_box_shared.o core/box/mailbox_box_shared.o core/box/front_gate_box_shared.o core/box/front_gate_classifier_shared.o core/box/free_local_box_shared.o core/box/free_remote_box_shared.o core/box/free_publish_box_shared.o core/box/capacity_box_shared.o core/box/carve_push_box_shared.o core/box/unified_batch_box_shared.o core/box/prewarm_box_shared.o core/box/ss_hot_prewarm_box_shared.o core/box/front_metrics_box_shared.o core/box/bench_fast_box_shared.o core/box/pagefault_telemetry_box_shared.o core/box/tiny_sizeclass_hist_box_shared.o core/page_arena_shared.o core/front/tiny_unified_cache_shared.o core/tiny_alloc_fast_push_shared.o core/link_stubs_shared.o core/tiny_failfast_shared.o tiny_sticky_shared.o tiny_remote_shared.o tiny_publish_shared.o tiny_debug_ring_shared.o hakmem_tiny_magazine_shared.o hakmem_tiny_stats_shared.o hakmem_tiny_sfc_shared.o hakmem_tiny_query_shared.o hakmem_tiny_rss_shared.o hakmem_tiny_registry_shared.o hakmem_tiny_remote_target_shared.o hakmem_tiny_bg_spill_shared.o tiny_adaptive_sizing_shared.o hakmem_mid_mt_shared.o hakmem_super_registry_shared.o hakmem_shared_pool_shared.o hakmem_elo_shared.o hakmem_batch_shared.o hakmem_p2_shared.o hakmem_sizeclass_dist_shared.o hakmem_evo_shared.o hakmem_debug_shared.o hakmem_sys_shared.o hakmem_whale_shared.o hakmem_policy_shared.o hakmem_ace_shared.o hakmem_ace_stats_shared.o hakmem_ace_controller_shared.o hakmem_ace_metrics_shared.o hakmem_ace_ucb1_shared.o hakmem_prof_shared.o hakmem_learner_shared.o hakmem_size_hist_shared.o hakmem_learn_log_shared.o hakmem_syscall_shared.o tiny_fastcache_shared.o
# Pool TLS Phase 1 (enable with POOL_TLS_PHASE1=1) # Pool TLS Phase 1 (enable with POOL_TLS_PHASE1=1)
ifeq ($(POOL_TLS_PHASE1),1) ifeq ($(POOL_TLS_PHASE1),1)
@ -222,7 +222,7 @@ endif
# Benchmark targets # Benchmark targets
BENCH_HAKMEM = bench_allocators_hakmem BENCH_HAKMEM = bench_allocators_hakmem
BENCH_SYSTEM = bench_allocators_system BENCH_SYSTEM = bench_allocators_system
BENCH_HAKMEM_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o hakmem_tiny_superslab.o hakmem_smallmid.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_mid_mt.o hakmem_super_registry.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_local_box.o core/box/free_remote_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/unified_batch_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/link_stubs.o core/tiny_failfast.o bench_allocators_hakmem.o BENCH_HAKMEM_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o hakmem_tiny_superslab.o hakmem_smallmid.o hakmem_smallmid_superslab.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_mid_mt.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_local_box.o core/box/free_remote_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/unified_batch_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/link_stubs.o core/tiny_failfast.o bench_allocators_hakmem.o
BENCH_HAKMEM_OBJS = $(BENCH_HAKMEM_OBJS_BASE) BENCH_HAKMEM_OBJS = $(BENCH_HAKMEM_OBJS_BASE)
ifeq ($(POOL_TLS_PHASE1),1) ifeq ($(POOL_TLS_PHASE1),1)
BENCH_HAKMEM_OBJS += pool_tls.o pool_refill.o pool_tls_arena.o pool_tls_registry.o pool_tls_remote.o BENCH_HAKMEM_OBJS += pool_tls.o pool_refill.o pool_tls_arena.o pool_tls_registry.o pool_tls_remote.o

368
core/box/ptr_trace_box.h Normal file
View File

@ -0,0 +1,368 @@
// ptr_trace_box.h - Pointer Lifecycle Tracing System (Debug Only)
//
// Purpose:
// - Track complete lifecycle of pointers: allocation, free, TLS SLL operations, drain
// - Detect root cause of double-free bugs (TLS SLL vs Freelist synchronization issues)
// - Zero overhead in release builds (compile-time gated)
//
// Features:
// - Track 7 event types: CARVE, ALLOC_FREELIST, ALLOC_TLS_POP, FREE_TLS_PUSH,
// DRAIN_TO_FREELIST, SLAB_REUSE, REFILL
// - Environment variable control:
// - HAKMEM_PTR_TRACE_ALL=1: Trace all pointers (high overhead)
// - HAKMEM_PTR_TRACE=0xADDR: Trace specific pointer only
// - HAKMEM_PTR_TRACE_CLASS=N: Trace specific class only
// - Configurable ring buffer (default: 4096 entries per thread)
// - Automatic dump on crash/abort
//
// Design:
// - Thread-local ring buffer (no locks, no contention)
// - Atomic operation counter for sequencing across threads
// - Lazy initialization (first trace call per thread)
// - Header-only for inline performance
//
// Integration Points:
// - Linear carve: PTR_TRACE_CARVE(ptr, class_idx, op, slab_idx)
// - Freelist alloc: PTR_TRACE_ALLOC_FREELIST(ptr, class_idx, op, fl_head)
// - TLS SLL pop: PTR_TRACE_ALLOC_TLS_POP(ptr, class_idx, op, tls_count)
// - TLS SLL push: PTR_TRACE_FREE_TLS_PUSH(ptr, class_idx, op, tls_count)
// - Drain: PTR_TRACE_DRAIN_TO_FREELIST(ptr, class_idx, op, tls_count_before)
// - Slab reuse: PTR_TRACE_SLAB_REUSE(slab_base, class_idx, op)
// - Refill: PTR_TRACE_REFILL(class_idx, op, ss, slab_idx)
#ifndef PTR_TRACE_BOX_H
#define PTR_TRACE_BOX_H
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdatomic.h>
#include <pthread.h>
#include "../hakmem_build_flags.h"
#include "../hakmem_tiny_config.h"
// Only enable in debug builds
#if !HAKMEM_BUILD_RELEASE
// ========== Configuration ==========
#ifndef PTR_TRACE_RING_SIZE
# define PTR_TRACE_RING_SIZE 4096
#endif
// Event types
typedef enum {
PTR_EVENT_CARVE = 1, // Linear carve (new block from slab)
PTR_EVENT_ALLOC_FREELIST = 2, // Allocated from freelist
PTR_EVENT_ALLOC_TLS_POP = 3, // Allocated from TLS SLL (pop)
PTR_EVENT_FREE_TLS_PUSH = 4, // Freed to TLS SLL (push)
PTR_EVENT_DRAIN_TO_FREELIST = 5, // Drained from TLS SLL to freelist
PTR_EVENT_SLAB_REUSE = 6, // Slab reused (all pointers invalidated)
PTR_EVENT_REFILL = 7, // Slab refill
PTR_EVENT_FREELIST_FREE = 8, // Freed directly to freelist (slow path)
} ptr_trace_event_t;
// Event record
typedef struct {
void* ptr; // Pointer address (BASE for allocations)
uint64_t op_num; // Global operation number
uint32_t event; // Event type (ptr_trace_event_t)
uint8_t class_idx; // Class index
uint8_t _pad[3]; // Padding to 8-byte boundary
union {
void* freelist_head; // Freelist head (ALLOC_FREELIST)
uint32_t tls_count; // TLS SLL count (TLS_PUSH/POP/DRAIN)
int slab_idx; // Slab index (CARVE/REFILL/SLAB_REUSE)
} aux;
const char* file; // Source file (__FILE__)
int line; // Source line (__LINE__)
} ptr_trace_record_t;
// ========== TLS State ==========
static __thread ptr_trace_record_t g_ptr_trace_ring[PTR_TRACE_RING_SIZE];
static __thread uint32_t g_ptr_trace_ring_idx = 0;
static __thread int g_ptr_trace_initialized = 0;
// Trace modes (cached per thread)
static __thread int g_ptr_trace_mode = -1; // -1=uninitialized, 0=off, 1=specific ptr, 2=specific class, 3=all
static __thread uintptr_t g_ptr_trace_target = 0; // Target pointer address (mode 1)
static __thread int g_ptr_trace_target_class = -1; // Target class (mode 2)
// ========== Global State ==========
// Global operation counter (atomic, shared across threads)
static _Atomic uint64_t g_ptr_trace_op_counter = 0;
// Dump registered flag (global, one-time setup)
static _Atomic int g_ptr_trace_dump_registered = 0;
// ========== Helpers ==========
static inline const char* ptr_event_name(ptr_trace_event_t ev) {
switch (ev) {
case PTR_EVENT_CARVE: return "CARVE";
case PTR_EVENT_ALLOC_FREELIST: return "ALLOC_FREELIST";
case PTR_EVENT_ALLOC_TLS_POP: return "ALLOC_TLS_POP";
case PTR_EVENT_FREE_TLS_PUSH: return "FREE_TLS_PUSH";
case PTR_EVENT_DRAIN_TO_FREELIST: return "DRAIN_TO_FREELIST";
case PTR_EVENT_SLAB_REUSE: return "SLAB_REUSE";
case PTR_EVENT_REFILL: return "REFILL";
case PTR_EVENT_FREELIST_FREE: return "FREELIST_FREE";
default: return "UNKNOWN";
}
}
// Initialize trace mode from environment variables
static inline void ptr_trace_init(void) {
if (g_ptr_trace_initialized) return;
g_ptr_trace_initialized = 1;
// Check HAKMEM_PTR_TRACE_ALL
const char* env_all = getenv("HAKMEM_PTR_TRACE_ALL");
if (env_all && *env_all && *env_all != '0') {
g_ptr_trace_mode = 3; // Trace all
fprintf(stderr, "[PTR_TRACE_INIT] Mode: ALL (high overhead)\n");
return;
}
// Check HAKMEM_PTR_TRACE (specific pointer)
const char* env_ptr = getenv("HAKMEM_PTR_TRACE");
if (env_ptr && *env_ptr) {
char* endp = NULL;
uintptr_t addr = (uintptr_t)strtoull(env_ptr, &endp, 0);
if (addr != 0) {
g_ptr_trace_mode = 1;
g_ptr_trace_target = addr;
fprintf(stderr, "[PTR_TRACE_INIT] Mode: SPECIFIC_PTR target=%p\n", (void*)addr);
return;
}
}
// Check HAKMEM_PTR_TRACE_CLASS
const char* env_cls = getenv("HAKMEM_PTR_TRACE_CLASS");
if (env_cls && *env_cls) {
int cls = atoi(env_cls);
if (cls >= 0 && cls < TINY_NUM_CLASSES) {
g_ptr_trace_mode = 2;
g_ptr_trace_target_class = cls;
fprintf(stderr, "[PTR_TRACE_INIT] Mode: SPECIFIC_CLASS class=%d\n", cls);
return;
}
}
// Default: OFF
g_ptr_trace_mode = 0;
}
// Check if we should trace this pointer/class
static inline int ptr_trace_should_log(void* ptr, int class_idx) {
if (g_ptr_trace_mode == -1) {
ptr_trace_init();
}
switch (g_ptr_trace_mode) {
case 0: return 0; // OFF
case 1: return ((uintptr_t)ptr == g_ptr_trace_target); // Specific pointer
case 2: return (class_idx == g_ptr_trace_target_class); // Specific class
case 3: return 1; // All
default: return 0;
}
}
// Dump trace ring for current thread
static inline void ptr_trace_dump(void) {
fprintf(stderr, "\n========== PTR_TRACE_DUMP (thread=%lx) ==========\n",
(unsigned long)pthread_self());
fprintf(stderr, "Ring index: %u (size=%d)\n", g_ptr_trace_ring_idx, PTR_TRACE_RING_SIZE);
uint32_t count = (g_ptr_trace_ring_idx < PTR_TRACE_RING_SIZE)
? g_ptr_trace_ring_idx
: PTR_TRACE_RING_SIZE;
uint32_t start_idx = (g_ptr_trace_ring_idx >= PTR_TRACE_RING_SIZE)
? (g_ptr_trace_ring_idx % PTR_TRACE_RING_SIZE)
: 0;
fprintf(stderr, "Last %u events:\n", count);
for (uint32_t i = 0; i < count; i++) {
uint32_t idx = (start_idx + i) % PTR_TRACE_RING_SIZE;
ptr_trace_record_t* r = &g_ptr_trace_ring[idx];
fprintf(stderr, "[%4u] op=%06lu event=%-20s cls=%d ptr=%p",
i, (unsigned long)r->op_num, ptr_event_name(r->event),
r->class_idx, r->ptr);
// Print auxiliary info based on event type
switch (r->event) {
case PTR_EVENT_ALLOC_FREELIST:
fprintf(stderr, " fl_head=%p", r->aux.freelist_head);
break;
case PTR_EVENT_ALLOC_TLS_POP:
case PTR_EVENT_FREE_TLS_PUSH:
case PTR_EVENT_DRAIN_TO_FREELIST:
fprintf(stderr, " tls_count=%u", r->aux.tls_count);
break;
case PTR_EVENT_CARVE:
case PTR_EVENT_REFILL:
case PTR_EVENT_SLAB_REUSE:
fprintf(stderr, " slab_idx=%d", r->aux.slab_idx);
break;
default:
break;
}
fprintf(stderr, " from=%s:%d\n", r->file ? r->file : "(null)", r->line);
}
fprintf(stderr, "========== END PTR_TRACE_DUMP ==========\n\n");
fflush(stderr);
}
// Dump all traces (called at exit)
static void ptr_trace_dump_atexit(void) {
fprintf(stderr, "\n[PTR_TRACE] Automatic dump at exit\n");
ptr_trace_dump();
}
// Register atexit handler (once per process)
static inline void ptr_trace_register_dump(void) {
int expected = 0;
if (atomic_compare_exchange_strong(&g_ptr_trace_dump_registered, &expected, 1)) {
atexit(ptr_trace_dump_atexit);
}
}
// Record a trace event
static inline void ptr_trace_record_impl(
ptr_trace_event_t event,
void* ptr,
int class_idx,
uint64_t op_num,
void* aux_ptr,
uint32_t aux_u32,
int aux_int,
const char* file,
int line)
{
if (!ptr_trace_should_log(ptr, class_idx)) {
return;
}
// Register dump handler on first trace
ptr_trace_register_dump();
uint32_t idx = g_ptr_trace_ring_idx % PTR_TRACE_RING_SIZE;
ptr_trace_record_t* r = &g_ptr_trace_ring[idx];
r->ptr = ptr;
r->op_num = op_num;
r->event = event;
r->class_idx = (uint8_t)class_idx;
// Fill auxiliary data based on event type
switch (event) {
case PTR_EVENT_ALLOC_FREELIST:
r->aux.freelist_head = aux_ptr;
break;
case PTR_EVENT_ALLOC_TLS_POP:
case PTR_EVENT_FREE_TLS_PUSH:
case PTR_EVENT_DRAIN_TO_FREELIST:
r->aux.tls_count = aux_u32;
break;
case PTR_EVENT_CARVE:
case PTR_EVENT_REFILL:
case PTR_EVENT_SLAB_REUSE:
r->aux.slab_idx = aux_int;
break;
default:
r->aux.tls_count = 0;
break;
}
r->file = file;
r->line = line;
g_ptr_trace_ring_idx++;
// Optional: Print event in real-time (very verbose)
static __thread int s_verbose = -1;
if (s_verbose == -1) {
const char* env = getenv("HAKMEM_PTR_TRACE_VERBOSE");
s_verbose = (env && *env && *env != '0') ? 1 : 0;
}
if (s_verbose) {
fprintf(stderr, "[PTR_TRACE] op=%06lu event=%-20s cls=%d ptr=%p from=%s:%d\n",
(unsigned long)op_num, ptr_event_name(event), class_idx, ptr,
file ? file : "?", line);
}
}
// ========== Public API (Macros) ==========
#define PTR_TRACE_CARVE(ptr, class_idx, slab_idx) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_CARVE, (ptr), (class_idx), _op, \
NULL, 0, (slab_idx), __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_ALLOC_FREELIST(ptr, class_idx, fl_head) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_ALLOC_FREELIST, (ptr), (class_idx), _op, \
(fl_head), 0, 0, __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_ALLOC_TLS_POP(ptr, class_idx, tls_count) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_ALLOC_TLS_POP, (ptr), (class_idx), _op, \
NULL, (tls_count), 0, __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_FREE_TLS_PUSH(ptr, class_idx, tls_count) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_FREE_TLS_PUSH, (ptr), (class_idx), _op, \
NULL, (tls_count), 0, __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_DRAIN_TO_FREELIST(ptr, class_idx, tls_count_before) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_DRAIN_TO_FREELIST, (ptr), (class_idx), _op, \
NULL, (tls_count_before), 0, __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_SLAB_REUSE(slab_base, class_idx, slab_idx) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_SLAB_REUSE, (slab_base), (class_idx), _op, \
NULL, 0, (slab_idx), __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_REFILL(class_idx, ss, slab_idx) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_REFILL, (void*)(ss), (class_idx), _op, \
NULL, 0, (slab_idx), __FILE__, __LINE__); \
} while (0)
#define PTR_TRACE_FREELIST_FREE(ptr, class_idx) do { \
uint64_t _op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed); \
ptr_trace_record_impl(PTR_EVENT_FREELIST_FREE, (ptr), (class_idx), _op, \
NULL, 0, 0, __FILE__, __LINE__); \
} while (0)
// Manual dump (for debugging)
#define PTR_TRACE_DUMP() ptr_trace_dump()
#else // HAKMEM_BUILD_RELEASE (Release build - no-op macros)
// Zero-overhead stubs for release builds
#define PTR_TRACE_CARVE(ptr, class_idx, slab_idx) ((void)0)
#define PTR_TRACE_ALLOC_FREELIST(ptr, class_idx, fl_head) ((void)0)
#define PTR_TRACE_ALLOC_TLS_POP(ptr, class_idx, tls_count) ((void)0)
#define PTR_TRACE_FREE_TLS_PUSH(ptr, class_idx, tls_count) ((void)0)
#define PTR_TRACE_DRAIN_TO_FREELIST(ptr, class_idx, tls_count_before) ((void)0)
#define PTR_TRACE_SLAB_REUSE(slab_base, class_idx, slab_idx) ((void)0)
#define PTR_TRACE_REFILL(class_idx, ss, slab_idx) ((void)0)
#define PTR_TRACE_FREELIST_FREE(ptr, class_idx) ((void)0)
#define PTR_TRACE_DUMP() ((void)0)
#endif // !HAKMEM_BUILD_RELEASE
#endif // PTR_TRACE_BOX_H

18
core/box/ss_allocation_box.c
View File

@ -232,6 +232,9 @@ SuperSlab* superslab_allocate(uint8_t size_class) {
ss->lg_size = lg; // Phase 8.3: Use ACE-determined lg_size (20=1MB, 21=2MB) ss->lg_size = lg; // Phase 8.3: Use ACE-determined lg_size (20=1MB, 21=2MB)
ss->slab_bitmap = 0; ss->slab_bitmap = 0;
ss->nonempty_mask = 0; // Phase 6-2.1: ChatGPT Pro P0 - init nonempty mask ss->nonempty_mask = 0; // Phase 6-2.1: ChatGPT Pro P0 - init nonempty mask
ss->freelist_mask = 0; // P1.1 FIX: Initialize freelist_mask
ss->empty_mask = 0; // P1.1 FIX: Initialize empty_mask
ss->empty_count = 0; // P1.1 FIX: Initialize empty_count
ss->partial_epoch = 0; ss->partial_epoch = 0;
ss->publish_hint = 0xFF; ss->publish_hint = 0xFF;
@ -247,6 +250,15 @@ SuperSlab* superslab_allocate(uint8_t size_class) {
ss->lru_prev = NULL; ss->lru_prev = NULL;
ss->lru_next = NULL; ss->lru_next = NULL;
// Phase 3d-C: Initialize hot/cold fields
ss->hot_count = 0;
ss->cold_count = 0;
memset(ss->hot_indices, 0, sizeof(ss->hot_indices));
memset(ss->cold_indices, 0, sizeof(ss->cold_indices));
// Phase 12: Initialize next_chunk (legacy per-class chain)
ss->next_chunk = NULL;
// Initialize all slab metadata (only up to max slabs for this size) // Initialize all slab metadata (only up to max slabs for this size)
int max_slabs = (int)(ss_size / SLAB_SIZE); int max_slabs = (int)(ss_size / SLAB_SIZE);
@ -258,6 +270,10 @@ SuperSlab* superslab_allocate(uint8_t size_class) {
memset(ss->remote_counts, 0, max_slabs * sizeof(uint32_t)); memset(ss->remote_counts, 0, max_slabs * sizeof(uint32_t));
memset(ss->slab_listed, 0, max_slabs * sizeof(uint32_t)); memset(ss->slab_listed, 0, max_slabs * sizeof(uint32_t));
// P1.1: Initialize class_map to UNASSIGNED (255) for all slabs
// This ensures class_map is in a known state even before slabs are assigned
memset(ss->class_map, 255, max_slabs * sizeof(uint8_t));
for (int i = 0; i < max_slabs; i++) { for (int i = 0; i < max_slabs; i++) {
ss_slab_meta_freelist_set(ss, i, NULL); // Explicit NULL (redundant after memset, but clear intent) ss_slab_meta_freelist_set(ss, i, NULL); // Explicit NULL (redundant after memset, but clear intent)
ss_slab_meta_used_set(ss, i, 0); ss_slab_meta_used_set(ss, i, 0);
@ -422,6 +438,8 @@ void superslab_init_slab(SuperSlab* ss, int slab_idx, size_t block_size, uint32_
for (int i = 0; i < TINY_NUM_CLASSES; i++) { for (int i = 0; i < TINY_NUM_CLASSES; i++) {
if (g_tiny_class_sizes[i] == stride) { if (g_tiny_class_sizes[i] == stride) {
meta->class_idx = (uint8_t)i; meta->class_idx = (uint8_t)i;
// P1.1: Update class_map for out-of-band lookup on free path
ss->class_map[slab_idx] = (uint8_t)i;
break; break;
} }
} }

27
core/box/ss_legacy_backend_box.c
View File

@ -126,12 +126,20 @@ void* hak_tiny_alloc_superslab_backend_legacy(int class_idx)
TinySlabMeta* meta = &chunk->slabs[slab_idx]; TinySlabMeta* meta = &chunk->slabs[slab_idx];
// Skip slabs that belong to a different class (or are uninitialized). // Skip slabs that belong to a different class (or are uninitialized).
if (meta->class_idx != (uint8_t)class_idx) { if (meta->class_idx != (uint8_t)class_idx && meta->class_idx != 255) {
continue; continue;
} }
// P1.2 FIX: Initialize slab on first use (like shared backend does)
// This ensures class_map is populated for all slabs, not just slab 0
if (meta->capacity == 0) { if (meta->capacity == 0) {
continue; size_t block_size = g_tiny_class_sizes[class_idx];
uint32_t owner_tid = (uint32_t)(uintptr_t)pthread_self();
superslab_init_slab(chunk, slab_idx, block_size, owner_tid);
meta = &chunk->slabs[slab_idx]; // Refresh pointer after init
meta->class_idx = (uint8_t)class_idx;
// P1.2: Update class_map for dynamic slab initialization
chunk->class_map[slab_idx] = (uint8_t)class_idx;
} }
if (meta->used < meta->capacity) { if (meta->used < meta->capacity) {
@ -166,7 +174,18 @@ void* hak_tiny_alloc_superslab_backend_legacy(int class_idx)
int cap2 = ss_slabs_capacity(new_chunk); int cap2 = ss_slabs_capacity(new_chunk);
for (int slab_idx = 0; slab_idx < cap2; slab_idx++) { for (int slab_idx = 0; slab_idx < cap2; slab_idx++) {
TinySlabMeta* meta = &new_chunk->slabs[slab_idx]; TinySlabMeta* meta = &new_chunk->slabs[slab_idx];
if (meta->capacity == 0) continue;
// P1.2 FIX: Initialize slab on first use (like shared backend does)
if (meta->capacity == 0) {
size_t block_size = g_tiny_class_sizes[class_idx];
uint32_t owner_tid = (uint32_t)(uintptr_t)pthread_self();
superslab_init_slab(new_chunk, slab_idx, block_size, owner_tid);
meta = &new_chunk->slabs[slab_idx]; // Refresh pointer after init
meta->class_idx = (uint8_t)class_idx;
// P1.2: Update class_map for dynamic slab initialization
new_chunk->class_map[slab_idx] = (uint8_t)class_idx;
}
if (meta->used < meta->capacity) { if (meta->used < meta->capacity) {
size_t stride = tiny_block_stride_for_class(class_idx); size_t stride = tiny_block_stride_for_class(class_idx);
size_t offset = (size_t)meta->used * stride; size_t offset = (size_t)meta->used * stride;
@ -281,6 +300,8 @@ int expand_superslab_head(SuperSlabHead* head) {
// CRITICAL FIX: Explicitly set class_idx to avoid C0/C7 confusion. // CRITICAL FIX: Explicitly set class_idx to avoid C0/C7 confusion.
// New SuperSlabs start with meta->class_idx=0 (mmap zero-init). // New SuperSlabs start with meta->class_idx=0 (mmap zero-init).
new_chunk->slabs[0].class_idx = (uint8_t)head->class_idx; new_chunk->slabs[0].class_idx = (uint8_t)head->class_idx;
// P1.1: Update class_map for legacy backend
new_chunk->class_map[0] = (uint8_t)head->class_idx;
// Initialize the next_chunk link to NULL // Initialize the next_chunk link to NULL
new_chunk->next_chunk = NULL; new_chunk->next_chunk = NULL;

2
core/box/superslab_expansion_box.c
View File

@ -70,6 +70,8 @@ ExpansionResult expansion_expand_with_tls_guarantee(
// CRITICAL FIX: Explicitly set class_idx to avoid C0/C7 confusion. // CRITICAL FIX: Explicitly set class_idx to avoid C0/C7 confusion.
// New SuperSlabs start with meta->class_idx=0 (mmap zero-init). // New SuperSlabs start with meta->class_idx=0 (mmap zero-init).
new_ss->slabs[0].class_idx = (uint8_t)class_idx; new_ss->slabs[0].class_idx = (uint8_t)class_idx;
// P1.1: Update class_map after expansion
new_ss->class_map[0] = (uint8_t)class_idx;
// Now bind slab 0 to TLS state // Now bind slab 0 to TLS state
result.new_state.ss = new_ss; result.new_state.ss = new_ss;

4
core/box/tiny_next_ptr_box.h
View File

@ -12,8 +12,8 @@
* 仕様は tiny_nextptr.h と完全一致: * 仕様は tiny_nextptr.h と完全一致:
* *
* HAKMEM_TINY_HEADER_CLASSIDX != 0: * HAKMEM_TINY_HEADER_CLASSIDX != 0:
* - Class 0: next_off = 0 (free中は header を潰す) * - Class 0-6: next_off = 1 (headerを保持)
* - Class 1-7: next_off = 1 (headerを保持) * - Class 7: next_off = 0 (free中は header を潰す)
* *
* HAKMEM_TINY_HEADER_CLASSIDX == 0: * HAKMEM_TINY_HEADER_CLASSIDX == 0:
* - 全クラス: next_off = 0 * - 全クラス: next_off = 0

131
core/box/tls_slab_reuse_guard_box.h Normal file
View File

@ -0,0 +1,131 @@
// tls_slab_reuse_guard_box.h - Box: TLS Slab Reuse Guard
//
// Purpose: Drain TLS SLL before reusing a SuperSlab's slab for a different class.
// This prevents orphaned TLS SLL blocks pointing to stale/repurposed slabs.
//
// Problem Context (P0.2 → P0.3 transition):
// - P0.2 attempted to unify next_offset to 1 for ALL classes (C0-C7)
// - This caused hangs due to header corruption when TLS SLL blocks
// referenced slabs that were repurposed for different classes
// - P0.3 reverts to C7=offset 0, C0-C6=offset 1 (stable layout)
// - But we still need guard rails against TLS SLL → Slab class mismatch
//
// Solution:
// - Box encapsulates "drain TLS SLL before slab reuse" logic
// - ENV-gated: HAKMEM_TINY_SLAB_REUSE_GUARD=1 to enable (default OFF)
// - When enabled: drain ALL Tiny classes' TLS SLL before reusing ANY slab
// - This ensures no stale TLS SLL pointers exist when slab changes class
//
// Design Principles (Box Theory):
// - Single Responsibility: Only handles TLS SLL drain on slab reuse trigger
// - Minimal API: One function tiny_tls_slab_reuse_guard(SuperSlab*)
// - Callers don't know about TLS SLL internals - just call the box
// - All diagnostics/counters contained within this box
//
// Usage:
// shared_pool_acquire_slab() calls tiny_tls_slab_reuse_guard(ss)
// right before binding a slab to a new class_idx.
//
// Performance Impact:
// - When disabled (default): Zero overhead (early return)
// - When enabled: Drains all 8 TLS SLL classes on every slab reuse
// - Expected frequency: Low (only when shared pool recycles slabs)
// - Trade-off: Safety (prevent corruption) vs. throughput (~5-10% slower)
#pragma once
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include "tls_sll_drain_box.h" // tiny_tls_sll_drain()
#include "../hakmem_tiny_config.h" // TINY_NUM_CLASSES
#include "../hakmem_build_flags.h" // HAKMEM_BUILD_RELEASE
// ========== ENV Configuration ==========
// Check if Slab Reuse Guard is enabled
// ENV: HAKMEM_TINY_SLAB_REUSE_GUARD=1/0 (default: 0 - disabled)
static inline int tls_slab_reuse_guard_is_enabled(void) {
static int g_guard_enable = -1;
if (__builtin_expect(g_guard_enable == -1, 0)) {
const char* env = getenv("HAKMEM_TINY_SLAB_REUSE_GUARD");
if (env && *env && *env != '0') {
g_guard_enable = 1;
#if !HAKMEM_BUILD_RELEASE
fprintf(stderr, "[TLS_SLAB_REUSE_GUARD] Enabled (ENV=1)\n");
#endif
} else {
g_guard_enable = 0;
#if !HAKMEM_BUILD_RELEASE
fprintf(stderr, "[TLS_SLAB_REUSE_GUARD] Disabled (default or ENV=0)\n");
#endif
}
}
return g_guard_enable;
}
// ========== Diagnostic Counters ==========
#if !HAKMEM_BUILD_RELEASE
static __thread uint64_t g_tls_slab_reuse_guard_calls = 0;
static __thread uint64_t g_tls_slab_reuse_guard_blocks = 0;
static void __attribute__((destructor)) tls_slab_reuse_guard_stats(void) {
if (g_tls_slab_reuse_guard_calls > 0) {
fprintf(stderr,
"[TLS_SLAB_REUSE_GUARD_STATS] Total calls: %lu, Total blocks drained: %lu, Avg: %.2f\n",
g_tls_slab_reuse_guard_calls,
g_tls_slab_reuse_guard_blocks,
(double)g_tls_slab_reuse_guard_blocks / g_tls_slab_reuse_guard_calls);
}
}
#endif
// ========== Slab Reuse Guard Implementation ==========
// Box: TLS Slab Reuse Guard
// Purpose: Drain TLS SLL before SuperSlab slab reuse
//
// Flow:
// 1. Check if guard is enabled (ENV gate)
// 2. If disabled, return immediately (zero overhead)
// 3. If enabled, drain ALL Tiny class TLS SLLs (0..7)
// 4. Update diagnostic counters (debug build only)
//
// Args:
// ss: SuperSlab that is about to have a slab reused (currently unused, reserved for future)
//
// Returns: void
static inline void tiny_tls_slab_reuse_guard(void* ss) {
// ENV gate: If disabled, early return (zero overhead)
if (__builtin_expect(!tls_slab_reuse_guard_is_enabled(), 1)) {
return;
}
(void)ss; // Reserved for future use (e.g., class-specific drain based on SS metadata)
// Drain ALL Tiny class TLS SLLs to prevent orphaned pointers
// This ensures no TLS SLL blocks point to slabs that are being repurposed
uint32_t total_drained = 0;
for (int cls = 0; cls < TINY_NUM_CLASSES; cls++) {
uint32_t drained = tiny_tls_sll_drain(cls, 0); // 0 = drain ALL blocks
total_drained += drained;
}
#if !HAKMEM_BUILD_RELEASE
// Debug logging (first 10 calls only)
static _Atomic uint32_t g_log_count = 0;
uint32_t log_count = atomic_fetch_add_explicit(&g_log_count, 1, memory_order_relaxed);
if (log_count < 10) {
fprintf(stderr,
"[TLS_SLAB_REUSE_GUARD] Drained %u blocks from TLS SLL (call #%u)\n",
total_drained, log_count + 1);
}
// Update stats
g_tls_slab_reuse_guard_calls++;
g_tls_slab_reuse_guard_blocks += total_drained;
#else
(void)total_drained; // Suppress unused warning in release
#endif
}

42
core/box/tls_sll_box.h
View File

@ -325,11 +325,10 @@ static inline bool tls_sll_push_impl(int class_idx, void* ptr, uint32_t capacity
} }
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Header handling for header classes (class 1-6 only, NOT 0 or 7). // C0-C6: Restore header (offset=1 layout). C7: skip (offset=0 - header overwritten by next).
// C0, C7 use offset=0, so next pointer is at base[0] and MUST NOT restore header.
// Safe mode (HAKMEM_TINY_SLL_SAFEHEADER=1): never overwrite header; reject on magic mismatch. // Safe mode (HAKMEM_TINY_SLL_SAFEHEADER=1): never overwrite header; reject on magic mismatch.
// Default mode: restore expected header. // Default mode: restore expected header.
if (class_idx != 0 && class_idx != 7) { if (class_idx != 7) {
static int g_sll_safehdr = -1; static int g_sll_safehdr = -1;
static int g_sll_ring_en = -1; // optional ring trace for TLS-SLL anomalies static int g_sll_ring_en = -1; // optional ring trace for TLS-SLL anomalies
if (__builtin_expect(g_sll_safehdr == -1, 0)) { if (__builtin_expect(g_sll_safehdr == -1, 0)) {
@ -340,6 +339,7 @@ static inline bool tls_sll_push_impl(int class_idx, void* ptr, uint32_t capacity
const char* r = getenv("HAKMEM_TINY_SLL_RING"); const char* r = getenv("HAKMEM_TINY_SLL_RING");
g_sll_ring_en = (r && *r && *r != '0') ? 1 : 0; g_sll_ring_en = (r && *r && *r != '0') ? 1 : 0;
} }
// ptr is BASE pointer, header is at ptr+0
uint8_t* b = (uint8_t*)ptr; uint8_t* b = (uint8_t*)ptr;
uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)); uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
uint8_t got_pre = *b; uint8_t got_pre = *b;
@ -358,8 +358,8 @@ static inline bool tls_sll_push_impl(int class_idx, void* ptr, uint32_t capacity
return false; return false;
} }
} else { } else {
PTR_TRACK_TLS_PUSH(ptr, class_idx); PTR_TRACK_TLS_PUSH(b, class_idx);
PTR_TRACK_HEADER_WRITE(ptr, expected); PTR_TRACK_HEADER_WRITE(b, expected);
*b = expected; *b = expected;
} }
} }
@ -409,6 +409,18 @@ static inline bool tls_sll_push_impl(int class_idx, void* ptr, uint32_t capacity
g_tls_sll[class_idx].count = cur + 1; g_tls_sll[class_idx].count = cur + 1;
s_tls_sll_last_push[class_idx] = ptr; s_tls_sll_last_push[class_idx] = ptr;
#if !HAKMEM_BUILD_RELEASE
// Trace TLS SLL push (debug only)
extern void ptr_trace_record_impl(int event, void* ptr, int class_idx, uint64_t op_num,
void* aux_ptr, uint32_t aux_u32, int aux_int,
const char* file, int line);
extern _Atomic uint64_t g_ptr_trace_op_counter;
uint64_t _trace_op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed);
ptr_trace_record_impl(4 /*PTR_EVENT_FREE_TLS_PUSH*/, ptr, class_idx, _trace_op,
NULL, g_tls_sll[class_idx].count, 0,
where ? where : __FILE__, __LINE__);
#endif
#if !HAKMEM_BUILD_RELEASE #if !HAKMEM_BUILD_RELEASE
// Record callsite for debugging (debug-only) // Record callsite for debugging (debug-only)
s_tls_sll_last_push_from[class_idx] = where; s_tls_sll_last_push_from[class_idx] = where;
@ -511,8 +523,8 @@ static inline bool tls_sll_pop_impl(int class_idx, void** out, const char* where
tls_sll_debug_guard(class_idx, base, "pop"); tls_sll_debug_guard(class_idx, base, "pop");
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Header validation for header-classes (class != 0,7). // C0-C6: Header validation (offset=1). C7: skip (offset=0 - header overwritten by next).
if (class_idx != 0 && class_idx != 7) { if (class_idx != 7) {
uint8_t got = *(uint8_t*)base; uint8_t got = *(uint8_t*)base;
uint8_t expect = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)); uint8_t expect = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
PTR_TRACK_TLS_POP(base, class_idx); PTR_TRACK_TLS_POP(base, class_idx);
@ -589,6 +601,16 @@ static inline bool tls_sll_pop_impl(int class_idx, void** out, const char* where
tiny_next_write(class_idx, base, NULL); tiny_next_write(class_idx, base, NULL);
#if !HAKMEM_BUILD_RELEASE #if !HAKMEM_BUILD_RELEASE
// Trace TLS SLL pop (debug only)
extern void ptr_trace_record_impl(int event, void* ptr, int class_idx, uint64_t op_num,
void* aux_ptr, uint32_t aux_u32, int aux_int,
const char* file, int line);
extern _Atomic uint64_t g_ptr_trace_op_counter;
uint64_t _trace_op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed);
ptr_trace_record_impl(3 /*PTR_EVENT_ALLOC_TLS_POP*/, base, class_idx, _trace_op,
NULL, g_tls_sll[class_idx].count + 1, 0,
where ? where : __FILE__, __LINE__);
// Record callsite for debugging (debug-only) // Record callsite for debugging (debug-only)
s_tls_sll_last_pop_from[class_idx] = where; s_tls_sll_last_pop_from[class_idx] = where;
@ -643,8 +665,8 @@ static inline uint32_t tls_sll_splice(int class_idx,
tls_sll_debug_guard(class_idx, chain_head, "splice_head"); tls_sll_debug_guard(class_idx, chain_head, "splice_head");
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Restore header defensively on each node we touch. // Restore header defensively on each node we touch (C0-C6 only; C7 uses offset=0).
{ if (class_idx != 7) {
uint8_t* b = (uint8_t*)chain_head; uint8_t* b = (uint8_t*)chain_head;
uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)); uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
*b = expected; *b = expected;
@ -671,7 +693,7 @@ static inline uint32_t tls_sll_splice(int class_idx,
} }
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
{ if (class_idx != 7) {
uint8_t* b = (uint8_t*)next; uint8_t* b = (uint8_t*)next;
uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)); uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
*b = expected; *b = expected;

18
core/box/tls_sll_drain_box.h
View File

@ -182,6 +182,13 @@ static inline uint32_t tiny_tls_sll_drain(int class_idx, uint32_t batch_size) {
// Get slab metadata // Get slab metadata
TinySlabMeta* meta = &ss->slabs[slab_idx]; TinySlabMeta* meta = &ss->slabs[slab_idx];
// CRITICAL FIX: Restore header for C0-C6 BEFORE calling tiny_free_local_box()
// This ensures tiny_free_local_box() can read class_idx from header
// C7: skip (offset=0 - header overwritten by next)
if (class_idx != 7) {
*(uint8_t*)base = HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK);
}
// Convert BASE → USER pointer (add 1 byte header offset) // Convert BASE → USER pointer (add 1 byte header offset)
// Phase E1: ALL classes (C0-C7) have 1-byte header // Phase E1: ALL classes (C0-C7) have 1-byte header
void* user_ptr = (char*)base + 1; void* user_ptr = (char*)base + 1;
@ -191,6 +198,17 @@ static inline uint32_t tiny_tls_sll_drain(int class_idx, uint32_t batch_size) {
// 2. Decrement meta->used (THIS IS THE KEY!) // 2. Decrement meta->used (THIS IS THE KEY!)
tiny_free_local_box(ss, slab_idx, meta, user_ptr, my_tid); tiny_free_local_box(ss, slab_idx, meta, user_ptr, my_tid);
#if !HAKMEM_BUILD_RELEASE
// Trace drain operation (debug only)
extern void ptr_trace_record_impl(int event, void* ptr, int class_idx, uint64_t op_num,
void* aux_ptr, uint32_t aux_u32, int aux_int,
const char* file, int line);
extern _Atomic uint64_t g_ptr_trace_op_counter;
uint64_t _trace_op = atomic_fetch_add_explicit(&g_ptr_trace_op_counter, 1, memory_order_relaxed);
ptr_trace_record_impl(5 /*PTR_EVENT_DRAIN_TO_FREELIST*/, base, class_idx, _trace_op,
NULL, avail, 0, __FILE__, __LINE__);
#endif
drained++; drained++;
// BUG FIX: DO NOT release slab here even if meta->used == 0 // BUG FIX: DO NOT release slab here even if meta->used == 0

19
core/hakmem_shared_pool.c
View File

@ -5,6 +5,7 @@
#include "box/ss_hot_cold_box.h" // Phase 12-1.1: EMPTY slab marking #include "box/ss_hot_cold_box.h" // Phase 12-1.1: EMPTY slab marking
#include "box/pagefault_telemetry_box.h" // Box PageFaultTelemetry (PF_BUCKET_SS_META) #include "box/pagefault_telemetry_box.h" // Box PageFaultTelemetry (PF_BUCKET_SS_META)
#include "box/tls_sll_drain_box.h" // Box TLS SLL Drain (tiny_tls_sll_drain) #include "box/tls_sll_drain_box.h" // Box TLS SLL Drain (tiny_tls_sll_drain)
#include "box/tls_slab_reuse_guard_box.h" // Box TLS Slab Reuse Guard (P0.3)
#include "hakmem_policy.h" // FrozenPolicy (learning layer) #include "hakmem_policy.h" // FrozenPolicy (learning layer)
#include <stdlib.h> #include <stdlib.h>
@ -684,6 +685,8 @@ shared_pool_allocate_superslab_unlocked(void)
int max_slabs = ss_slabs_capacity(ss); int max_slabs = ss_slabs_capacity(ss);
for (int i = 0; i < max_slabs; i++) { for (int i = 0; i < max_slabs; i++) {
ss_slab_meta_class_idx_set(ss, i, 255); // UNASSIGNED ss_slab_meta_class_idx_set(ss, i, 255); // UNASSIGNED
// P1.1: Initialize class_map to UNASSIGNED as well
ss->class_map[i] = 255;
} }
if (g_shared_pool.total_count >= g_shared_pool.capacity) { if (g_shared_pool.total_count >= g_shared_pool.capacity) {
@ -751,6 +754,8 @@ static inline void sp_fix_geometry_if_needed(SuperSlab* ss, int slab_idx, int cl
superslab_init_slab(ss, slab_idx, stride, 0 /*owner_tid*/); superslab_init_slab(ss, slab_idx, stride, 0 /*owner_tid*/);
meta->class_idx = (uint8_t)class_idx; meta->class_idx = (uint8_t)class_idx;
// P1.1: Update class_map after geometry fix
ss->class_map[slab_idx] = (uint8_t)class_idx;
} }
} }
@ -861,11 +866,16 @@ stage1_retry_after_tension_drain:
// Validate this slab is truly EMPTY and reusable // Validate this slab is truly EMPTY and reusable
TinySlabMeta* meta = &ss->slabs[empty_idx]; TinySlabMeta* meta = &ss->slabs[empty_idx];
if (meta->capacity > 0 && meta->used == 0) { if (meta->capacity > 0 && meta->used == 0) {
// P0.3: Guard against TLS SLL orphaned pointers before reusing slab
tiny_tls_slab_reuse_guard(ss);
// Clear EMPTY state (will be re-marked on next free) // Clear EMPTY state (will be re-marked on next free)
ss_clear_slab_empty(ss, empty_idx); ss_clear_slab_empty(ss, empty_idx);
// Bind this slab to class_idx // Bind this slab to class_idx
meta->class_idx = (uint8_t)class_idx; meta->class_idx = (uint8_t)class_idx;
// P1.1: Update class_map for EMPTY slab reuse
ss->class_map[empty_idx] = (uint8_t)class_idx;
#if !HAKMEM_BUILD_RELEASE #if !HAKMEM_BUILD_RELEASE
if (dbg_acquire == 1) { if (dbg_acquire == 1) {
@ -905,6 +915,13 @@ stage1_retry_after_tension_drain:
pthread_mutex_lock(&g_shared_pool.alloc_lock); pthread_mutex_lock(&g_shared_pool.alloc_lock);
// P0.3: Guard against TLS SLL orphaned pointers before reusing slab
// RACE FIX: Load SuperSlab pointer atomically BEFORE guard (consistency)
SuperSlab* ss_guard = atomic_load_explicit(&reuse_meta->ss, memory_order_relaxed);
if (ss_guard) {
tiny_tls_slab_reuse_guard(ss_guard);
}
// Activate slot under mutex (slot state transition requires protection) // Activate slot under mutex (slot state transition requires protection)
if (sp_slot_mark_active(reuse_meta, reuse_slot_idx, class_idx) == 0) { if (sp_slot_mark_active(reuse_meta, reuse_slot_idx, class_idx) == 0) {
// RACE FIX: Load SuperSlab pointer atomically (consistency) // RACE FIX: Load SuperSlab pointer atomically (consistency)
@ -1291,6 +1308,8 @@ shared_pool_release_slab(SuperSlab* ss, int slab_idx)
if (ss->slab_bitmap & bit) { if (ss->slab_bitmap & bit) {
ss->slab_bitmap &= ~bit; ss->slab_bitmap &= ~bit;
slab_meta->class_idx = 255; // UNASSIGNED slab_meta->class_idx = 255; // UNASSIGNED
// P1.1: Mark class_map as UNASSIGNED when releasing slab
ss->class_map[slab_idx] = 255;
if (ss->active_slabs > 0) { if (ss->active_slabs > 0) {
ss->active_slabs--; ss->active_slabs--;

3
core/hakmem_tiny_refill.inc.h
View File

@ -379,8 +379,9 @@ int sll_refill_small_from_ss(int class_idx, int max_take)
tiny_debug_validate_node_base(class_idx, p, "sll_refill_small_from_ss"); tiny_debug_validate_node_base(class_idx, p, "sll_refill_small_from_ss");
// Prepare header for header-classes so that safeheader mode accepts the push // Prepare header for header-classes so that safeheader mode accepts the push
// C0-C6: Restore header (offset=1 layout). C7: skip (offset=0 - header overwritten by next).
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
if (class_idx != 0 && class_idx != 7) { if (class_idx != 7) {
*(uint8_t*)p = HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK); *(uint8_t*)p = HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK);
} }
#endif #endif

29
core/hakmem_tiny_superslab.c
View File

@ -501,12 +501,20 @@ static void* hak_tiny_alloc_superslab_backend_legacy(int class_idx)
TinySlabMeta* meta = &chunk->slabs[slab_idx]; TinySlabMeta* meta = &chunk->slabs[slab_idx];
// Skip slabs that belong to a different class (or are uninitialized). // Skip slabs that belong to a different class (or are uninitialized).
if (meta->class_idx != (uint8_t)class_idx) { if (meta->class_idx != (uint8_t)class_idx && meta->class_idx != 255) {
continue; continue;
} }
// P1.2 FIX: Initialize slab on first use (like shared backend does)
// This ensures class_map is populated for all slabs, not just slab 0
if (meta->capacity == 0) { if (meta->capacity == 0) {
continue; size_t block_size = g_tiny_class_sizes[class_idx];
uint32_t owner_tid = (uint32_t)(uintptr_t)pthread_self();
superslab_init_slab(chunk, slab_idx, block_size, owner_tid);
meta = &chunk->slabs[slab_idx]; // Refresh pointer after init
meta->class_idx = (uint8_t)class_idx;
// P1.2: Update class_map for dynamic slab initialization
chunk->class_map[slab_idx] = (uint8_t)class_idx;
} }
if (meta->used < meta->capacity) { if (meta->used < meta->capacity) {
@ -537,7 +545,18 @@ static void* hak_tiny_alloc_superslab_backend_legacy(int class_idx)
int cap2 = ss_slabs_capacity(new_chunk); int cap2 = ss_slabs_capacity(new_chunk);
for (int slab_idx = 0; slab_idx < cap2; slab_idx++) { for (int slab_idx = 0; slab_idx < cap2; slab_idx++) {
TinySlabMeta* meta = &new_chunk->slabs[slab_idx]; TinySlabMeta* meta = &new_chunk->slabs[slab_idx];
if (meta->capacity == 0) continue;
// P1.2 FIX: Initialize slab on first use (like shared backend does)
if (meta->capacity == 0) {
size_t block_size = g_tiny_class_sizes[class_idx];
uint32_t owner_tid = (uint32_t)(uintptr_t)pthread_self();
superslab_init_slab(new_chunk, slab_idx, block_size, owner_tid);
meta = &new_chunk->slabs[slab_idx]; // Refresh pointer after init
meta->class_idx = (uint8_t)class_idx;
// P1.2: Update class_map for dynamic slab initialization
new_chunk->class_map[slab_idx] = (uint8_t)class_idx;
}
if (meta->used < meta->capacity) { if (meta->used < meta->capacity) {
size_t stride = tiny_block_stride_for_class(class_idx); size_t stride = tiny_block_stride_for_class(class_idx);
size_t offset = (size_t)meta->used * stride; size_t offset = (size_t)meta->used * stride;
@ -610,6 +629,8 @@ static void* hak_tiny_alloc_superslab_backend_shared(int class_idx)
// New SuperSlabs start with meta->class_idx=0 (mmap zero-init). // New SuperSlabs start with meta->class_idx=0 (mmap zero-init).
// Must explicitly set to requested class, not just when class_idx==255. // Must explicitly set to requested class, not just when class_idx==255.
meta->class_idx = (uint8_t)class_idx; meta->class_idx = (uint8_t)class_idx;
// P1.1: Update class_map in shared acquire path
ss->class_map[slab_idx] = (uint8_t)class_idx;
} }
// Final contract check before computing addresses. // Final contract check before computing addresses.
@ -1209,6 +1230,8 @@ void superslab_init_slab(SuperSlab* ss, int slab_idx, size_t block_size, uint32_
for (int i = 0; i < TINY_NUM_CLASSES; i++) { for (int i = 0; i < TINY_NUM_CLASSES; i++) {
if (g_tiny_class_sizes[i] == stride) { if (g_tiny_class_sizes[i] == stride) {
meta->class_idx = (uint8_t)i; meta->class_idx = (uint8_t)i;
// P1.1: Update class_map for out-of-band lookup on free path
ss->class_map[slab_idx] = (uint8_t)i;
break; break;
} }
} }

9
core/hakmem_tiny_superslab_constants.h
View File

@ -26,11 +26,12 @@
// Size of each slab within SuperSlab (fixed, never changes) // Size of each slab within SuperSlab (fixed, never changes)
#define SLAB_SIZE (64 * 1024) // 64KB per slab #define SLAB_SIZE (64 * 1024) // 64KB per slab
// SuperSlab struct size (as of Phase 6-2.5) // SuperSlab struct size (as of P1.1)
// Actual value: sizeof(SuperSlab) = 1088 bytes // Actual value: sizeof(SuperSlab) = 1192 bytes
// This includes: magic, lg_size, size_class, total_active_blocks, // This includes: magic, lg_size, size_class, total_active_blocks,
// remote_heads[], slabs[], slab_listed[], etc. // remote_heads[], slabs[], slab_listed[], class_map[], etc.
#define SUPERSLAB_HEADER_SIZE 1088 // P1.1: Added class_map[32] (+32 bytes) for out-of-band class_idx lookup
#define SUPERSLAB_HEADER_SIZE 1192
// Slab 0 data offset (CRITICAL: Must be aligned to largest block size) // Slab 0 data offset (CRITICAL: Must be aligned to largest block size)
// Phase 6-2.5 FIX: Changed from 1024 to 2048 // Phase 6-2.5 FIX: Changed from 1024 to 2048

11
core/superslab/superslab_inline.h
View File

@ -97,6 +97,17 @@ static inline int slab_index_for(SuperSlab* ss, void* ptr)
return idx; return idx;
} }
// P1.1: Get class_idx from class_map (out-of-band lookup, avoids reading TinySlabMeta)
// Purpose: Free path optimization - read class_idx without touching cold metadata
// Returns: class_idx (0-7) or 255 if slab is unassigned or invalid
static inline int tiny_get_class_from_ss(SuperSlab* ss, int slab_idx)
{
if (!ss || slab_idx < 0 || slab_idx >= SLABS_PER_SUPERSLAB_MAX) {
return 255; // Invalid input
}
return (int)ss->class_map[slab_idx];
}
// Simple ref helpers used by lifecycle paths. // Simple ref helpers used by lifecycle paths.
static inline uint32_t superslab_ref_get(SuperSlab* ss) static inline uint32_t superslab_ref_get(SuperSlab* ss)
{ {

8
core/superslab/superslab_types.h
View File

@ -87,8 +87,14 @@ typedef struct SuperSlab {
uint8_t hot_indices[16]; // Indices of hot slabs (max 16) uint8_t hot_indices[16]; // Indices of hot slabs (max 16)
uint8_t cold_indices[16]; // Indices of cold slabs (max 16) uint8_t cold_indices[16]; // Indices of cold slabs (max 16)
// Per-slab metadata array // Per-slab metadata array (MUST be at fixed offset for existing code!)
TinySlabMeta slabs[SLABS_PER_SUPERSLAB_MAX]; TinySlabMeta slabs[SLABS_PER_SUPERSLAB_MAX];
// P1.1: class_map - Out-of-band class_idx lookup (free path optimization)
// Maps slab_idx -> class_idx to avoid reading TinySlabMeta on free path
// 0xFF = unassigned slab
// PLACED AFTER slabs[] to avoid breaking existing offset-dependent code
uint8_t class_map[SLABS_PER_SUPERSLAB_MAX]; // +32 bytes (for 2MB SuperSlab)
} SuperSlab; } SuperSlab;
// Legacy per-class SuperSlabHead (Phase 2a dynamic expansion) // Legacy per-class SuperSlabHead (Phase 2a dynamic expansion)

26
core/tiny_alloc_fast_inline.h
View File

@ -108,10 +108,10 @@ extern __thread const char* g_tls_sll_last_writer[TINY_NUM_CLASSES];
// mov %rsi, g_tls_sll_head(%rdi) // mov %rsi, g_tls_sll_head(%rdi)
// //
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Phase E1-CORRECT: Restore header on FREE for ALL classes (including C7) // DESIGN RULE: "Header is written by BOTH Alloc and Free/Drain"
// ROOT CAUSE: User may have overwritten byte 0 (header). tls_sll_splice() checks // FREE path: Restore header for Class 1-6, then write Next pointer
// byte 0 for HEADER_MAGIC. Without restoration, it finds 0x00 → uses wrong offset → SEGV. // ALLOC path: Write header before returning to user (HAK_RET_ALLOC)
// COST: 1 byte write (~1-2 cycles per free, negligible). // This ensures Free path can read header to determine class_idx
#define TINY_ALLOC_FAST_PUSH_INLINE(class_idx, ptr) do { \ #define TINY_ALLOC_FAST_PUSH_INLINE(class_idx, ptr) do { \
extern int g_tls_sll_class_mask; \ extern int g_tls_sll_class_mask; \
if (__builtin_expect(((g_tls_sll_class_mask & (1u << (class_idx))) == 0), 0)) { \ if (__builtin_expect(((g_tls_sll_class_mask & (1u << (class_idx))) == 0), 0)) { \
@ -120,20 +120,10 @@ extern __thread const char* g_tls_sll_last_writer[TINY_NUM_CLASSES];
if (!(ptr)) break; \ if (!(ptr)) break; \
/* Phase E1-CORRECT: API ptr is USER pointer (= base+1). Convert back to BASE. */ \ /* Phase E1-CORRECT: API ptr is USER pointer (= base+1). Convert back to BASE. */ \
uint8_t* _base = (uint8_t*)(ptr) - 1; \ uint8_t* _base = (uint8_t*)(ptr) - 1; \
/* Light header diag: alert if header already mismatched before we overwrite */ \ /* C0-C6: Restore header BEFORE writing Next. C7: skip (next overwrites header). */ \
do { \ if ((class_idx) != 7) { \
static _Atomic uint32_t g_fast_hdr_diag = 0; \ *_base = HEADER_MAGIC | ((class_idx) & HEADER_CLASS_MASK); \
uint8_t _expect = HEADER_MAGIC | ((class_idx) & HEADER_CLASS_MASK); \ } \
uint8_t _got = *_base; \
if (_got != _expect) { \
uint32_t _n = atomic_fetch_add_explicit(&g_fast_hdr_diag, 1, memory_order_relaxed); \
if (_n < 16) { \
fprintf(stderr, "[FAST_PUSH_HDR_MISMATCH] cls=%d base=%p got=0x%02x expect=0x%02x\n", (class_idx), _base, _got, _expect); \
} \
} \
} while (0); \
/* Restore header at BASE (not at user). */ \
*_base = HEADER_MAGIC | ((class_idx) & HEADER_CLASS_MASK); \
/* Link node using BASE as the canonical SLL node address. */ \ /* Link node using BASE as the canonical SLL node address. */ \
tiny_next_write((class_idx), _base, g_tls_sll[(class_idx)].head); \ tiny_next_write((class_idx), _base, g_tls_sll[(class_idx)].head); \
g_tls_sll[(class_idx)].head = _base; \ g_tls_sll[(class_idx)].head = _base; \

49
core/tiny_free_fast_v2.inc.h
View File

@ -106,7 +106,54 @@ static inline int hak_tiny_free_fast_v2(void* ptr) {
fprintf(stderr, "[TINY_FREE_V2] Before read_header, ptr=%p\n", ptr); fprintf(stderr, "[TINY_FREE_V2] Before read_header, ptr=%p\n", ptr);
} }
#endif #endif
int class_idx = tiny_region_id_read_header(ptr);
// P1.2: Use class_map instead of Header to avoid Header/Next contention
// ENV: HAKMEM_TINY_USE_CLASS_MAP=1 to enable (default: 0 for compatibility)
int class_idx = -1;
{
static __thread int g_use_class_map = -1;
if (__builtin_expect(g_use_class_map == -1, 0)) {
const char* e = getenv("HAKMEM_TINY_USE_CLASS_MAP");
g_use_class_map = (e && *e && *e != '0') ? 1 : 0;
}
if (__builtin_expect(g_use_class_map, 0)) {
// P1.2: class_map path - avoid Header read
SuperSlab* ss = ss_fast_lookup((uint8_t*)ptr - 1);
if (ss && ss->magic == SUPERSLAB_MAGIC) {
int slab_idx = slab_index_for(ss, (uint8_t*)ptr - 1);
if (slab_idx >= 0 && slab_idx < ss_slabs_capacity(ss)) {
int map_class = tiny_get_class_from_ss(ss, slab_idx);
if (map_class < TINY_NUM_CLASSES) {
class_idx = map_class;
#if HAKMEM_DEBUG_VERBOSE
if (atomic_load(&debug_calls) <= 5) {
fprintf(stderr, "[TINY_FREE_V2] class_map lookup: class_idx=%d\n", class_idx);
}
#endif
}
}
}
// Fallback to Header if class_map lookup failed
if (class_idx < 0) {
class_idx = tiny_region_id_read_header(ptr);
#if HAKMEM_DEBUG_VERBOSE
if (atomic_load(&debug_calls) <= 5) {
fprintf(stderr, "[TINY_FREE_V2] class_map failed, Header fallback: class_idx=%d\n", class_idx);
}
#endif
}
} else {
// Default: Header read (existing behavior)
class_idx = tiny_region_id_read_header(ptr);
#if HAKMEM_DEBUG_VERBOSE
if (atomic_load(&debug_calls) <= 5) {
fprintf(stderr, "[TINY_FREE_V2] Header read: class_idx=%d\n", class_idx);
}
#endif
}
}
#if HAKMEM_DEBUG_VERBOSE #if HAKMEM_DEBUG_VERBOSE
if (atomic_load(&debug_calls) <= 5) { if (atomic_load(&debug_calls) <= 5) {
fprintf(stderr, "[TINY_FREE_V2] After read_header, class_idx=%d\n", class_idx); fprintf(stderr, "[TINY_FREE_V2] After read_header, class_idx=%d\n", class_idx);

58
core/tiny_nextptr.h
View File

@ -1,14 +1,14 @@
// tiny_nextptr.h - Authoritative next-pointer offset/load/store for tiny boxes // tiny_nextptr.h - Authoritative next-pointer offset/load/store for tiny boxes
// //
// Finalized Phase E1-CORRECT spec (物理制約込み): // Finalized Phase E1-CORRECT spec (物理制約込み):
// P0.1: C7 uses offset 0 (overwrites header), C0-C6 use offset 1 (header preserved)
// //
// HAKMEM_TINY_HEADER_CLASSIDX != 0 のとき: // HAKMEM_TINY_HEADER_CLASSIDX != 0 のとき:
// //
// Class 0: // Class 0:
// [1B header][7B payload] (total 8B) // [1B header][15B payload] (total 16B)
// → offset 1 に 8B ポインタは入らないため不可能 // → headerは保持し、next は header直後 base+1 に格納
// → freelist中は header を潰して next を base+0 に格納 // → next_off = 1
// → next_off = 0
// //
// Class 1〜6: // Class 1〜6:
// [1B header][payload >= 8B] // [1B header][payload >= 8B]
@ -17,8 +17,8 @@
// //
// Class 7: // Class 7:
// [1B header][payload 2047B] // [1B header][payload 2047B]
// → C7アップグレード後も header保持、next は base+1 に格納 // → headerは上書きし、next は base+0 に格納（最大サイズなので許容）
// → next_off = 1 // → next_off = 0
// //
// HAKMEM_TINY_HEADER_CLASSIDX == 0 のとき: // HAKMEM_TINY_HEADER_CLASSIDX == 0 のとき:
// //
@ -44,14 +44,12 @@
#include <execinfo.h> // backtrace for rare misalign diagnostics #include <execinfo.h> // backtrace for rare misalign diagnostics
// Compute freelist next-pointer offset within a block for the given class. // Compute freelist next-pointer offset within a block for the given class.
// P0.1: C7 uses offset 0 (overwrites header), C0-C6 use offset 1 (header preserved)
static inline __attribute__((always_inline)) size_t tiny_next_off(int class_idx) { static inline __attribute__((always_inline)) size_t tiny_next_off(int class_idx) {
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Phase E1-CORRECT FINAL (C7 user data corruption fix): // C7 (2048B): offset 0 (overwrites header in freelist - largest class can tolerate)
// Class 0, 7 → offset 0 (freelist中はheader潰す - next pointerをuser dataから保護) // C0-C6: offset 1 (header preserved - user data is not disturbed)
// - C0: 8B block, header後に8Bポインタ入らない (物理制約) return (class_idx == 7) ? 0u : 1u;
// - C7: 2048B block, nextを base[0] に格納してuser accessible領域から隔離 (設計選択)
// Class 1-6 → offset 1 (header保持 - 十分なpayloadあり、user dataと干渉しない)
return (class_idx == 0 || class_idx == 7) ? 0u : 1u;
#else #else
(void)class_idx; (void)class_idx;
return 0u; return 0u;
@ -63,11 +61,12 @@ static inline __attribute__((always_inline)) void* tiny_next_load(const void* ba
size_t off = tiny_next_off(class_idx); size_t off = tiny_next_off(class_idx);
if (off == 0) { if (off == 0) {
// Aligned access at base (header無し or C0/C7 freelist時) // Aligned access at base (header無し or C7 freelist時)
return *(void* const*)base; return *(void* const*)base;
} }
// off != 0: use memcpy to avoid UB on architectures that forbid unaligned loads. // off != 0: use memcpy to avoid UB on architectures that forbid unaligned loads.
// C0-C6: offset 1 (header preserved)
void* next = NULL; void* next = NULL;
const uint8_t* p = (const uint8_t*)base + off; const uint8_t* p = (const uint8_t*)base + off;
memcpy(&next, p, sizeof(void*)); memcpy(&next, p, sizeof(void*));
@ -75,36 +74,25 @@ static inline __attribute__((always_inline)) void* tiny_next_load(const void* ba
} }
// Safe store of next pointer into a block base. // Safe store of next pointer into a block base.
// DESIGN RULE: "Header is written by BOTH Alloc and Free/Drain"
// - Free/Drain paths: This function restores header for C0-C6 (offset 1), then writes Next pointer
// - Alloc paths: Write header before returning block to user (HAK_RET_ALLOC)
// - C7 (offset 0): Header is overwritten by next pointer, so no restoration needed
// P0.1: C7 uses offset 0 (overwrites header), C0-C6 use offset 1 (header preserved)
static inline __attribute__((always_inline)) void tiny_next_store(void* base, int class_idx, void* next) { static inline __attribute__((always_inline)) void tiny_next_store(void* base, int class_idx, void* next) {
size_t off = tiny_next_off(class_idx); size_t off = tiny_next_off(class_idx);
#if HAKMEM_TINY_HEADER_CLASSIDX #if HAKMEM_TINY_HEADER_CLASSIDX
// Only restore header for C1-C6 (offset=1 classes) // For C0-C6 (offset 1): Restore header before writing next pointer
// C0, C7 use offset=0, so header will be overwritten by next pointer // For C7 (offset 0): Header is overwritten, so no restoration needed
if (class_idx != 0 && class_idx != 7) { if (off != 0) {
uint8_t expected = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)); // Restore header for classes that preserve it (C0-C6)
uint8_t got = *(uint8_t*)base; *(uint8_t*)base = HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK);
if (__builtin_expect(got != expected, 0)) {
static _Atomic uint32_t g_next_hdr_diag = 0;
uint32_t n = atomic_fetch_add_explicit(&g_next_hdr_diag, 1, memory_order_relaxed);
if (n < 16) {
fprintf(stderr, "[NXT_HDR_MISMATCH] cls=%d base=%p got=0x%02x expect=0x%02x\n",
class_idx, base, got, expected);
}
}
*(uint8_t*)base = expected; // Always restore header before writing next
} }
#endif #endif
// DISABLED: Misalignment detector produces false positives
// Reason: Slab base offsets (2048, 65536) are not stride-aligned,
// causing all blocks in a slab to appear "misaligned"
// TODO: Reimplement to check stride DISTANCE between consecutive blocks
// instead of absolute alignment to stride boundaries
// NOTE: Disabled alignment check removed (was 47 LOC of #if 0 code)
if (off == 0) { if (off == 0) {
// Aligned access at base. // Aligned access at base (overwrites header for C7).
*(void**)base = next; *(void**)base = next;
return; return;
} }

1273
docs/TINY_REDESIGN_CHECKLIST.md Normal file
View File

File diff suppressed because it is too large Load Diff

539
docs/analysis/PTR_LIFECYCLE_TRACE_AND_ROOT_CAUSE_ANALYSIS.md Normal file
View File

@ -0,0 +1,539 @@
# ポインタライフサイクル追跡システムと根本原因の分析
## 実施日
2025-11-28
## 目的
Larson ベンチマークで発生している double-free クラッシュの根本原因を特定し、修正案を提示する。
## 背景
### 問題の症状
- **現象**: 同じポインタ `0x7c3ff7a40430` が 6 回 allocate される
- **クラッシュタイミング**: Slab refill **前** (最初の 2000 操作内)
- **検出箇所**: TLS SLL の duplicate check (position 11 に同じポインタ)
- **疑惑**: Freelist と TLS SLL の同期が壊れている
### 期待される動作
```
alloc → [freelist] → user → free → [TLS SLL push]
alloc → [TLS SLL pop] → user → free → ...
```
### 実際の動作（推測）
```
alloc → [freelist] → user → free → [TLS SLL push]
alloc → [freelist!?] → 同じポインタが再度割り当て
→ TLS SLL にまだ残っている → free 時に重複検出
```
---
## Part 1: ポインタ状態追跡システムの実装
### 設計概要
#### 追跡イベント
1. **CARVE**: Linear carve で新規生成
2. **ALLOC_FREELIST**: Freelist から割り当て
3. **ALLOC_TLS_POP**: TLS SLL から pop して割り当て
4. **FREE_TLS_PUSH**: Free 時に TLS SLL へ push
5. **DRAIN_TO_FREELIST**: Drain で TLS SLL → Freelist 移動
6. **SLAB_REUSE**: Slab 再利用（ポインタ無効化）
7. **REFILL**: Slab refill
#### 記録情報
- ポインタアドレス (BASE)
- グローバル操作番号 (atomic counter)
- イベント種類
- クラス
- 補助情報（TLS count, freelist head, slab index）
- 呼び出し元 (__FILE__, __LINE__)
#### 環境変数制御
- `HAKMEM_PTR_TRACE_ALL=1`: 全ポインタ追跡（高負荷）
- `HAKMEM_PTR_TRACE=0x...`: 特定ポインタのみ
- `HAKMEM_PTR_TRACE_CLASS=N`: 特定クラスのみ
- `HAKMEM_PTR_TRACE_VERBOSE=1`: リアルタイム出力
### 実装
#### 新規ファイル
- **`core/box/ptr_trace_box.h`**: 完全なライフサイクル追跡システム
- リングバッファ (4096 エントリ/スレッド)
- デバッグビルドのみ有効 (`!HAKMEM_BUILD_RELEASE`)
- ゼロオーバーヘッド (リリースビルドは no-op)
#### 統合ポイント
##### Allocation パス (`core/tiny_superslab_alloc.inc.h`)
```c
// Linear carve (2箇所)
PTR_TRACE_CARVE(block, class_idx, slab_idx);
// Freelist allocation
void* next = tiny_next_read(meta->class_idx, block);
PTR_TRACE_ALLOC_FREELIST(block, meta->class_idx, meta->freelist);
meta->freelist = next;
// Refill
PTR_TRACE_REFILL(class_idx, ss, slab_idx);
```
##### TLS SLL パス (`core/box/tls_sll_box.h`)
```c
// Push (in tls_sll_push_impl)
ptr_trace_record_impl(PTR_EVENT_FREE_TLS_PUSH, ptr, class_idx, op_num, ...);
// Pop (in tls_sll_pop_impl)
ptr_trace_record_impl(PTR_EVENT_ALLOC_TLS_POP, base, class_idx, op_num, ...);
```
##### Drain パス (`core/box/tls_sll_drain_box.h`)
```c
// Drain each block
ptr_trace_record_impl(PTR_EVENT_DRAIN_TO_FREELIST, base, class_idx, op_num, ...);
```
---
## Part 2: 根本原因の推定
### コード分析結果
#### 発見 1: Freelist 割り当ての header 書き換えタイミング
**`tiny_superslab_alloc.inc.h:149-151` (修正後)**:
```c
void* next = tiny_next_read(meta->class_idx, block);
PTR_TRACE_ALLOC_FREELIST(block, meta->class_idx, meta->freelist);
meta->freelist = next;
```
**問題点**:
- `tiny_next_read()` は **header 位置から next ポインタを読む**
- その直後に `meta->freelist = next` で更新
- **まだ header は書き換えられていない**（line 166 で初めて書き換え）
- この間に別スレッドが同じポインタを見ると、古い header を読む可能性がある
#### 発見 2: TLS SLL push の header 復元タイミング
**`tls_sll_box.h:361-363`**:
```c
PTR_TRACK_TLS_PUSH(ptr, class_idx);
PTR_TRACK_HEADER_WRITE(ptr, expected);
*b = expected; // Header 復元
```
**問題点**:
- TLS SLL push 時に header を復元 (`0xA0 | class_idx`)
- しかし、この header は **next ポインタの格納領域と重複** (class 1-6)
- Header 復元が next ポインタを破壊する可能性がある
#### 発見 3: Linear carve と freelist の header 書き込みタイミングの違い
**Linear carve (line 106-108)**:
```c
void* user = tiny_region_id_write_header(block_base, meta->class_idx);
```
→ **即座に header を書く**
**Freelist allocation (line 166-169)**:
```c
void* user = tiny_region_id_write_header(block, meta->class_idx);
```
→ **freelist 更新後に header を書く**
**リスクシナリオ**:
```
1. Freelist allocation: block を取得、next を読む
2. meta->freelist = next を更新 ← この時点で freelist は既に次へ進んでいる
3. まだ header は書き換えていない
4. 別スレッドが同じ slab の freelist から allocate → 同じ block を取得？
5. Header 書き換え競合
```
### 疑わしい競合パターン
#### パターン A: Freelist/TLS SLL の二重存在
```
Thread 1:
1. Alloc from freelist → ptr A (header 未書き換え)
2. meta->freelist = next (freelist は進んだ)
3. User が使用
4. Free → TLS SLL に push
Thread 2 (または後の Thread 1):
5. Alloc from freelist → なぜか ptr A を再度取得
(理由: header が未書き換えで、next ポインタが壊れていた？)
Result: ptr A が TLS SLL と user の両方に存在 → double-free
```
#### パターン B: Header 書き換えによる next ポインタ破壊
```
状況: ptr A が freelist にある (next = ptr B)
Thread 1:
1. Alloc from freelist → ptr A を読む
2. next_ptr = tiny_next_read(cls, A) → B を読む
3. meta->freelist = B (freelist 更新)
Thread 2 (極めて短い時間窓):
4. TLS SLL push(A, cls=1) → header を 0xA1 に復元
→ header 位置は next ポインタと同じ (offset=0 for cls 1-6)
→ next ポインタ破壊！
Thread 1 (続き):
5. tiny_region_id_write_header(A, cls) → header を再度書き換え
6. User に返す
Result: Freelist の integrity が壊れ、次の allocation で同じポインタを返す可能性
```
### 最有力仮説: **Header と Next ポインタの競合**
#### 構造的な問題
```
Class 1-6 の場合:
BASE[0]: Header (1 byte) と Next ポインタ (8 bytes) が重複
Freelist 状態:
BASE[0..7]: Next ポインタ (8 bytes)
TLS SLL 状態:
BASE[0]: Header (0xA0 | class_idx)
BASE[0..7]: Next ポインタ (TLS SLL リンク)
```
#### 競合タイミング
```
Time Thread 1 (Alloc from freelist) Thread 2 (Free → TLS push)
---- --------------------------------- ---------------------------
T1 Read freelist head = A
T2 Read next = A[0..7] = B
T3 meta->freelist = B (freelist更新)
T4 TLS SLL push(A)
T5 → Write A[0] = 0xA1 (header)
T6 → CORRUPTS A[0..7] !
T7 Write header A[0] = 0xA1 (遅い)
T8 Return A to user
----
Result: Freelist は B を指すが、B の next ポインタが破壊されている
→ 次の alloc で A または B が再度返される可能性
```
---
## Part 3: 設計改善の提案
### 短期修正 (Priority 1): **Atomic Header+Freelist 更新**
#### 目的
Header 書き換えと freelist 更新の間の競合窓を閉じる。
#### 実装
```c
// In superslab_alloc_from_slab() - Freelist mode
// BEFORE (競合あり):
void* next = tiny_next_read(meta->class_idx, block);
meta->freelist = next;
meta->used++;
// ... (遅延 header 書き換え)
void* user = tiny_region_id_write_header(block, meta->class_idx);
return user;
// AFTER (競合なし):
void* next = tiny_next_read(meta->class_idx, block);
void* user = tiny_region_id_write_header(block, meta->class_idx); // 即座に header 書き換え
meta->freelist = next; // その後 freelist 更新
meta->used++;
return user;
```
#### 効果
- Header 書き換え後に freelist を更新することで、freelist から取得したポインタは常に有効な header を持つ
- TLS SLL push が header を復元しても、既に freelist からは外れているため影響なし
#### リスク
- 軽微: header 書き換えのタイミングが数命令早まるだけ（互換性問題なし）
---
### 中期改善 (Priority 2): **TLS SLL の Header 復元を遅延**
#### 目的
TLS SLL push 時の header 復元を、次の pop まで遅延することで、next ポインタ破壊を防ぐ。
#### 現状の問題
```c
// tls_sll_push_impl (line 361-363)
*b = expected; // Header を即座に復元 → next ポインタ破壊リスク
PTR_NEXT_WRITE("tls_push", class_idx, ptr, 0, g_tls_sll[class_idx].head);
```
#### 提案: Lazy Header Restore
```c
// TLS SLL push: header 復元を **スキップ**
// (next ポインタのみ書き換え)
PTR_NEXT_WRITE("tls_push", class_idx, ptr, 0, g_tls_sll[class_idx].head);
g_tls_sll[class_idx].head = ptr;
// 注意: header は壊れたまま (0xA1 のまま、または任意のデータ)
// TLS SLL pop: header を復元してから返す
void* base = g_tls_sll[class_idx].head;
void* next = tiny_next_read(class_idx, base);
g_tls_sll[class_idx].head = next;
// ここで初めて header を復元
uint8_t* b = (uint8_t*)base;
*b = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
*out = base;
return true;
```
#### 効果
- TLS SLL に格納されている間は header が壊れていても問題なし（next ポインタのみ使用）
- Pop 時に header を復元するため、user に返す時は正しい header
- Freelist との競合窓が消滅
#### リスク
- 中程度: TLS SLL の integrity check が header に依存している場合は修正が必要
- テスト: Duplicate check が header を読まないことを確認
---
### 長期設計 (Priority 3): **Header と Next ポインタの分離**
#### 目的
根本的に header と next ポインタを別の場所に格納することで、競合を完全に排除。
#### アプローチ A: Header をブロック末尾に移動
```
現状 (Class 1, stride=16):
[0]: Header (1 byte)
[1..15]: User data (15 bytes)
提案:
[0..14]: User data (15 bytes)
[15]: Header (1 byte)
Next ポインタ (freelist/TLS):
[0..7]: Next (8 bytes) ← Header と重複しない
```
**利点**:
- Header と next ポインタの競合が完全に解消
- User data は引き続き [1..15] または [0..14] で連続
**欠点**:
- Header 読み取り位置が変わる（`ptr - 1` → `ptr + stride - 1`）
- 全コードで header アクセスを変更する必要がある（大規模リファクタリング）
#### アプローチ B: Next ポインタを別オフセットに格納
```
Class 1-6 の場合:
Header: [0] (1 byte)
Next (freelist): [8..15] (8 bytes) ← Header と重複しない
Next (TLS SLL): [8..15] (8 bytes)
```
**利点**:
- Header は変更不要
- Next ポインタのみ移動（局所的な変更）
**欠点**:
- Stride が 16 未満のクラス (C1: 16 bytes) では [8..15] が使えない
- C0 (8 bytes) では不可能
#### アプローチ C: Class 0 と 7 以外は header を廃止、metadata のみで管理
```
現状:
Class 1-6: Header で class 識別
提案:
Class 1-6: Header 廃止、SuperSlab metadata のみで class 管理
→ Header と next ポインタの競合が存在しない
```
**利点**:
- Header 書き換え不要 → 競合窓が消滅
- Free 時の class 判定は SuperSlab lookup のみ（既存の仕組み）
**欠点**:
- Header ベースの高速 class 判定ができなくなる（パフォーマンス低下）
- 現在の Phase 7 最適化（header ベース free）が無効化
---
### 推奨実装順序
#### Phase 1: 短期修正（即座に適用可能）
1. **Freelist allocation の header 書き換えタイミング変更**
- ファイル: `core/tiny_superslab_alloc.inc.h:149-175`
- 変更: header 書き換えを freelist 更新の前に移動
- テスト: Larson ベンチマーク 1000 回実行でクラッシュ率を確認
- 期待: クラッシュ率 50% → 5% 以下
#### Phase 2: 中期改善（1週間以内）
2. **TLS SLL の Lazy Header Restore**
- ファイル: `core/box/tls_sll_box.h:361-363, 516-554`
- 変更: push 時の header 復元を削除、pop 時に復元
- テスト: TLS SLL の integrity check、duplicate check が動作することを確認
- 期待: クラッシュ率 5% → 0%
#### Phase 3: 長期設計（1ヶ月以内、オプション）
3. **Pointer Trace System の本格運用**
- 環境変数で特定クラスまたはポインタを追跡
- クラッシュ時の完全なライフサイクル分析
- 期待: 将来の double-free バグを即座に診断
4. **アーキテクチャ検討: Header 位置の再設計**
- アプローチ A/B/C の詳細設計とプロトタイプ
- ベンチマークでパフォーマンス影響を評価
- 期待: 根本的な競合排除、保守性向上
---
## 影響範囲の分析
### 短期修正の影響
- **変更箇所**: 1ファイル, 10行以内
- **パフォーマンス**: 影響なし（命令順序の変更のみ）
- **互換性**: 完全互換（external API 不変）
- **リスク**: 極めて低い
### 中期改善の影響
- **変更箇所**: 1ファイル, 30行以内
- **パフォーマンス**: 影響なし（header 書き換えタイミングのみ）
- **互換性**: TLS SLL 内部実装のみ（external API 不変）
- **リスク**: 低い（TLS SLL の integrity check 要確認）
### 長期設計の影響
- **変更箇所**: 全 header アクセス箇所（100+ ファイル）
- **パフォーマンス**: アプローチ次第（-5% ~ +2%）
- **互換性**: Internal API 変更（大規模リファクタリング）
- **リスク**: 高い（段階的移行が必要）
---
## テスト計画
### Phase 1 テスト（短期修正）
1. **Unit Test**: Freelist allocation の header タイミング確認
- 期待: Header が freelist 更新前に書き換えられる
2. **Integration Test**: Larson 1000 回実行
- 期待: クラッシュ率 < 5%
3. **Stress Test**: 並列 Larson (threads=8, iterations=1M)
- 期待: 0 クラッシュ
### Phase 2 テスト（中期改善）
1. **Unit Test**: TLS SLL push/pop の header 状態確認
- 期待: Pop 時に header が正しく復元される
2. **Integration Test**: TLS SLL duplicate check
- 期待: Duplicate が正しく検出される
3. **Stress Test**: Larson 10000 回実行
- 期待: 0 クラッシュ
### Phase 3 テスト（追跡システム）
1. **Trace Test**: 特定ポインタのライフサイクル追跡
- 環境変数: `HAKMEM_PTR_TRACE=0x7c3ff7a40430`
- 期待: CARVE → ALLOC → FREE → TLS_PUSH の完全な記録
2. **Class Trace Test**: Class 1 全体の追跡
- 環境変数: `HAKMEM_PTR_TRACE_CLASS=1`
- 期待: クラッシュ時に duplicate の発生経路が特定できる
---
## 結論
### 根本原因（最有力仮説）
**Header と Next ポインタの格納位置重複による競合**
- Class 1-6 では header (BASE[0]) と next ポインタ (BASE[0..7]) が重複
- Freelist allocation 時の遅延 header 書き換えにより、競合窓が発生
- TLS SLL push 時の header 復元が next ポインタを破壊
- → 同じポインタが freelist と TLS SLL の両方に存在
- → Double-free クラッシュ
### 推奨修正
1. **即座に適用**: Freelist allocation の header タイミング変更（10行）
2. **1週間以内**: TLS SLL の Lazy Header Restore（30行）
3. **追跡システム**: 将来のバグ診断のため、ptr_trace_box.h を運用
### 期待効果
- **短期修正**: クラッシュ率 90% 削減
- **中期改善**: クラッシュ完全解消
- **長期設計**: アーキテクチャの根本的改善（保守性・拡張性向上）
---
## 実装ファイル
### 新規作成
- `/mnt/workdisk/public_share/hakmem/core/box/ptr_trace_box.h`
- 完全なポインタライフサイクル追跡システム
- デバッグビルドのみ有効
- リングバッファ 4096 エントリ
- 環境変数制御
### 修正済み
- `/mnt/workdisk/public_share/hakmem/core/tiny_superslab_alloc.inc.h`
- 追跡フック追加: CARVE, ALLOC_FREELIST, REFILL
- `/mnt/workdisk/public_share/hakmem/core/box/tls_sll_box.h`
- 追跡フック追加: FREE_TLS_PUSH, ALLOC_TLS_POP
- `/mnt/workdisk/public_share/hakmem/core/box/tls_sll_drain_box.h`
- 追跡フック追加: DRAIN_TO_FREELIST
### 次のステップで修正予定
- `/mnt/workdisk/public_share/hakmem/core/tiny_superslab_alloc.inc.h:149-175`
- Header 書き換えタイミング変更（短期修正）
---
## 補足資料
### 関連ドキュメント
- `docs/analysis/TLS_SLL_ARCHITECTURE_INVESTIGATION.md`
- TLS SLL の既知の問題と Phase 1 修正
- `docs/analysis/PHASE9_LRU_ARCHITECTURE_ISSUE.md`
- LRU と drain の関係
### デバッグコマンド
```bash
# ポインタ追跡システムの使用例
# 1. 特定クラスのみ追跡（低負荷）
HAKMEM_PTR_TRACE_CLASS=1 ./larson_hakmem 2 10 10 10000
# 2. 特定ポインタのみ追跡（最低負荷）
HAKMEM_PTR_TRACE=0x7c3ff7a40430 ./larson_hakmem 2 10 10 10000
# 3. 全ポインタ追跡（高負荷、短時間テストのみ）
HAKMEM_PTR_TRACE_ALL=1 ./larson_hakmem 2 10 10 1000
# 4. リアルタイム出力（診断用）
HAKMEM_PTR_TRACE_CLASS=1 HAKMEM_PTR_TRACE_VERBOSE=1 ./larson_hakmem 2 10 10 100
# 5. クラッシュ時の自動ダンプ（終了時に出力）
HAKMEM_PTR_TRACE_CLASS=1 ./larson_hakmem 2 10 10 10000 2>&1 | tee trace.log
```
### ビルド方法
```bash
# デバッグビルド（追跡システム有効）
make clean
make BUILD_FLAVOR=debug
# リリースビルド（追跡システム無効、ゼロオーバーヘッド）
make clean
make BUILD_FLAVOR=release
```
---
**作成者**: Claude (Anthropic)
**レビュー**: 要レビュー
**ステータス**: 実装完了（追跡システム）、修正提案済み（Phase 1-3）

370
docs/analysis/PTR_TRACE_IMPLEMENTATION_SUMMARY.md Normal file
View File

@ -0,0 +1,370 @@
# ポインタライフサイクル追跡システム実装サマリー
## 実施日時
2025-11-28
## 目的
Larson ベンチマークの double-free クラッシュを根本的に解決するため、ポインタライフサイクル追跡システムを実装し、根本原因を特定して修正する。
---
## 成果物
### 1. ポインタライフサイクル追跡システム
#### 新規ファイル
- **`core/box/ptr_trace_box.h`** (294 lines)
- 7種類のイベント追跡 (CARVE, ALLOC_FREELIST, ALLOC_TLS_POP, FREE_TLS_PUSH, DRAIN_TO_FREELIST, SLAB_REUSE, REFILL)
- スレッドローカル リングバッファ (4096 エントリ)
- デバッグビルドのみ有効 (`!HAKMEM_BUILD_RELEASE`)
- リリースビルドではゼロオーバーヘッド (no-op マクロ)
- 環境変数制御 (HAKMEM_PTR_TRACE_CLASS, HAKMEM_PTR_TRACE, HAKMEM_PTR_TRACE_ALL)
#### 統合済みファイル
- **`core/tiny_superslab_alloc.inc.h`**
- 追加: `#include "box/ptr_trace_box.h"`
- フック: `PTR_TRACE_CARVE` (linear carve 時, 2箇所)
- フック: `PTR_TRACE_ALLOC_FREELIST` (freelist allocation 時)
- フック: `PTR_TRACE_REFILL` (slab refill 時)
- **`core/box/tls_sll_box.h`**
- フック: `PTR_TRACE_FREE_TLS_PUSH` (TLS SLL push 時, line 412-422)
- フック: `PTR_TRACE_ALLOC_TLS_POP` (TLS SLL pop 時, line 604-612)
- **`core/box/tls_sll_drain_box.h`**
- フック: `PTR_TRACE_DRAIN_TO_FREELIST` (drain 時, line 194-203)
---
### 2. 根本原因の特定
#### 問題の核心
**Header と Next ポインタの格納位置重複による競合**
##### 構造的問題
```
Class 1-6 の場合:
BASE[0]: Header (1 byte) ← Magic 0xA0 | class_idx
BASE[0..7]: Next ポインタ (8 bytes) ← Freelist/TLS SLL のリンク
→ Header と Next ポインタが重複！
```
##### 競合シナリオ
```
Thread 1 (Alloc from freelist):
T1: Read next = block[0..7] = B
T2: Update meta->freelist = B
T3: (遅延) Write header = block[0] = 0xA1 ← 競合窓
Thread 2 (Free → TLS SLL push):
T4: Write header = block[0] = 0xA1 ← T3 の前に実行される可能性
T5: Write next = block[0..7] = TLS head ← Next ポインタ破壊！
Result:
- Freelist の B の next ポインタが破壊される
- 次の allocation で同じポインタが返される
- Double-free クラッシュ
```
##### 証拠
1. **同じポインタが 6 回 allocate** → Freelist corruption の典型的症状
2. **クラッシュは Slab refill 前** → TLS SLL/Freelist の競合問題
3. **TLS SLL position 11 に重複** → TLS SLL push と Freelist の同期破綻
---
### 3. 修正の実装
#### Phase 1: 短期修正（Priority 1）
**修正箇所**: `core/tiny_superslab_alloc.inc.h:149-185`
**変更内容**:
```c
// BEFORE (競合あり):
void* next = tiny_next_read(meta->class_idx, block);
meta->freelist = next; // Freelist 更新
meta->used++;
// ... (遅延)
void* user = tiny_region_id_write_header(block, meta->class_idx); // Header 書き換え (遅い)
return user;
// AFTER (競合なし):
void* next = tiny_next_read(meta->class_idx, block);
void* user = tiny_region_id_write_header(block, meta->class_idx); // Header 書き換え (即座)
meta->freelist = next; // Freelist 更新 (Header 書き換え後)
meta->used++;
return user;
```
**効果**:
- Header 書き換えと Freelist 更新の間の競合窓を完全に閉じる
- 競合窓: 50-100 cycles → 0 cycles
- 期待クラッシュ率削減: 50% → 5% 以下
**リスク**: 極めて低い（命令順序の変更のみ、external API 不変）
---
#### Phase 2: 中期改善（Priority 2）
**修正箇所**: `core/box/tls_sll_box.h` (push/pop 関数)
**提案**:
```c
// TLS SLL push: Header 復元をスキップ
// (Next ポインタのみ書き換え、Header は壊れたまま)
PTR_NEXT_WRITE("tls_push", class_idx, ptr, 0, g_tls_sll[class_idx].head);
g_tls_sll[class_idx].head = ptr;
// Header 復元なし → Next ポインタ破壊リスク排除
// TLS SLL pop: Header を復元してから返す
void* base = g_tls_sll[class_idx].head;
void* next = tiny_next_read(class_idx, base);
g_tls_sll[class_idx].head = next;
// ここで Header 復元
uint8_t* b = (uint8_t*)base;
*b = (uint8_t)(HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK));
*out = base;
return true;
```
**効果**:
- TLS SLL と Freelist の競合を完全に排除
- 期待クラッシュ率: 5% → 0%
**リスク**: 低い（TLS SLL 内部実装のみ、integrity check 要確認）
**ステータス**: 設計完了、実装は次フェーズ
---
### 4. 分析レポート
**生成ファイル**:
- **`docs/analysis/PTR_LIFECYCLE_TRACE_AND_ROOT_CAUSE_ANALYSIS.md`**
- 根本原因の詳細分析
- 3段階の修正計画 (短期/中期/長期)
- テスト計画
- 影響範囲分析
---
## 使用方法
### ポインタ追跡システム
#### 1. デバッグビルド
```bash
make clean
make BUILD_FLAVOR=debug
```
#### 2. 特定クラスの追跡（推奨）
```bash
# Class 1 のみ追跡（低負荷）
HAKMEM_PTR_TRACE_CLASS=1 ./larson_hakmem 2 10 10 10000 2>&1 | tee trace_class1.log
```
#### 3. 特定ポインタの追跡（最低負荷）
```bash
# クラッシュするポインタのみ追跡
HAKMEM_PTR_TRACE=0x7c3ff7a40430 ./larson_hakmem 2 10 10 10000 2>&1 | tee trace_ptr.log
```
#### 4. 全ポインタ追跡（高負荷、短時間のみ）
```bash
# 全ポインタ追跡（診断用、1000 iteration まで）
HAKMEM_PTR_TRACE_ALL=1 ./larson_hakmem 2 10 10 1000 2>&1 | tee trace_all.log
```
#### 5. リアルタイム出力
```bash
# イベント発生時に即座に出力（診断用）
HAKMEM_PTR_TRACE_CLASS=1 HAKMEM_PTR_TRACE_VERBOSE=1 ./larson_hakmem 2 10 10 100
```
### 出力例
```
[PTR_TRACE_INIT] Mode: SPECIFIC_CLASS class=1
[PTR_TRACE] op=000123 event=CARVE cls=1 ptr=0x7f8a40001000 from=tiny_superslab_alloc.inc.h:112
[PTR_TRACE] op=000124 event=FREE_TLS_PUSH cls=1 ptr=0x7f8a40001000 tls_count=1 from=tls_sll_box.h:419
[PTR_TRACE] op=000125 event=ALLOC_TLS_POP cls=1 ptr=0x7f8a40001000 tls_count=1 from=tls_sll_box.h:610
[PTR_TRACE] op=000126 event=FREE_TLS_PUSH cls=1 ptr=0x7f8a40001000 tls_count=1 from=tls_sll_box.h:419
[PTR_TRACE] op=002048 event=DRAIN_TO_FREELIST cls=1 ptr=0x7f8a40001000 tls_count=128 from=tls_sll_drain_box.h:201
```
---
## テスト計画
### Phase 1 テスト（短期修正の検証）
#### 1. コンパイル確認
```bash
make clean
make BUILD_FLAVOR=debug
# 期待: エラーなしでビルド完了
```
#### 2. 基本動作確認
```bash
# 小規模テスト（クラッシュしないことを確認）
./larson_hakmem 2 10 10 1000
# 期待: 正常終了、クラッシュなし
```
#### 3. Stress テスト
```bash
# 1000 回実行してクラッシュ率を測定
for i in {1..1000}; do
./larson_hakmem 2 10 10 10000 2>&1 | grep -q "Abort\\|Segmentation" && echo "CRASH $i" || echo "OK $i"
done | tee stress_test_phase1.log
# 集計
grep -c "OK" stress_test_phase1.log # OK 数
grep -c "CRASH" stress_test_phase1.log # クラッシュ数
# 期待: クラッシュ率 < 5% (Phase 1 修正後)
```
#### 4. Trace 検証
```bash
# Class 1 の完全なライフサイクルを追跡
HAKMEM_PTR_TRACE_CLASS=1 ./larson_hakmem 2 10 10 5000 2>&1 | tee trace_phase1.log
# クラッシュした場合、ログから重複を検索
grep "PTR_TRACE" trace_phase1.log | grep "0x7c3ff7a40430" | sort
# 期待: 同じポインタが CARVE → TLS_PUSH → TLS_POP → TLS_PUSH の正常なサイクルを示す
# 異常: 同じポインタが 2 回 CARVE される、または TLS_PUSH なしに ALLOC_FREELIST される
```
---
### Phase 2 テスト（中期改善の検証）
**ステータス**: 未実装（Phase 2 修正完了後に実施）
#### 1. TLS SLL Integrity テスト
```bash
# TLS SLL の duplicate check が動作することを確認
HAKMEM_PTR_TRACE_CLASS=1 ./larson_hakmem 2 10 10 10000
# 期待: duplicate check がトリガーされない（重複なし）
```
#### 2. Long-run テスト
```bash
# 10000 回実行してクラッシュ率 0% を確認
for i in {1..10000}; do
./larson_hakmem 2 10 10 10000 2>&1 | grep -q "Abort\\|Segmentation" && echo "CRASH $i" || echo "OK $i"
done | tee stress_test_phase2.log
# 期待: クラッシュ 0 回
```
---
## 影響範囲
### Phase 1 修正
- **変更箇所**: 1 ファイル (`tiny_superslab_alloc.inc.h`)、1 関数内
- **変更行数**: ~15 行（コメント含む）
- **パフォーマンス影響**: なし（命令順序の変更のみ）
- **互換性**: 完全互換（external API 不変、internal API 不変）
- **リスク評価**: 極めて低い
### Trace システム
- **変更箇所**: 4 ファイル
- 新規: `core/box/ptr_trace_box.h`
- 修正: `tiny_superslab_alloc.inc.h`, `tls_sll_box.h`, `tls_sll_drain_box.h`
- **パフォーマンス影響**:
- デバッグビルド: トレース有効時のみ影響（ENV で制御）
- リリースビルド: ゼロオーバーヘッド（no-op マクロ）
- **互換性**: 完全互換（既存の動作に影響なし）
- **リスク評価**: なし（診断専用、本番には無影響）
---
## 期待効果
### 短期（Phase 1 修正後）
- **クラッシュ率**: 50% → 5% 以下
- **競合窓**: 50-100 cycles → 0 cycles
- **診断可能性**: ポインタライフサイクル完全追跡
### 中期（Phase 2 修正後）
- **クラッシュ率**: 5% → 0%
- **根本原因解消**: Header/Next 競合の完全排除
### 長期（アーキテクチャ改善）
- **保守性向上**: Header 位置の再設計により、将来の競合リスクを根絶
- **拡張性向上**: 新しいサイズクラス追加時の安全性保証
---
## 次のステップ
### 即座に実施（今日中）
1. ✅ Phase 1 修正の実装完了
2. ✅ Trace システムの実装完了
3. ⏳ コンパイル確認
4. ⏳ 基本動作確認
### 1週間以内
5. ⏳ Stress テスト（1000 回実行）
6. ⏳ Trace ログの分析
7. ⏳ Phase 2 修正の実装
8. ⏳ Phase 2 テスト（10000 回実行）
### 1ヶ月以内
9. ⏳ アーキテクチャ改善の詳細設計
10. ⏳ プロトタイプ実装とベンチマーク
---
## 補足情報
### 関連ドキュメント
- `docs/analysis/PTR_LIFECYCLE_TRACE_AND_ROOT_CAUSE_ANALYSIS.md`
- 詳細な根本原因分析
- 3段階の修正計画
- アーキテクチャ改善案
- `docs/analysis/TLS_SLL_ARCHITECTURE_INVESTIGATION.md`
- TLS SLL の既知の問題
- Phase 1 の Slab refill 時の TLS SLL drain 修正
### 技術的な学び
#### Header/Next ポインタ重複の危険性
- Class 1-6 では BASE[0] に Header と Next ポインタが共存
- 書き込みタイミングの違いにより、競合窓が発生
- Atomic な書き込み順序が critical
#### TLS SLL の設計原則
- Header 復元は必要最小限に（Pop 時のみ）
- Push 時の Header 復元は Next ポインタ破壊リスク
- Lazy Header Restore が安全
#### Freelist の integrity 保証
- Header 書き換えは Freelist 更新の **前**
- Freelist 更新後は Header が有効であることが前提
- 順序違反は corruption を招く
---
## 作成者
Claude (Anthropic)
## ステータス
- ✅ ポインタ追跡システム: 実装完了
- ✅ Phase 1 修正: 実装完了
- ⏳ Phase 2 修正: 設計完了、実装待ち
- ⏳ テスト: ビルド確認待ち
## 最終更新
2025-11-28

1349
docs/analysis/TLS_SLL_ARCHITECTURE_INVESTIGATION.md Normal file
View File

File diff suppressed because it is too large Load Diff