Fix: Larson multi-threaded crash - 3 critical race conditions in SharedSuperSlabPool
Root Cause Analysis (via Task agent investigation):
Larson benchmark crashed with SEGV due to 3 separate race conditions between
lock-free Stage 2 readers and mutex-protected writers in shared_pool_acquire_slab().
Race Condition 1: Non-Atomic Counter
- **Problem**: `ss_meta_count` was `uint32_t` (non-atomic) but read atomically via cast
- **Impact**: Thread A reads partially-updated count, accesses uninitialized metadata[N]
- **Fix**: Changed to `_Atomic uint32_t`, use memory_order_release/acquire
Race Condition 2: Non-Atomic Pointer
- **Problem**: `meta->ss` was plain pointer, read lock-free but freed under mutex
- **Impact**: Thread A loads `meta->ss` after Thread B frees SuperSlab → use-after-free
- **Fix**: Changed to `_Atomic(SuperSlab*)`, set NULL before free, check for NULL
Race Condition 3: realloc() vs Lock-Free Readers (CRITICAL)
- **Problem**: `sp_meta_ensure_capacity()` used `realloc()` which MOVES the array
- **Impact**: Thread B reallocs `ss_metadata`, Thread A accesses OLD (freed) array
- **Fix**: **Removed realloc entirely** - use fixed-size array `ss_metadata[2048]`
Fixes Applied:
1. **core/hakmem_shared_pool.h** (Line 53, 125-126):
- `SuperSlab* ss` → `_Atomic(SuperSlab*) ss`
- `uint32_t ss_meta_count` → `_Atomic uint32_t ss_meta_count`
- `SharedSSMeta* ss_metadata` → `SharedSSMeta ss_metadata[MAX_SS_METADATA_ENTRIES]`
- Removed `ss_meta_capacity` (no longer needed)
2. **core/hakmem_shared_pool.c** (Lines 223-233, 248-287, 577, 631-635, 812-815, 872):
- **sp_meta_ensure_capacity()**: Replaced realloc with capacity check
- **sp_meta_find_or_create()**: atomic_load/store for count and ss pointer
- **Stage 1 (line 577)**: atomic_load for meta->ss
- **Stage 2 (line 631-635)**: atomic_load with NULL check + skip
- **shared_pool_release_slab()**: atomic_store(NULL) BEFORE superslab_free()
- All metadata searches: atomic_load for consistency
Memory Ordering:
- **Release** (line 285): `atomic_fetch_add(&ss_meta_count, 1, memory_order_release)`
→ Publishes all metadata[N] writes before count increment is visible
- **Acquire** (line 620, 631): `atomic_load(..., memory_order_acquire)`
→ Synchronizes-with release, ensures initialized metadata is seen
- **Release** (line 872): `atomic_store(&meta->ss, NULL, memory_order_release)`
→ Prevents Stage 2 from seeing dangling pointer
Test Results:
- **Before**: SEGV crash (1 thread, 2 threads, any iteration count)
- **After**: No crashes, stable execution
- 1 thread: 266K ops/sec (stable, no SEGV)
- 2 threads: 193K ops/sec (stable, no SEGV)
- Warning: `[SP_META_CAPACITY_ERROR] Exceeded MAX_SS_METADATA_ENTRIES=2048`
→ Non-fatal, indicates metadata recycling needed (future optimization)
Known Limitation:
- Fixed array size (2048) may be insufficient for extreme workloads
- Workaround: Increase MAX_SS_METADATA_ENTRIES if needed
- Proper solution: Implement metadata recycling when SuperSlabs are freed
Performance Note:
- Larson still slow (~200K ops/sec vs System 20M ops/sec, 100x slower)
- This is due to lock contention (separate issue, not race condition)
- Crash bug is FIXED, performance optimization is next step
Related Issues:
- Original report: Commit 93cc23450 claimed to fix 500K SEGV but crashes persisted
- This fix addresses the ROOT CAUSE, not just symptoms
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@ -50,7 +50,7 @@ typedef struct {
|
||||
// Per-SuperSlab metadata for slot management
|
||||
#define MAX_SLOTS_PER_SS 32 // Typical: 1MB SS has 32 slabs of 32KB each
|
||||
typedef struct SharedSSMeta {
|
||||
SuperSlab* ss; // Physical SuperSlab pointer
|
||||
_Atomic(SuperSlab*) ss; // Physical SuperSlab pointer (atomic for lock-free Stage 2)
|
||||
SharedSlot slots[MAX_SLOTS_PER_SS]; // Slot state for each slab
|
||||
uint8_t active_slots; // Number of SLOT_ACTIVE slots
|
||||
uint8_t total_slots; // Total available slots (from ss_slabs_capacity)
|
||||
@ -120,9 +120,10 @@ typedef struct SharedSuperSlabPool {
|
||||
FreeSlotList free_slots[TINY_NUM_CLASSES_SS];
|
||||
|
||||
// SharedSSMeta array for all SuperSlabs in pool
|
||||
SharedSSMeta* ss_metadata; // Dynamic array
|
||||
uint32_t ss_meta_capacity; // Allocated entries
|
||||
uint32_t ss_meta_count; // Used entries
|
||||
// RACE FIX: Fixed-size array (no realloc!) to avoid race with lock-free Stage 2
|
||||
#define MAX_SS_METADATA_ENTRIES 2048
|
||||
SharedSSMeta ss_metadata[MAX_SS_METADATA_ENTRIES]; // Fixed-size array
|
||||
_Atomic uint32_t ss_meta_count; // Used entries (atomic for lock-free Stage 2)
|
||||
} SharedSuperSlabPool;
|
||||
|
||||
// Global singleton
|
||||
|
||||
Reference in New Issue
Block a user