Phase 9-2 Fix: SuperSlab registry exhaustion workaround

Problem:
- Legacy-allocated SuperSlabs had slot states stuck at SLOT_UNUSED
- sp_slot_mark_empty() failed, preventing EMPTY transition
- Slots never returned to freelist → registry exhaustion
- "SuperSlab registry full" errors flooded the system

Root Cause:
- Dual management: Legacy path vs Shared Pool path
- Legacy SuperSlabs not synced with Shared Pool metadata
- Inconsistent slot state tracking

Solution (Workaround):
- Added sp_meta_sync_slots_from_ss(): Syncs SP metadata from SuperSlab
- Modified shared_pool_release_slab(): Detects SLOT_ACTIVE mismatch
- On mismatch: Syncs from SuperSlab bitmap/class_map, then proceeds
- Allows EMPTY transition → freelist insertion → registry unregister

Implementation:
1. sp_meta_sync_slots_from_ss() (core/hakmem_shared_pool.c:418-452)
   - Rebuilds slot states from SuperSlab->slab_bitmap
   - Updates total_slots, active_slots, class_idx
   - Handles SLOT_ACTIVE, SLOT_EMPTY, SLOT_UNUSED states

2. shared_pool_release_slab() (core/hakmem_shared_pool.c:1336-1349)
   - Checks slot_state != SLOT_ACTIVE but slab_bitmap set
   - Calls sp_meta_sync_slots_from_ss() to rebuild state
   - Allows normal EMPTY flow to proceed

Results (verified by testing):
- "SuperSlab registry full" errors: ELIMINATED (0 occurrences)
- Throughput: 118-125 M ops/sec (stable)
- 3 consecutive stress tests: All passed
- Medium load test (15K iterations): Success

Nature of Fix:
- WORKAROUND (not root cause fix)
- Detects and repairs inconsistency at release time
- Root fix would require: Legacy path elimination + unified architecture
- This fix ensures stability while preserving existing code paths

Next Steps:
- Benchmark performance improvement vs Phase 9-1 baseline
- Plan root cause fix (Phase 10): Unify SuperSlab management
- Consider gradual Legacy path deprecation

Credit: ChatGPT for root cause analysis and implementation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-11-30 07:36:02 +09:00
parent 87b7d30998
commit adb5913af5

View File

@ -412,6 +412,45 @@ static int sp_slot_mark_empty(SharedSSMeta* meta, int slot_idx) {
return -1; // Not ACTIVE
}
// Sync SP-SLOT view from an existing SuperSlab.
// This is needed when a legacy-allocated SuperSlab reaches the shared-pool
// release path for the first time (slot states are still SLOT_UNUSED).
static void sp_meta_sync_slots_from_ss(SharedSSMeta* meta, SuperSlab* ss) {
if (!meta || !ss) return;
int cap = ss_slabs_capacity(ss);
if (cap > MAX_SLOTS_PER_SS) {
cap = MAX_SLOTS_PER_SS;
}
meta->total_slots = (uint8_t)cap;
meta->active_slots = 0;
for (int i = 0; i < cap; i++) {
SlotState state = SLOT_UNUSED;
uint32_t bit = (1u << i);
if (ss->slab_bitmap & bit) {
state = SLOT_ACTIVE;
meta->active_slots++;
} else {
TinySlabMeta* smeta = &ss->slabs[i];
uint16_t used = atomic_load_explicit(&smeta->used, memory_order_relaxed);
if (smeta->capacity > 0 && used == 0) {
state = SLOT_EMPTY;
}
}
uint8_t cls = ss->class_map[i];
if (cls == 255) {
cls = ss->slabs[i].class_idx;
}
meta->slots[i].class_idx = cls;
meta->slots[i].slab_idx = (uint8_t)i;
atomic_store_explicit(&meta->slots[i].state, state, memory_order_release);
}
}
// ---------- Layer 2: Metadata Management (Mid-level) ----------
// Ensure ss_metadata array has capacity for at least min_count entries
@ -1297,7 +1336,19 @@ shared_pool_release_slab(SuperSlab* ss, int slab_idx)
}
// Mark slot as EMPTY (ACTIVE → EMPTY)
if (sp_slot_mark_empty(sp_meta, slab_idx) != 0) {
uint32_t slab_bit = (1u << slab_idx);
SlotState slot_state = atomic_load_explicit(
&sp_meta->slots[slab_idx].state,
memory_order_acquire);
if (slot_state != SLOT_ACTIVE && (ss->slab_bitmap & slab_bit)) {
// Legacy path import: rebuild slot states from SuperSlab bitmap/class_map
sp_meta_sync_slots_from_ss(sp_meta, ss);
slot_state = atomic_load_explicit(
&sp_meta->slots[slab_idx].state,
memory_order_acquire);
}
if (slot_state != SLOT_ACTIVE || sp_slot_mark_empty(sp_meta, slab_idx) != 0) {
if (g_lock_stats_enabled == 1) {
atomic_fetch_add(&g_lock_release_count, 1);
}