ccf604778c
## Summary
Implemented Front-Direct architecture with complete SLL bypass:
- Direct SuperSlab → FastCache refill (1-hop, bypasses SLL)
- SLL-free allocation/free paths when Front-Direct enabled
- Legacy path sealing (SLL inline opt-in, SFC cascade ENV-only)
## New Modules
- core/refill/ss_refill_fc.h (236 lines): Standard SS→FC refill entry point
- Remote drain → Freelist → Carve priority
- Header restoration for C1-C6 (NOT C0/C7)
- ENV: HAKMEM_TINY_P0_DRAIN_THRESH, HAKMEM_TINY_P0_NO_DRAIN
- core/front/fast_cache.h: FastCache (L1) type definition
- core/front/quick_slot.h: QuickSlot (L0) type definition
## Allocation Path (core/tiny_alloc_fast.inc.h)
- Added s_front_direct_alloc TLS flag (lazy ENV check)
- SLL pop guarded by: g_tls_sll_enable && !s_front_direct_alloc
- Refill dispatch:
- Front-Direct: ss_refill_fc_fill() → fastcache_pop() (1-hop)
- Legacy: sll_refill_batch_from_ss() → SLL → FC (2-hop, A/B only)
- SLL inline pop sealed (requires HAKMEM_TINY_INLINE_SLL=1 opt-in)
## Free Path (core/hakmem_tiny_free.inc, core/hakmem_tiny_fastcache.inc.h)
- FC priority: Try fastcache_push() first (same-thread free)
- tiny_fast_push() bypass: Returns 0 when s_front_direct_free || !g_tls_sll_enable
- Fallback: Magazine/slow path (safe, bypasses SLL)
## Legacy Sealing
- SFC cascade: Default OFF (ENV-only via HAKMEM_TINY_SFC_CASCADE=1)
- Deleted: core/hakmem_tiny_free.inc.bak, core/pool_refill_legacy.c.bak
- Documentation: ss_refill_fc_fill() promoted as CANONICAL refill entry
## ENV Controls
- HAKMEM_TINY_FRONT_DIRECT=1: Enable Front-Direct (SS→FC direct)
- HAKMEM_TINY_P0_DIRECT_FC_ALL=1: Same as above (alt name)
- HAKMEM_TINY_REFILL_BATCH=1: Enable batch refill (also enables Front-Direct)
- HAKMEM_TINY_SFC_CASCADE=1: Enable SFC cascade (default OFF)
- HAKMEM_TINY_INLINE_SLL=1: Enable inline SLL pop (default OFF, requires AGGRESSIVE_INLINE)
## Benchmarks (Front-Direct Enabled)
```bash
ENV: HAKMEM_BENCH_FAST_FRONT=1 HAKMEM_TINY_FRONT_DIRECT=1
HAKMEM_TINY_REFILL_BATCH=1 HAKMEM_TINY_P0_DIRECT_FC_ALL=1
HAKMEM_TINY_REFILL_COUNT_HOT=256 HAKMEM_TINY_REFILL_COUNT_MID=96
HAKMEM_TINY_BUMP_CHUNK=256
bench_random_mixed (16-1040B random, 200K iter):
256 slots: 1.44M ops/s (STABLE, 0 SEGV)
128 slots: 1.44M ops/s (STABLE, 0 SEGV)
bench_fixed_size (fixed size, 200K iter):
256B: 4.06M ops/s (has debug logs, expected >10M without logs)
128B: Similar (debug logs affect)
```
## Verification
- TRACE_RING test (10K iter): **0 SLL events** detected ✅
- Complete SLL bypass confirmed when Front-Direct=1
- Stable execution: 200K iterations × multiple sizes, 0 SEGV
## Next Steps
- Disable debug logs in hak_alloc_api.inc.h (call_num 14250-14280 range)
- Re-benchmark with clean Release build (target: 10-15M ops/s)
- 128/256B shortcut path optimization (FC hit rate improvement)
Co-Authored-By: ChatGPT <chatgpt@openai.com>
Suggested-By: ultrathink
49 lines
1.6 KiB
C
49 lines
1.6 KiB
C
#ifndef TINY_DEBUG_RING_H
|
|
#define TINY_DEBUG_RING_H
|
|
|
|
#include <stdint.h>
|
|
#include <stddef.h>
|
|
#include "hakmem_build_flags.h"
|
|
|
|
// Tiny Debug Ring Trace (Phase 8 tooling)
|
|
// Environment: HAKMEM_TINY_TRACE_RING=1 to enable
|
|
// Records recent alloc/free events and dumps them on SIGSEGV.
|
|
|
|
enum {
|
|
TINY_RING_EVENT_ALLOC_ENTER = 1,
|
|
TINY_RING_EVENT_ALLOC_SUCCESS,
|
|
TINY_RING_EVENT_ALLOC_NULL,
|
|
TINY_RING_EVENT_ALLOC_REFILL_START,
|
|
TINY_RING_EVENT_ALLOC_REFILL_NULL,
|
|
TINY_RING_EVENT_ALLOC_BIND,
|
|
TINY_RING_EVENT_FREE_ENTER,
|
|
TINY_RING_EVENT_FREE_FAST,
|
|
TINY_RING_EVENT_FREE_REMOTE,
|
|
TINY_RING_EVENT_FREE_LOCAL,
|
|
TINY_RING_EVENT_FREE_RETURN_MAG,
|
|
TINY_RING_EVENT_SUPERSLAB_ADOPT,
|
|
TINY_RING_EVENT_SUPERSLAB_ALLOC,
|
|
TINY_RING_EVENT_SUPERSLAB_PUBLISH,
|
|
TINY_RING_EVENT_SUPERSLAB_ADOPT_FAIL,
|
|
TINY_RING_EVENT_REMOTE_PUSH,
|
|
TINY_RING_EVENT_REMOTE_INVALID,
|
|
TINY_RING_EVENT_REMOTE_DRAIN,
|
|
TINY_RING_EVENT_OWNER_ACQUIRE,
|
|
TINY_RING_EVENT_OWNER_RELEASE,
|
|
TINY_RING_EVENT_FRONT_BYPASS,
|
|
TINY_RING_EVENT_MAILBOX_PUBLISH,
|
|
TINY_RING_EVENT_MAILBOX_FETCH,
|
|
TINY_RING_EVENT_MAILBOX_FETCH_NULL,
|
|
TINY_RING_EVENT_ROUTE,
|
|
// TLS SLL anomalies (investigation aid, gated by HAKMEM_TINY_SLL_RING)
|
|
TINY_RING_EVENT_TLS_SLL_REJECT = 0x7F10,
|
|
TINY_RING_EVENT_TLS_SLL_SENTINEL = 0x7F11,
|
|
TINY_RING_EVENT_TLS_SLL_HDR_CORRUPT = 0x7F12
|
|
};
|
|
|
|
// Function declarations (implementation in tiny_debug_ring.c)
|
|
void tiny_debug_ring_init(void);
|
|
void tiny_debug_ring_record(uint16_t event, uint16_t class_idx, void* ptr, uintptr_t aux);
|
|
|
|
#endif // TINY_DEBUG_RING_H
|