tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
9472ee90c95f5b74fd2debd654e1a619c8d69d3e
hakmem/core/hakmem_tiny.d
Moe Charm (CI) ccf604778c Front-Direct implementation: SS→FC direct refill + SLL complete bypass 
## Summary

Implemented Front-Direct architecture with complete SLL bypass:
- Direct SuperSlab → FastCache refill (1-hop, bypasses SLL)
- SLL-free allocation/free paths when Front-Direct enabled
- Legacy path sealing (SLL inline opt-in, SFC cascade ENV-only)

## New Modules

- core/refill/ss_refill_fc.h (236 lines): Standard SS→FC refill entry point
  - Remote drain → Freelist → Carve priority
  - Header restoration for C1-C6 (NOT C0/C7)
  - ENV: HAKMEM_TINY_P0_DRAIN_THRESH, HAKMEM_TINY_P0_NO_DRAIN

- core/front/fast_cache.h: FastCache (L1) type definition
- core/front/quick_slot.h: QuickSlot (L0) type definition

## Allocation Path (core/tiny_alloc_fast.inc.h)

- Added s_front_direct_alloc TLS flag (lazy ENV check)
- SLL pop guarded by: g_tls_sll_enable && !s_front_direct_alloc
- Refill dispatch:
  - Front-Direct: ss_refill_fc_fill() → fastcache_pop() (1-hop)
  - Legacy: sll_refill_batch_from_ss() → SLL → FC (2-hop, A/B only)
- SLL inline pop sealed (requires HAKMEM_TINY_INLINE_SLL=1 opt-in)

## Free Path (core/hakmem_tiny_free.inc, core/hakmem_tiny_fastcache.inc.h)

- FC priority: Try fastcache_push() first (same-thread free)
- tiny_fast_push() bypass: Returns 0 when s_front_direct_free || !g_tls_sll_enable
- Fallback: Magazine/slow path (safe, bypasses SLL)

## Legacy Sealing

- SFC cascade: Default OFF (ENV-only via HAKMEM_TINY_SFC_CASCADE=1)
- Deleted: core/hakmem_tiny_free.inc.bak, core/pool_refill_legacy.c.bak
- Documentation: ss_refill_fc_fill() promoted as CANONICAL refill entry

## ENV Controls

- HAKMEM_TINY_FRONT_DIRECT=1: Enable Front-Direct (SS→FC direct)
- HAKMEM_TINY_P0_DIRECT_FC_ALL=1: Same as above (alt name)
- HAKMEM_TINY_REFILL_BATCH=1: Enable batch refill (also enables Front-Direct)
- HAKMEM_TINY_SFC_CASCADE=1: Enable SFC cascade (default OFF)
- HAKMEM_TINY_INLINE_SLL=1: Enable inline SLL pop (default OFF, requires AGGRESSIVE_INLINE)

## Benchmarks (Front-Direct Enabled)

```bash
ENV: HAKMEM_BENCH_FAST_FRONT=1 HAKMEM_TINY_FRONT_DIRECT=1
     HAKMEM_TINY_REFILL_BATCH=1 HAKMEM_TINY_P0_DIRECT_FC_ALL=1
     HAKMEM_TINY_REFILL_COUNT_HOT=256 HAKMEM_TINY_REFILL_COUNT_MID=96
     HAKMEM_TINY_BUMP_CHUNK=256

bench_random_mixed (16-1040B random, 200K iter):
  256 slots: 1.44M ops/s (STABLE, 0 SEGV)
  128 slots: 1.44M ops/s (STABLE, 0 SEGV)

bench_fixed_size (fixed size, 200K iter):
  256B: 4.06M ops/s (has debug logs, expected >10M without logs)
  128B: Similar (debug logs affect)
```

## Verification

- TRACE_RING test (10K iter): **0 SLL events** detected ✅
- Complete SLL bypass confirmed when Front-Direct=1
- Stable execution: 200K iterations × multiple sizes, 0 SEGV

## Next Steps

- Disable debug logs in hak_alloc_api.inc.h (call_num 14250-14280 range)
- Re-benchmark with clean Release build (target: 10-15M ops/s)
- 128/256B shortcut path optimization (FC hit rate improvement)

Co-Authored-By: ChatGPT <chatgpt@openai.com>
Suggested-By: ultrathink
2025-11-14 05:41:49 +09:00

169 lines
6.3 KiB
Makefile
Raw Blame History

core/hakmem_tiny.o: core/hakmem_tiny.c core/hakmem_tiny.h \
core/hakmem_build_flags.h core/hakmem_trace.h \
core/hakmem_tiny_mini_mag.h core/hakmem_tiny_config.h \
core/hakmem_phase7_config.h core/hakmem_tiny_superslab.h \
core/superslab/superslab_types.h core/hakmem_tiny_superslab_constants.h \
core/superslab/superslab_inline.h core/superslab/superslab_types.h \
core/tiny_debug_ring.h core/tiny_remote.h \
core/hakmem_tiny_superslab_constants.h core/hakmem_super_registry.h \
core/hakmem_internal.h core/hakmem.h core/hakmem_config.h \
core/hakmem_features.h core/hakmem_sys.h core/hakmem_whale.h \
core/hakmem_syscall.h core/hakmem_tiny_magazine.h \
core/hakmem_tiny_integrity.h core/box/tiny_next_ptr_box.h \
core/hakmem_tiny_config.h core/tiny_nextptr.h \
core/hakmem_tiny_batch_refill.h core/hakmem_tiny_stats.h core/tiny_api.h \
core/hakmem_tiny_stats_api.h core/hakmem_tiny_query_api.h \
core/hakmem_tiny_rss_api.h core/hakmem_tiny_registry_api.h \
core/tiny_tls.h core/tiny_debug.h core/tiny_mmap_gate.h \
core/tiny_refill.h core/slab_handle.h core/tiny_sticky.h \
core/tiny_ready.h core/box/mailbox_box.h core/hakmem_tiny_superslab.h \
core/tiny_remote_bg.h core/hakmem_tiny_remote_target.h \
core/tiny_ready_bg.h core/tiny_route.h core/box/adopt_gate_box.h \
core/tiny_tls_guard.h core/hakmem_tiny_tls_list.h \
core/hakmem_tiny_bg_spill.h core/tiny_adaptive_sizing.h \
core/tiny_system.h core/hakmem_prof.h core/front/quick_slot.h \
core/front/../hakmem_tiny.h core/front/fast_cache.h \
core/front/quick_slot.h core/front/../hakmem_tiny_fastcache.inc.h \
core/front/../hakmem_tiny.h core/front/../tiny_remote.h \
core/tiny_publish.h core/box/tls_sll_box.h \
core/box/../hakmem_tiny_config.h core/box/../hakmem_build_flags.h \
core/box/../tiny_remote.h core/box/../tiny_region_id.h \
core/box/../hakmem_build_flags.h core/box/../tiny_box_geometry.h \
core/box/../hakmem_tiny_superslab_constants.h \
core/box/../hakmem_tiny_config.h core/box/../ptr_track.h \
core/box/../hakmem_tiny_integrity.h core/box/../ptr_track.h \
core/box/../ptr_trace.h core/box/../tiny_debug_ring.h \
core/hakmem_tiny_hotmag.inc.h core/hakmem_tiny_hot_pop.inc.h \
core/hakmem_tiny_refill.inc.h core/tiny_box_geometry.h \
core/tiny_region_id.h core/refill/ss_refill_fc.h \
core/hakmem_tiny_ultra_front.inc.h core/hakmem_tiny_intel.inc \
core/hakmem_tiny_background.inc core/hakmem_tiny_bg_bin.inc.h \
core/hakmem_tiny_tls_ops.h core/hakmem_tiny_remote.inc \
core/hakmem_tiny_init.inc core/box/prewarm_box.h \
core/hakmem_tiny_bump.inc.h core/hakmem_tiny_smallmag.inc.h \
core/tiny_atomic.h core/tiny_alloc_fast.inc.h \
core/tiny_alloc_fast_sfc.inc.h core/hakmem_tiny_fastcache.inc.h \
core/tiny_alloc_fast_inline.h core/tiny_free_fast.inc.h \
core/hakmem_tiny_alloc.inc core/hakmem_tiny_slow.inc \
core/hakmem_tiny_free.inc core/box/free_publish_box.h core/mid_tcache.h \
core/tiny_free_magazine.inc.h core/tiny_superslab_alloc.inc.h \
core/box/superslab_expansion_box.h \
core/box/../superslab/superslab_types.h core/box/../tiny_tls.h \
core/tiny_superslab_free.inc.h core/box/free_remote_box.h \
core/box/free_local_box.h core/hakmem_tiny_lifecycle.inc \
core/hakmem_tiny_slab_mgmt.inc core/tiny_fc_api.h
core/hakmem_tiny.h:
core/hakmem_build_flags.h:
core/hakmem_trace.h:
core/hakmem_tiny_mini_mag.h:
core/hakmem_tiny_config.h:
core/hakmem_phase7_config.h:
core/hakmem_tiny_superslab.h:
core/superslab/superslab_types.h:
core/hakmem_tiny_superslab_constants.h:
core/superslab/superslab_inline.h:
core/superslab/superslab_types.h:
core/tiny_debug_ring.h:
core/tiny_remote.h:
core/hakmem_tiny_superslab_constants.h:
core/hakmem_super_registry.h:
core/hakmem_internal.h:
core/hakmem.h:
core/hakmem_config.h:
core/hakmem_features.h:
core/hakmem_sys.h:
core/hakmem_whale.h:
core/hakmem_syscall.h:
core/hakmem_tiny_magazine.h:
core/hakmem_tiny_integrity.h:
core/box/tiny_next_ptr_box.h:
core/hakmem_tiny_config.h:
core/tiny_nextptr.h:
core/hakmem_tiny_batch_refill.h:
core/hakmem_tiny_stats.h:
core/tiny_api.h:
core/hakmem_tiny_stats_api.h:
core/hakmem_tiny_query_api.h:
core/hakmem_tiny_rss_api.h:
core/hakmem_tiny_registry_api.h:
core/tiny_tls.h:
core/tiny_debug.h:
core/tiny_mmap_gate.h:
core/tiny_refill.h:
core/slab_handle.h:
core/tiny_sticky.h:
core/tiny_ready.h:
core/box/mailbox_box.h:
core/hakmem_tiny_superslab.h:
core/tiny_remote_bg.h:
core/hakmem_tiny_remote_target.h:
core/tiny_ready_bg.h:
core/tiny_route.h:
core/box/adopt_gate_box.h:
core/tiny_tls_guard.h:
core/hakmem_tiny_tls_list.h:
core/hakmem_tiny_bg_spill.h:
core/tiny_adaptive_sizing.h:
core/tiny_system.h:
core/hakmem_prof.h:
core/front/quick_slot.h:
core/front/../hakmem_tiny.h:
core/front/fast_cache.h:
core/front/quick_slot.h:
core/front/../hakmem_tiny_fastcache.inc.h:
core/front/../hakmem_tiny.h:
core/front/../tiny_remote.h:
core/tiny_publish.h:
core/box/tls_sll_box.h:
core/box/../hakmem_tiny_config.h:
core/box/../hakmem_build_flags.h:
core/box/../tiny_remote.h:
core/box/../tiny_region_id.h:
core/box/../hakmem_build_flags.h:
core/box/../tiny_box_geometry.h:
core/box/../hakmem_tiny_superslab_constants.h:
core/box/../hakmem_tiny_config.h:
core/box/../ptr_track.h:
core/box/../hakmem_tiny_integrity.h:
core/box/../ptr_track.h:
core/box/../ptr_trace.h:
core/box/../tiny_debug_ring.h:
core/hakmem_tiny_hotmag.inc.h:
core/hakmem_tiny_hot_pop.inc.h:
core/hakmem_tiny_refill.inc.h:
core/tiny_box_geometry.h:
core/tiny_region_id.h:
core/refill/ss_refill_fc.h:
core/hakmem_tiny_ultra_front.inc.h:
core/hakmem_tiny_intel.inc:
core/hakmem_tiny_background.inc:
core/hakmem_tiny_bg_bin.inc.h:
core/hakmem_tiny_tls_ops.h:
core/hakmem_tiny_remote.inc:
core/hakmem_tiny_init.inc:
core/box/prewarm_box.h:
core/hakmem_tiny_bump.inc.h:
core/hakmem_tiny_smallmag.inc.h:
core/tiny_atomic.h:
core/tiny_alloc_fast.inc.h:
core/tiny_alloc_fast_sfc.inc.h:
core/hakmem_tiny_fastcache.inc.h:
core/tiny_alloc_fast_inline.h:
core/tiny_free_fast.inc.h:
core/hakmem_tiny_alloc.inc:
core/hakmem_tiny_slow.inc:
core/hakmem_tiny_free.inc:
core/box/free_publish_box.h:
core/mid_tcache.h:
core/tiny_free_magazine.inc.h:
core/tiny_superslab_alloc.inc.h:
core/box/superslab_expansion_box.h:
core/box/../superslab/superslab_types.h:
core/box/../tiny_tls.h:
core/tiny_superslab_free.inc.h:
core/box/free_remote_box.h:
core/box/free_local_box.h:
core/hakmem_tiny_lifecycle.inc:
core/hakmem_tiny_slab_mgmt.inc:
core/tiny_fc_api.h:
Reference in New Issue View Git Blame Copy Permalink