tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c6a2a6d38abebad80c45219982fc1b64faed9574
hakmem/hakmem.d
Moe Charm (CI) ccf604778c Front-Direct implementation: SS→FC direct refill + SLL complete bypass 
## Summary

Implemented Front-Direct architecture with complete SLL bypass:
- Direct SuperSlab → FastCache refill (1-hop, bypasses SLL)
- SLL-free allocation/free paths when Front-Direct enabled
- Legacy path sealing (SLL inline opt-in, SFC cascade ENV-only)

## New Modules

- core/refill/ss_refill_fc.h (236 lines): Standard SS→FC refill entry point
  - Remote drain → Freelist → Carve priority
  - Header restoration for C1-C6 (NOT C0/C7)
  - ENV: HAKMEM_TINY_P0_DRAIN_THRESH, HAKMEM_TINY_P0_NO_DRAIN

- core/front/fast_cache.h: FastCache (L1) type definition
- core/front/quick_slot.h: QuickSlot (L0) type definition

## Allocation Path (core/tiny_alloc_fast.inc.h)

- Added s_front_direct_alloc TLS flag (lazy ENV check)
- SLL pop guarded by: g_tls_sll_enable && !s_front_direct_alloc
- Refill dispatch:
  - Front-Direct: ss_refill_fc_fill() → fastcache_pop() (1-hop)
  - Legacy: sll_refill_batch_from_ss() → SLL → FC (2-hop, A/B only)
- SLL inline pop sealed (requires HAKMEM_TINY_INLINE_SLL=1 opt-in)

## Free Path (core/hakmem_tiny_free.inc, core/hakmem_tiny_fastcache.inc.h)

- FC priority: Try fastcache_push() first (same-thread free)
- tiny_fast_push() bypass: Returns 0 when s_front_direct_free || !g_tls_sll_enable
- Fallback: Magazine/slow path (safe, bypasses SLL)

## Legacy Sealing

- SFC cascade: Default OFF (ENV-only via HAKMEM_TINY_SFC_CASCADE=1)
- Deleted: core/hakmem_tiny_free.inc.bak, core/pool_refill_legacy.c.bak
- Documentation: ss_refill_fc_fill() promoted as CANONICAL refill entry

## ENV Controls

- HAKMEM_TINY_FRONT_DIRECT=1: Enable Front-Direct (SS→FC direct)
- HAKMEM_TINY_P0_DIRECT_FC_ALL=1: Same as above (alt name)
- HAKMEM_TINY_REFILL_BATCH=1: Enable batch refill (also enables Front-Direct)
- HAKMEM_TINY_SFC_CASCADE=1: Enable SFC cascade (default OFF)
- HAKMEM_TINY_INLINE_SLL=1: Enable inline SLL pop (default OFF, requires AGGRESSIVE_INLINE)

## Benchmarks (Front-Direct Enabled)

```bash
ENV: HAKMEM_BENCH_FAST_FRONT=1 HAKMEM_TINY_FRONT_DIRECT=1
     HAKMEM_TINY_REFILL_BATCH=1 HAKMEM_TINY_P0_DIRECT_FC_ALL=1
     HAKMEM_TINY_REFILL_COUNT_HOT=256 HAKMEM_TINY_REFILL_COUNT_MID=96
     HAKMEM_TINY_BUMP_CHUNK=256

bench_random_mixed (16-1040B random, 200K iter):
  256 slots: 1.44M ops/s (STABLE, 0 SEGV)
  128 slots: 1.44M ops/s (STABLE, 0 SEGV)

bench_fixed_size (fixed size, 200K iter):
  256B: 4.06M ops/s (has debug logs, expected >10M without logs)
  128B: Similar (debug logs affect)
```

## Verification

- TRACE_RING test (10K iter): **0 SLL events** detected ✅
- Complete SLL bypass confirmed when Front-Direct=1
- Stable execution: 200K iterations × multiple sizes, 0 SEGV

## Next Steps

- Disable debug logs in hak_alloc_api.inc.h (call_num 14250-14280 range)
- Re-benchmark with clean Release build (target: 10-15M ops/s)
- 128/256B shortcut path optimization (FC hit rate improvement)

Co-Authored-By: ChatGPT <chatgpt@openai.com>
Suggested-By: ultrathink
2025-11-14 05:41:49 +09:00

102 lines
3.8 KiB
Makefile
Raw Blame History

hakmem.o: core/hakmem.c core/hakmem.h core/hakmem_build_flags.h \
core/hakmem_config.h core/hakmem_features.h core/hakmem_internal.h \
core/hakmem_sys.h core/hakmem_whale.h core/hakmem_bigcache.h \
core/hakmem_pool.h core/hakmem_l25_pool.h core/hakmem_policy.h \
core/hakmem_learner.h core/hakmem_size_hist.h core/hakmem_ace.h \
core/hakmem_site_rules.h core/hakmem_tiny.h core/hakmem_trace.h \
core/hakmem_tiny_mini_mag.h core/hakmem_tiny_superslab.h \
core/superslab/superslab_types.h core/hakmem_tiny_superslab_constants.h \
core/superslab/superslab_inline.h core/superslab/superslab_types.h \
core/tiny_debug_ring.h core/tiny_remote.h \
core/hakmem_tiny_superslab_constants.h core/tiny_fastcache.h \
core/box/tiny_next_ptr_box.h core/hakmem_tiny_config.h \
core/tiny_nextptr.h core/hakmem_mid_mt.h core/hakmem_super_registry.h \
core/hakmem_elo.h core/hakmem_ace_stats.h core/hakmem_batch.h \
core/hakmem_evo.h core/hakmem_debug.h core/hakmem_prof.h \
core/hakmem_syscall.h core/hakmem_ace_controller.h \
core/hakmem_ace_metrics.h core/hakmem_ace_ucb1.h core/ptr_trace.h \
core/box/hak_exit_debug.inc.h core/box/hak_kpi_util.inc.h \
core/box/hak_core_init.inc.h core/hakmem_phase7_config.h \
core/box/hak_alloc_api.inc.h core/box/hak_free_api.inc.h \
core/hakmem_tiny_superslab.h core/box/../tiny_free_fast_v2.inc.h \
core/box/../tiny_region_id.h core/box/../hakmem_build_flags.h \
core/box/../tiny_box_geometry.h \
core/box/../hakmem_tiny_superslab_constants.h \
core/box/../hakmem_tiny_config.h core/box/../ptr_track.h \
core/box/../box/tls_sll_box.h core/box/../box/../hakmem_tiny_config.h \
core/box/../box/../hakmem_build_flags.h core/box/../box/../tiny_remote.h \
core/box/../box/../tiny_region_id.h \
core/box/../box/../hakmem_tiny_integrity.h \
core/box/../box/../hakmem_tiny.h core/box/../box/../ptr_track.h \
core/box/../box/../tiny_debug_ring.h core/box/../hakmem_tiny_integrity.h \
core/box/front_gate_classifier.h core/box/hak_wrappers.inc.h
core/hakmem.h:
core/hakmem_build_flags.h:
core/hakmem_config.h:
core/hakmem_features.h:
core/hakmem_internal.h:
core/hakmem_sys.h:
core/hakmem_whale.h:
core/hakmem_bigcache.h:
core/hakmem_pool.h:
core/hakmem_l25_pool.h:
core/hakmem_policy.h:
core/hakmem_learner.h:
core/hakmem_size_hist.h:
core/hakmem_ace.h:
core/hakmem_site_rules.h:
core/hakmem_tiny.h:
core/hakmem_trace.h:
core/hakmem_tiny_mini_mag.h:
core/hakmem_tiny_superslab.h:
core/superslab/superslab_types.h:
core/hakmem_tiny_superslab_constants.h:
core/superslab/superslab_inline.h:
core/superslab/superslab_types.h:
core/tiny_debug_ring.h:
core/tiny_remote.h:
core/hakmem_tiny_superslab_constants.h:
core/tiny_fastcache.h:
core/box/tiny_next_ptr_box.h:
core/hakmem_tiny_config.h:
core/tiny_nextptr.h:
core/hakmem_mid_mt.h:
core/hakmem_super_registry.h:
core/hakmem_elo.h:
core/hakmem_ace_stats.h:
core/hakmem_batch.h:
core/hakmem_evo.h:
core/hakmem_debug.h:
core/hakmem_prof.h:
core/hakmem_syscall.h:
core/hakmem_ace_controller.h:
core/hakmem_ace_metrics.h:
core/hakmem_ace_ucb1.h:
core/ptr_trace.h:
core/box/hak_exit_debug.inc.h:
core/box/hak_kpi_util.inc.h:
core/box/hak_core_init.inc.h:
core/hakmem_phase7_config.h:
core/box/hak_alloc_api.inc.h:
core/box/hak_free_api.inc.h:
core/hakmem_tiny_superslab.h:
core/box/../tiny_free_fast_v2.inc.h:
core/box/../tiny_region_id.h:
core/box/../hakmem_build_flags.h:
core/box/../tiny_box_geometry.h:
core/box/../hakmem_tiny_superslab_constants.h:
core/box/../hakmem_tiny_config.h:
core/box/../ptr_track.h:
core/box/../box/tls_sll_box.h:
core/box/../box/../hakmem_tiny_config.h:
core/box/../box/../hakmem_build_flags.h:
core/box/../box/../tiny_remote.h:
core/box/../box/../tiny_region_id.h:
core/box/../box/../hakmem_tiny_integrity.h:
core/box/../box/../hakmem_tiny.h:
core/box/../box/../ptr_track.h:
core/box/../box/../tiny_debug_ring.h:
core/box/../hakmem_tiny_integrity.h:
core/box/front_gate_classifier.h:
core/box/hak_wrappers.inc.h:
Reference in New Issue View Git Blame Copy Permalink