diff --git a/CURRENT_TASK.md b/CURRENT_TASK.md index b013ff13..c9ef235f 100644 --- a/CURRENT_TASK.md +++ b/CURRENT_TASK.md @@ -80,22 +80,22 @@ C7 ULTRA は既に完成・凍結されており、v6 は C6 専用の研究ラ - L1/L2 で個別 block の stats は取らない - page retire 時に summary (total_allocs, avg_lifetime_ns) を L3 へ push -### 実装タスク（本フェーズ） +### 実装タスク（Phase V6-HDR-0: 完了） | No | タスク | 状態 | |----|--------|------| | 1-1 | CURRENT_TASK.md 整理（本セクション追加）| ✅ | -| 1-2 | SMALLOBJECT_CORE_V6_DESIGN.md 新規作成 | pending | -| 1-3 | REGIONID_V6_DESIGN.md 新規作成 | pending | -| 2-1 | SmallTlsLaneV6 / SmallHeapCtxV6 型スケルトン | pending | -| 2-2 | v6 TLS API (small_v6_tls_alloc/free) | pending | -| 3-1 | RegionIdBox 型と lookup API スケルトン | pending | -| 3-2 | OBSERVE モード（v6 free 入口にログ）| pending | -| 4-1 | PageStatsV6 箱（未接続）| pending | -| 5-1 | AGENTS.md に v6 研究箱ルール追記 | pending | -| 5-2 | サニティベンチ（Mixed / C6-heavy）| pending | +| 1-2 | SMALLOBJECT_CORE_V6_DESIGN.md 新規作成 | ✅ | +| 1-3 | REGIONID_V6_DESIGN.md 新規作成 | ✅ | +| 2-1 | SmallTlsLaneV6 / SmallHeapCtxV6 型スケルトン | ✅ | +| 2-2 | v6 TLS API (small_v6_tls_alloc/free) | ✅ | +| 3-1 | RegionIdBox 型と lookup API スケルトン | ✅ | +| 3-2 | OBSERVE モード（v6 free 入口にログ）| ✅ | +| 4-1 | PageStatsV6 箱（未接続）| ✅ | +| 5-1 | AGENTS.md に v6 研究箱ルール追記 | ✅ | +| 5-2 | サニティベンチ（Mixed / C6-heavy）| ✅ | -### ENV（予定） +### ENV - `HAKMEM_SMALL_CORE_V6_ENABLED=0` (default OFF) - `HAKMEM_REGION_ID_V6_OBSERVE=0` (default OFF, ログ出力用) @@ -103,6 +103,117 @@ C7 ULTRA は既に完成・凍結されており、v6 は C6 専用の研究ラ --- +## Phase V6-HDR-2: C6 headerless free 実験 ON（進行中） + +### 目的 + +V6-HDR-0/1 で作成した箱と RegionId の配線を使い、C6 だけ headerless free を実際に通電する。 +挙動は C6-heavy 専用プロファイルでだけ変える前提。 + +### 実装タスク + +| No | タスク | 状態 | +|----|--------|------| +| 1 | smallobject_v6_env_box.h 作成（ENV ゲート管理）| ✅ | +| 2 | front から v6 free ルート接続 | ✅ | +| 3 | small_v6_headerless_free 本実装 | ✅ | +| 4 | front から v6 alloc ルート接続 | ✅ | +| 5 | small_v6_headerless_alloc 本実装 | ✅ | +| 6 | header 書き込み 1回だけ化確認 | ✅ | +| 7 | ドキュメント更新 | ✅ | +| 8 | C6-heavy ベンチテスト | ✅ | + +### 実装詳細 + +1. **smallobject_v6_env_box.h** (新規) + - `small_heap_v6_headerless_enabled()`: ENV `HAKMEM_SMALL_HEAP_V6_HEADERLESS` + - `small_v6_region_observe_enabled()`: ENV `HAKMEM_SMALL_V6_REGION_OBSERVE` + - `small_v6_headerless_route_enabled(class_idx)`: 結合ゲート + +2. **smallobject_core_v6.c** (変更) + - `small_v6_headerless_free()`: RegionIdBox lookup → class_idx 取得 → TLS push + - `small_v6_headerless_alloc()`: TLS pop (header 書き込みなし) + refill + +3. **malloc_tiny_fast.h** (変更) + - `TINY_ROUTE_SMALL_HEAP_V6` case で headerless free/alloc を呼び出し + - ENV ゲート付き（デフォルト OFF） + +### Header 書き込みポリシー + +- **refill 時のみ**: carve/refill で page から TLS にブロックを移動する際に header 書き込み +- **alloc/free では書かない**: v6 headerless route では header に一切触らない +- **front は従来通り**: class_idx hint は header byte から取得（front 側の読み取りは維持） + +### ENV 変数 + +| ENV | Default | Description | +|-----|---------|-------------| +| `HAKMEM_SMALL_HEAP_V6_ENABLED` | 0 | v6 route 有効化 | +| `HAKMEM_SMALL_HEAP_V6_CLASSES` | 0x40 | v6 対象クラスマスク (0x40=C6) | +| `HAKMEM_SMALL_HEAP_V6_HEADERLESS` | 0 | headerless mode 有効化 | +| `HAKMEM_SMALL_V6_REGION_OBSERVE` | 0 | class_idx 検証ログ | + +### ベンチマーク結果 + +``` +# Baseline (v6 OFF) +C6-heavy 257-768B: 26.8M ops/s + +# v6 headerless ON +C6-heavy 257-768B: 26.7M ops/s + +# v6 headerless ON + OBSERVE +C6-heavy 257-768B: 26.1M ops/s (MISMATCH なし) +``` + +### 次のフェーズ (V6-HDR-3 提案) + +- v6 segment 実割り当て（現状は RegionIdBox lookup が常に UNKNOWN を返す） +- C6 route を v6 専用にして header 依存を完全除去 +- Mixed benchmark でも v6 headerless を安全確認 + +--- + +## Phase V6-HDR-1: RegionIdBox 実配線・OBSERVE（完了） + +### 目的 + +V6-HDR-0 で作成した RegionIdBox を C6 segment に実配線し、OBSERVE モードで +`ptr → (kind, page_meta.class_idx)` の正当性を検証する。**挙動変更なし**。 + +### 実装タスク + +| No | タスク | 状態 | +|----|--------|------| +| 1 | RegionIdBox 実装を埋める（C6 segment lookup）| ✅ | +| 2 | v6 free の REGION_OBSERVE ロジック具体化 | ✅ | +| 3 | front 側 class_idx ヒント配線確認 | ✅ | +| 4 | ドキュメント更新（CURRENT_TASK.md, REGIONID_V6_DESIGN.md）| ✅ | +| 5 | テスト（OBSERVE ON でベンチ実行）| ✅ | + +### 実装詳細 + +1. **RegionIdBox 実装** (`core/region_id_v6.c`) + - `region_id_lookup_v6(ptr)`: `small_page_meta_v6_of(ptr)` を使用して C6 segment 判定 + - TLS cache 付き高速版: `region_id_lookup_cached_v6(ptr)` + - OBSERVE ログ: `HAKMEM_REGION_ID_V6_OBSERVE=1` で有効 + +2. **REGION_OBSERVE ロジック** (`core/smallobject_core_v6.c`) + - ENV: `HAKMEM_SMALL_V6_REGION_OBSERVE=1`（新設、V6-HDR-1 専用） + - free 入口で `region_id_lookup_v6(ptr)` を呼び、class_idx 検証 + - 不一致時: `[V6_REGION_OBSERVE] MISMATCH ptr=... hint=X actual=Y` + +3. **front 配線確認結果** + - `class_idx` hint は `free_tiny_fast()` 内でヘッダバイト `(header & 0x0F)` から取得 + - v6 route (`TINY_ROUTE_SMALL_HEAP_V6`) は現在未接続（break でスキップ） + - これは「OBSERVE only, 挙動変更なし」の仕様通り + +### ENV + +- `HAKMEM_SMALL_V6_REGION_OBSERVE=1`: free path で class_idx 検証ログを出力 + +--- + --- ### 1. ベースライン（1 thread, ws=400, iters=1M, seed=1） diff --git a/Makefile b/Makefile index 14e0aec6..f1ab2a3b 100644 --- a/Makefile +++ b/Makefile @@ -218,7 +218,7 @@ LDFLAGS += $(EXTRA_LDFLAGS) # Targets TARGET = test_hakmem -OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o +OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o core/region_id_v6.o OBJS = $(OBJS_BASE) # Shared library @@ -250,7 +250,7 @@ endif # Benchmark targets BENCH_HAKMEM = bench_allocators_hakmem BENCH_SYSTEM = bench_allocators_system -BENCH_HAKMEM_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o bench_allocators_hakmem.o +BENCH_HAKMEM_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/free_publish_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o core/region_id_v6.o bench_allocators_hakmem.o BENCH_HAKMEM_OBJS = $(BENCH_HAKMEM_OBJS_BASE) ifeq ($(POOL_TLS_PHASE1),1) BENCH_HAKMEM_OBJS += pool_tls.o pool_refill.o pool_tls_arena.o pool_tls_registry.o pool_tls_remote.o @@ -427,7 +427,7 @@ test-box-refactor: box-refactor ./larson_hakmem 10 8 128 1024 1 12345 4 # Phase 4: Tiny Pool benchmarks (properly linked with hakmem) -TINY_BENCH_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/free_publish_box.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o +TINY_BENCH_OBJS_BASE = hakmem.o hakmem_config.o hakmem_tiny_config.o hakmem_ucb1.o hakmem_bigcache.o hakmem_pool.o hakmem_l25_pool.o hakmem_site_rules.o hakmem_tiny.o core/box/ss_allocation_box.o superslab_stats.o superslab_cache.o superslab_ace.o superslab_slab.o superslab_backend.o core/superslab_head_stub.o hakmem_smallmid.o core/box/superslab_expansion_box.o core/box/integrity_box.o core/box/mailbox_box.o core/box/front_gate_box.o core/box/front_gate_classifier.o core/box/free_publish_box.o core/box/capacity_box.o core/box/carve_push_box.o core/box/prewarm_box.o core/box/ss_hot_prewarm_box.o core/box/front_metrics_box.o core/box/bench_fast_box.o core/box/ss_addr_map_box.o core/box/slab_recycling_box.o core/box/pagefault_telemetry_box.o core/box/tiny_sizeclass_hist_box.o core/box/tiny_env_box.o core/box/tiny_route_box.o core/box/free_front_v3_env_box.o core/box/free_path_stats_box.o core/box/free_dispatch_stats_box.o core/box/alloc_gate_stats_box.o core/box/tiny_c6_ultra_free_box.o core/box/tiny_c5_ultra_free_box.o core/box/tiny_c4_ultra_free_box.o core/box/tiny_page_box.o core/box/tiny_class_policy_box.o core/box/tiny_class_stats_box.o core/box/tiny_policy_learner_box.o core/box/ss_budget_box.o core/box/tiny_mem_stats_box.o core/box/c7_meta_used_counter_box.o core/box/wrapper_env_box.o core/box/madvise_guard_box.o core/box/libm_reloc_guard_box.o core/box/ptr_trace_box.o core/box/link_missing_stubs.o core/box/super_reg_box.o core/box/shared_pool_box.o core/box/remote_side_box.o core/page_arena.o core/front/tiny_unified_cache.o tiny_sticky.o tiny_remote.o tiny_publish.o tiny_debug_ring.o hakmem_tiny_magazine.o hakmem_tiny_stats.o hakmem_tiny_sfc.o hakmem_tiny_query.o hakmem_tiny_rss.o hakmem_tiny_registry.o hakmem_tiny_remote_target.o hakmem_tiny_bg_spill.o tiny_adaptive_sizing.o hakmem_super_registry.o hakmem_shared_pool.o hakmem_shared_pool_acquire.o hakmem_shared_pool_release.o hakmem_elo.o hakmem_batch.o hakmem_p2.o hakmem_sizeclass_dist.o hakmem_evo.o hakmem_debug.o hakmem_sys.o hakmem_whale.o hakmem_policy.o hakmem_ace.o hakmem_ace_stats.o hakmem_prof.o hakmem_learner.o hakmem_size_hist.o hakmem_learn_log.o hakmem_syscall.o hakmem_ace_metrics.o hakmem_ace_ucb1.o hakmem_ace_controller.o tiny_fastcache.o core/tiny_alloc_fast_push.o core/tiny_c7_ultra_segment.o core/tiny_c7_ultra.o core/link_stubs.o core/tiny_failfast.o core/tiny_destructors.o core/smallobject_hotbox_v3.o core/smallobject_hotbox_v4.o core/smallobject_hotbox_v5.o core/smallsegment_v5.o core/smallobject_cold_iface_v5.o core/smallsegment_v6.o core/smallobject_cold_iface_v6.o core/smallobject_core_v6.o core/region_id_v6.o TINY_BENCH_OBJS = $(TINY_BENCH_OBJS_BASE) ifeq ($(POOL_TLS_PHASE1),1) TINY_BENCH_OBJS += pool_tls.o pool_refill.o core/pool_tls_arena.o pool_tls_registry.o pool_tls_remote.o diff --git a/core/box/region_id_v6_box.h b/core/box/region_id_v6_box.h index 15ce4d52..90f21ff6 100644 --- a/core/box/region_id_v6_box.h +++ b/core/box/region_id_v6_box.h @@ -10,6 +10,7 @@ #include #include #include +#include "smallsegment_v6_box.h" // For SmallSegmentV6 type // ============================================================================ // Region Kind Enum @@ -130,6 +131,15 @@ typedef struct RegionIdTlsCache { /// Get TLS cache for current thread RegionIdTlsCache* region_id_tls_cache_get(void); +// ============================================================================ +// Phase V6-HDR-3: Segment Registration (TLS scope) +// ============================================================================ + +/// Register this thread's v6 segment (Phase V6-HDR-3) +/// Called from small_heap_ctx_v6() when segment is first acquired +/// @param seg: SmallSegmentV6 pointer (TLS-owned) +void region_id_register_v6_segment(SmallSegmentV6* seg); + /// Inline fast lookup with TLS cache hit static inline RegionLookupV6 region_id_lookup_fast_v6(void* ptr) { RegionIdTlsCache* cache = region_id_tls_cache_get(); diff --git a/core/box/smallobject_core_v6_box.h b/core/box/smallobject_core_v6_box.h index 730eceb1..b4cb67a3 100644 --- a/core/box/smallobject_core_v6_box.h +++ b/core/box/smallobject_core_v6_box.h @@ -324,4 +324,22 @@ void* small_v6_alloc(SmallHeapCtxV6* ctx, uint8_t lane_idx); /// @param lane_idx: lane index (0=C4, 1=C5, 2=C6) void small_v6_free(SmallHeapCtxV6* ctx, void* ptr, uint8_t lane_idx); +// ============================================================================ +// Phase V6-HDR-2: Headerless Free/Alloc API +// ============================================================================ + +/// Headerless free: uses RegionIdBox for ptr classification +/// @param ctx: TLS context +/// @param ptr: USER pointer to free +/// @param class_idx_hint: class_idx from front (header byte) +/// @return: true if handled by v6, false if fallback needed +bool small_v6_headerless_free(SmallHeapCtxV6* ctx, void* ptr, uint8_t class_idx_hint); + +/// Headerless alloc: TLS pop without header write +/// Header is already written during carve/refill +/// @param ctx: TLS context +/// @param class_idx: class index (4=C4, 5=C5, 6=C6) +/// @return: USER pointer or NULL (fallback needed) +void* small_v6_headerless_alloc(SmallHeapCtxV6* ctx, uint8_t class_idx); + #endif // HAKMEM_SMALLOBJECT_CORE_V6_BOX_H diff --git a/core/box/smallobject_v6_env_box.h b/core/box/smallobject_v6_env_box.h new file mode 100644 index 00000000..18e6b8ee --- /dev/null +++ b/core/box/smallobject_v6_env_box.h @@ -0,0 +1,91 @@ +// smallobject_v6_env_box.h - SmallObject v6 ENV Control Box (Headerless Mode) +// +// Phase V6-HDR-2: Additional ENV control for headerless mode +// - Uses tiny_route_env_box.h for v6_enabled and class_enabled +// - Adds headerless-specific ENV gates + +#ifndef HAKMEM_SMALLOBJECT_V6_ENV_BOX_H +#define HAKMEM_SMALLOBJECT_V6_ENV_BOX_H + +#include +#include +#include +#include "tiny_route_env_box.h" // For small_heap_v6_enabled, small_heap_v6_class_enabled + +#ifndef likely +#define likely(x) __builtin_expect(!!(x), 1) +#define unlikely(x) __builtin_expect(!!(x), 0) +#endif + +// ============================================================================ +// ENV: HAKMEM_SMALL_HEAP_V6_HEADERLESS +// ============================================================================ +// Enable headerless mode for v6. Default OFF. +// When ON: +// - free: use RegionIdBox lookup instead of header +// - alloc: skip header write (rely on carve-time header) + +#define V6_HEADERLESS_UNINIT (-1) +#define V6_HEADERLESS_OFF 0 +#define V6_HEADERLESS_ON 1 + +static int g_small_heap_v6_headerless = V6_HEADERLESS_UNINIT; + +static inline bool small_heap_v6_headerless_enabled(void) { + if (unlikely(g_small_heap_v6_headerless == V6_HEADERLESS_UNINIT)) { + const char* env = getenv("HAKMEM_SMALL_HEAP_V6_HEADERLESS"); + g_small_heap_v6_headerless = (env && env[0] == '1') ? V6_HEADERLESS_ON : V6_HEADERLESS_OFF; + } + return g_small_heap_v6_headerless == V6_HEADERLESS_ON; +} + +// ============================================================================ +// ENV: HAKMEM_SMALL_V6_REGION_OBSERVE (moved from smallobject_core_v6.c) +// ============================================================================ +// Enable OBSERVE mode for class_idx validation. Default OFF. + +#define V6_REGION_OBS_UNINIT (-1) +#define V6_REGION_OBS_OFF 0 +#define V6_REGION_OBS_ON 1 + +static int g_small_v6_region_observe_env = V6_REGION_OBS_UNINIT; + +static inline bool small_v6_region_observe_enabled(void) { + if (unlikely(g_small_v6_region_observe_env == V6_REGION_OBS_UNINIT)) { + const char* env = getenv("HAKMEM_SMALL_V6_REGION_OBSERVE"); + g_small_v6_region_observe_env = (env && env[0] == '1') ? V6_REGION_OBS_ON : V6_REGION_OBS_OFF; + } + return g_small_v6_region_observe_env == V6_REGION_OBS_ON; +} + +// ============================================================================ +// Combined Gate: v6 headerless route enabled for class +// ============================================================================ + +/// Check if v6 headerless route is enabled for a given class +/// Uses small_heap_v6_enabled() and small_heap_v6_class_enabled() from tiny_route_env_box.h +/// @param class_idx: class index (4=C4, 5=C5, 6=C6) +/// @return: true if v6 headerless free/alloc should be used +static inline bool small_v6_headerless_route_enabled(uint8_t class_idx) { + return small_heap_v6_enabled() && + small_heap_v6_headerless_enabled() && + small_heap_v6_class_enabled((uint32_t)class_idx); +} + +// ============================================================================ +// Debug: Dump ENV state +// ============================================================================ + +static inline void small_v6_env_dump(void) { + // Force init + (void)small_heap_v6_enabled(); + (void)small_heap_v6_headerless_enabled(); + (void)small_v6_region_observe_enabled(); + + fprintf(stderr, "[V6_ENV] enabled=%d headerless=%d observe=%d\n", + small_heap_v6_enabled() ? 1 : 0, + g_small_heap_v6_headerless == V6_HEADERLESS_ON, + g_small_v6_region_observe_env == V6_REGION_OBS_ON); +} + +#endif // HAKMEM_SMALLOBJECT_V6_ENV_BOX_H diff --git a/core/front/malloc_tiny_fast.h b/core/front/malloc_tiny_fast.h index 6e137fcd..b97d7169 100644 --- a/core/front/malloc_tiny_fast.h +++ b/core/front/malloc_tiny_fast.h @@ -43,8 +43,8 @@ #include "../box/smallobject_hotbox_v3_box.h" // SmallObject HotHeap v3 skeleton #include "../box/smallobject_hotbox_v4_box.h" // SmallObject HotHeap v4 (C7 stub) #include "../box/smallobject_hotbox_v5_box.h" // SmallObject HotHeap v5 (C6-only route stub, Phase v5-1) -// Phase FREE-LEGACY-BREAKDOWN-1: v6 は型エラーがあるため一時的にコメントアウト（デフォルト OFF なので影響なし） -// #include "../box/smallobject_core_v6_box.h" // SmallObject Core v6 (C6-only route stub, Phase v6-1) +#include "../box/smallobject_core_v6_box.h" // SmallObject Core v6 (Phase V6-HDR-2) +#include "../box/smallobject_v6_env_box.h" // SmallObject v6 ENV control (Phase V6-HDR-2) #include "../box/tiny_c7_ultra_box.h" // C7 ULTRA stub (UF-1, delegates to v3) #include "../box/tiny_c6_ultra_free_box.h" // Phase 4-2: C6 ULTRA-free (free-only, C6-only) #include "../box/tiny_c5_ultra_free_box.h" // Phase 5-1/5-2: C5 ULTRA-free + alloc integration @@ -224,9 +224,15 @@ static inline void* malloc_tiny_fast(size_t size) { switch (route) { case TINY_ROUTE_SMALL_HEAP_V6: { - // Phase FREE-LEGACY-BREAKDOWN-1: v6 は既存のビルドエラーがあるため一時的にスキップ - // (v6 はデフォルト OFF なので測定には影響なし) - // fallthrough to v5/v2/v1 + // Phase V6-HDR-2: Headerless alloc (ENV gated) + if (small_v6_headerless_route_enabled((uint8_t)class_idx)) { + SmallHeapCtxV6* ctx_v6 = small_heap_ctx_v6(); + void* v6p = small_v6_headerless_alloc(ctx_v6, (uint8_t)class_idx); + if (TINY_HOT_LIKELY(v6p != NULL)) { + return v6p; // No header write needed - done in refill + } + // v6 returned NULL -> fallback to legacy + } __attribute__((fallthrough)); } case TINY_ROUTE_SMALL_HEAP_V3: { @@ -459,10 +465,15 @@ static inline int free_tiny_fast(void* ptr) { if (__builtin_expect(use_tiny_heap, 0)) { switch (route) { case TINY_ROUTE_SMALL_HEAP_V6: { - // Phase FREE-LEGACY-BREAKDOWN-1: v6 は既存のビルドエラーがあるため、今回は skip - // (v6 はデフォルト OFF なので測定には影響なし) - // fallthrough to v5/v2/v1 - break; + // Phase V6-HDR-2: Headerless free (ENV gated) + if (small_v6_headerless_route_enabled((uint8_t)class_idx)) { + SmallHeapCtxV6* ctx_v6 = small_heap_ctx_v6(); + if (small_v6_headerless_free(ctx_v6, ptr, (uint8_t)class_idx)) { + return 1; // Handled by v6 + } + // v6 returned false -> fallback to legacy + } + break; // fallthrough to legacy } case TINY_ROUTE_SMALL_HEAP_V5: { // Phase v5-2: C6-only full implementation diff --git a/core/region_id_v6.c b/core/region_id_v6.c new file mode 100644 index 00000000..cdc0ce6c --- /dev/null +++ b/core/region_id_v6.c @@ -0,0 +1,231 @@ +// region_id_v6.c - RegionIdBox implementation for V6-HDR-1 +// +// Purpose: Centralized ptr -> region lookup for headerless design +// Phase V6-HDR-1: OBSERVE only (no behavior change) + +#include +#include +#include +#include +#include "box/region_id_v6_box.h" +#include "box/smallsegment_v6_box.h" + +#ifndef likely +#define likely(x) __builtin_expect(!!(x), 1) +#define unlikely(x) __builtin_expect(!!(x), 0) +#endif + +// ============================================================================ +// ENV Control +// ============================================================================ + +#define REGION_OBSERVE_UNINIT (-1) +#define REGION_OBSERVE_OFF 0 +#define REGION_OBSERVE_ON 1 + +static int g_region_observe = REGION_OBSERVE_UNINIT; + +bool region_id_observe_enabled(void) { + if (unlikely(g_region_observe == REGION_OBSERVE_UNINIT)) { + const char* env = getenv("HAKMEM_REGION_ID_V6_OBSERVE"); + g_region_observe = (env && env[0] == '1') ? REGION_OBSERVE_ON : REGION_OBSERVE_OFF; + } + return g_region_observe == REGION_OBSERVE_ON; +} + +// ============================================================================ +// TLS Cache + Segment Registration (Phase V6-HDR-3) +// ============================================================================ + +static __thread RegionIdTlsCache g_region_tls_cache; +static __thread int g_region_tls_cache_init = 0; + +// TLS segment info (registered on first access) +static __thread uintptr_t g_v6_segment_base = 0; +static __thread uintptr_t g_v6_segment_end = 0; +static __thread SmallSegmentV6* g_v6_segment = NULL; +static __thread int g_v6_segment_registered = 0; + +RegionIdTlsCache* region_id_tls_cache_get(void) { + if (unlikely(!g_region_tls_cache_init)) { + memset(&g_region_tls_cache, 0, sizeof(g_region_tls_cache)); + g_region_tls_cache_init = 1; + } + return &g_region_tls_cache; +} + +/// Register this thread's v6 segment (Phase V6-HDR-3) +/// Called from small_heap_ctx_v6() when segment is first acquired +void region_id_register_v6_segment(SmallSegmentV6* seg) { + if (!seg || g_v6_segment_registered) { + return; // Already registered or invalid + } + + g_v6_segment_base = seg->base; + g_v6_segment_end = seg->base + SMALL_SEGMENT_V6_SIZE; + g_v6_segment = seg; + g_v6_segment_registered = 1; +} + +// ============================================================================ +// Global RegionIdBox (stub for now) +// ============================================================================ + +// Define the opaque RegionIdBox structure +struct RegionIdBox { + uint32_t next_id; + // Stub implementation - full implementation in V6-HDR-2 +}; + +static struct RegionIdBox g_region_id_box; + +RegionIdBox* region_id_box_get(void) { + return &g_region_id_box; +} + +// ============================================================================ +// Lookup Implementation (V6-HDR-1: TLS segment only) +// ============================================================================ + +// Forward declaration from smallsegment_v6.c +extern SmallPageMetaV6* small_page_meta_v6_of(void* ptr); + +RegionLookupV6 region_id_lookup_v6(void* ptr) { + RegionLookupV6 result = { + .kind = REGION_KIND_UNKNOWN, + .region_id = 0, + .page_meta = NULL + }; + + if (unlikely(!ptr)) { + return result; + } + + uintptr_t addr = (uintptr_t)ptr; + + // Phase V6-HDR-3: Check TLS segment registration first (fast path) + if (g_v6_segment_registered && addr >= g_v6_segment_base && addr < g_v6_segment_end) { + // ptr is in this thread's v6 segment - get page_meta + SmallPageMetaV6* page = small_page_meta_v6_of(ptr); + if (page != NULL) { + result.kind = REGION_KIND_SMALL_V6; + result.region_id = 1; // Single TLS segment per thread + result.page_meta = page; + + // Update TLS cache + RegionIdTlsCache* cache = region_id_tls_cache_get(); + cache->last_base = g_v6_segment_base; + cache->last_end = g_v6_segment_end; + cache->last_result = result; + } + return result; + } + + // Legacy fallback (if TLS not registered yet) + SmallPageMetaV6* page = small_page_meta_v6_of(ptr); + if (page != NULL) { + result.kind = REGION_KIND_SMALL_V6; + result.region_id = 1; + result.page_meta = page; + + RegionIdTlsCache* cache = region_id_tls_cache_get(); + SmallSegmentV6* seg = (SmallSegmentV6*)page->segment; + if (seg) { + cache->last_base = seg->base; + cache->last_end = seg->base + SMALL_SEGMENT_V6_SIZE; + cache->last_result = result; + } + } + + return result; +} + +RegionLookupV6 region_id_lookup_cached_v6(void* ptr) { + RegionIdTlsCache* cache = region_id_tls_cache_get(); + uintptr_t addr = (uintptr_t)ptr; + + // TLS cache hit? + if (addr >= cache->last_base && addr < cache->last_end && + cache->last_result.kind != REGION_KIND_UNKNOWN) { + // Cache hit - but still need to get the correct page_meta + // for this specific ptr (page_idx may differ) + SmallPageMetaV6* page = small_page_meta_v6_of(ptr); + if (page) { + RegionLookupV6 result = cache->last_result; + result.page_meta = page; + return result; + } + } + + // Cache miss -> slow path + return region_id_lookup_v6(ptr); +} + +// ============================================================================ +// Registration API (Stub for V6-HDR-1) +// ============================================================================ + +uint32_t region_id_register_v6(void* base, size_t size, region_kind_t kind, void* metadata) { + (void)base; + (void)size; + (void)kind; + (void)metadata; + // V6-HDR-1: Registration is not yet implemented + // TLS segment is implicitly "registered" by its existence + return 1; // Single region for now +} + +void region_id_unregister_v6(uint32_t region_id) { + (void)region_id; + // V6-HDR-1: No-op +} + +bool region_id_is_valid_v6(uint32_t region_id) { + return region_id == 1; // Only TLS segment is "registered" +} + +// ============================================================================ +// Utility API +// ============================================================================ + +const char* region_kind_to_string(region_kind_t kind) { + switch (kind) { + case REGION_KIND_UNKNOWN: return "UNKNOWN"; + case REGION_KIND_SMALL_V6: return "SMALL_V6"; + case REGION_KIND_C7_ULTRA: return "C7_ULTRA"; + case REGION_KIND_POOL_V1: return "POOL_V1"; + case REGION_KIND_LARGE: return "LARGE"; + case REGION_KIND_TINY_LEGACY: return "TINY_LEGACY"; + default: return "INVALID"; + } +} + +void region_id_box_dump(void) { + fprintf(stderr, "[REGION_ID_BOX] V6-HDR-1: TLS segment lookup only\n"); + fprintf(stderr, "[REGION_ID_BOX] observe=%d\n", region_id_observe_enabled()); +} + +// ============================================================================ +// OBSERVE Mode Logging +// ============================================================================ + +void region_id_observe_lookup(void* ptr, const RegionLookupV6* result) { + if (region_id_observe_enabled()) { + fprintf(stderr, "[REGION_ID] lookup ptr=%p kind=%s region_id=%u page_meta=%p\n", + ptr, region_kind_to_string(result->kind), + result->region_id, result->page_meta); + } +} + +void region_id_observe_register(void* base, size_t size, region_kind_t kind, uint32_t id) { + if (region_id_observe_enabled()) { + fprintf(stderr, "[REGION_ID] register base=%p size=%zu kind=%s id=%u\n", + base, size, region_kind_to_string(kind), id); + } +} + +void region_id_observe_unregister(uint32_t id) { + if (region_id_observe_enabled()) { + fprintf(stderr, "[REGION_ID] unregister id=%u\n", id); + } +} diff --git a/core/smallobject_core_v6.c b/core/smallobject_core_v6.c index cb7b0858..cebc315a 100644 --- a/core/smallobject_core_v6.c +++ b/core/smallobject_core_v6.c @@ -9,6 +9,8 @@ #include "box/smallobject_cold_iface_v6.h" #include "box/smallsegment_v6_box.h" #include "box/tiny_route_env_box.h" +#include "box/region_id_v6_box.h" +#include "box/smallobject_v6_env_box.h" #ifndef likely #define likely(x) __builtin_expect(!!(x), 1) @@ -40,6 +42,36 @@ static void small_v6_observe_free(void* ptr, uint32_t class_idx, int tls_owned) ptr, class_idx, tls_owned); } +// ============================================================================ +// REGION_OBSERVE Validation (V6-HDR-1) +// ============================================================================ +// Note: small_v6_region_observe_enabled() is now in smallobject_v6_env_box.h + +/// Validate class_idx via RegionIdBox lookup (called when REGION_OBSERVE=1) +/// @param ptr: USER pointer +/// @param class_idx_hint: class_idx from front caller +static void small_v6_region_observe_validate(void* ptr, uint32_t class_idx_hint) { + RegionLookupV6 lk = region_id_lookup_v6(ptr); + + // Log the lookup to REGION_ID_BOX observe (if enabled) + region_id_observe_lookup(ptr, &lk); + + if (lk.kind == REGION_KIND_SMALL_V6 && lk.page_meta != NULL) { + SmallPageMetaV6* page = (SmallPageMetaV6*)lk.page_meta; + if (page->class_idx != class_idx_hint) { + fprintf(stderr, "[V6_REGION_OBSERVE] MISMATCH ptr=%p " + "hint=%u actual=%u page_meta=%p\n", + ptr, class_idx_hint, page->class_idx, (void*)page); + } + } else if (lk.kind != REGION_KIND_UNKNOWN) { + // ptr is in a different kind of region (not v6) + fprintf(stderr, "[V6_REGION_OBSERVE] KIND_MISMATCH ptr=%p " + "kind=%s (expected SMALL_V6)\n", + ptr, region_kind_to_string(lk.kind)); + } + // REGION_KIND_UNKNOWN: ptr not in any v6 segment (OK for now) +} + // TLS context static __thread struct SmallHeapCtxV6 g_small_heap_ctx_v6; static __thread int g_small_heap_ctx_v6_init = 0; @@ -59,6 +91,9 @@ SmallHeapCtxV6* small_heap_ctx_v6(void) { if (seg && small_segment_v6_valid(seg)) { g_small_heap_ctx_v6.tls_seg_base = seg->base; g_small_heap_ctx_v6.tls_seg_end = seg->base + SMALL_SEGMENT_V6_SIZE; + + // Phase V6-HDR-3: Register segment with RegionIdBox (TLS scope) + region_id_register_v6_segment(seg); } g_small_heap_ctx_v6_init = 1; @@ -284,6 +319,11 @@ void small_free_fast_v6(void* ptr, small_v6_observe_free(ptr, class_idx, tls_owned); } + // V6-HDR-1: REGION_OBSERVE mode - validate class_idx via RegionIdBox + if (unlikely(small_v6_region_observe_enabled())) { + small_v6_region_observe_validate(ptr, class_idx); + } + // Fast path: TLS segment ownership + TLS push if (likely(tls_owned)) { // C6 TLS push @@ -442,3 +482,181 @@ void small_free_cold_v6(void* ptr, uint32_t class_idx) { small_cold_v6_retire_page(page); } } + +// ============================================================================ +// Phase V6-HDR-2: Headerless Free/Alloc Implementation +// ============================================================================ + +/// Headerless free: uses RegionIdBox for ptr classification +/// @param ctx: TLS context +/// @param ptr: USER pointer to free +/// @param class_idx_hint: class_idx from front (header byte) +/// @return: true if handled by v6, false if fallback needed +bool small_v6_headerless_free(SmallHeapCtxV6* ctx, void* ptr, uint8_t class_idx_hint) { + // Step 1: RegionIdBox lookup (no header read) + RegionLookupV6 lk = region_id_lookup_v6(ptr); + + if (lk.kind != REGION_KIND_SMALL_V6) { + // Not a v6 managed region -> front should fallback + return false; + } + + SmallPageMetaV6* page = (SmallPageMetaV6*)lk.page_meta; + if (!page) { + return false; + } + + uint8_t class_idx = page->class_idx; + + // Step 2: OBSERVE mode - validate class_idx hint + if (unlikely(small_v6_region_observe_enabled())) { + if (class_idx != class_idx_hint) { + fprintf(stderr, "[V6_HDR_FREE] MISMATCH ptr=%p hint=%u actual=%u\n", + ptr, class_idx_hint, class_idx); + } + } + + // Step 3: Convert USER -> BASE (no header touch) + void* base = SMALL_V6_BASE_FROM_USER(ptr); + + // Step 4: TLS ownership check + TLS push + if (small_tls_owns_ptr_v6(ctx, ptr)) { + // C6 TLS push + if (class_idx == SMALL_V6_C6_CLASS_IDX && ctx->tls_count_c6 < SMALL_V6_TLS_CAP) { + ctx->tls_freelist_c6[ctx->tls_count_c6++] = base; + return true; + } + // C5 TLS push + if (class_idx == SMALL_V6_C5_CLASS_IDX && ctx->tls_count_c5 < SMALL_V6_TLS_CAP) { + ctx->tls_freelist_c5[ctx->tls_count_c5++] = base; + return true; + } + // C4 TLS push + if (class_idx == SMALL_V6_C4_CLASS_IDX && ctx->tls_count_c4 < SMALL_V6_TLS_CAP) { + ctx->tls_freelist_c4[ctx->tls_count_c4++] = base; + return true; + } + } + + // Step 5: Cold path - push to page freelist + *(void**)base = page->free_list; + page->free_list = base; + if (page->used > 0) page->used--; + + // Retire empty page + if (page->used == 0) { + small_cold_v6_retire_page(page); + } + + return true; +} + +/// Headerless alloc: TLS pop without header write +/// Header is already written during carve/refill +/// @param ctx: TLS context +/// @param class_idx: class index (4=C4, 5=C5, 6=C6) +/// @return: USER pointer or NULL (fallback needed) +void* small_v6_headerless_alloc(SmallHeapCtxV6* ctx, uint8_t class_idx) { + // TLS fast path (no header write - already done in refill) + if (class_idx == SMALL_V6_C6_CLASS_IDX) { + if (likely(ctx->tls_count_c6 > 0)) { + void* blk = ctx->tls_freelist_c6[--ctx->tls_count_c6]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } else if (class_idx == SMALL_V6_C5_CLASS_IDX) { + if (likely(ctx->tls_count_c5 > 0)) { + void* blk = ctx->tls_freelist_c5[--ctx->tls_count_c5]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } else if (class_idx == SMALL_V6_C4_CLASS_IDX) { + if (likely(ctx->tls_count_c4 > 0)) { + void* blk = ctx->tls_freelist_c4[--ctx->tls_count_c4]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } + + // TLS empty -> need refill from cold path + // NOTE: Refill writes header, so alloc doesn't need to + SmallPageMetaV6* page = small_cold_v6_refill_page(class_idx); + if (!page || !page->free_list) { + return NULL; // Front should fallback to legacy + } + + uint8_t header_byte = SMALL_V6_HEADER_FROM_CLASS(class_idx); + int max_fill = SMALL_V6_TLS_CAP; + int filled = 0; + + // Refill TLS from page + if (class_idx == SMALL_V6_C6_CLASS_IDX) { + max_fill -= ctx->tls_count_c6; + while (page->free_list && filled < max_fill - 1) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + ((uint8_t*)blk)[0] = header_byte; // Header write on refill only + ctx->tls_freelist_c6[ctx->tls_count_c6++] = blk; + filled++; + } + page->used += filled; + + if (page->free_list) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + page->used++; + ((uint8_t*)blk)[0] = header_byte; + return SMALL_V6_USER_FROM_BASE(blk); + } + + if (ctx->tls_count_c6 > 0) { + void* blk = ctx->tls_freelist_c6[--ctx->tls_count_c6]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } else if (class_idx == SMALL_V6_C5_CLASS_IDX) { + max_fill -= ctx->tls_count_c5; + while (page->free_list && filled < max_fill - 1) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + ((uint8_t*)blk)[0] = header_byte; + ctx->tls_freelist_c5[ctx->tls_count_c5++] = blk; + filled++; + } + page->used += filled; + + if (page->free_list) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + page->used++; + ((uint8_t*)blk)[0] = header_byte; + return SMALL_V6_USER_FROM_BASE(blk); + } + + if (ctx->tls_count_c5 > 0) { + void* blk = ctx->tls_freelist_c5[--ctx->tls_count_c5]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } else if (class_idx == SMALL_V6_C4_CLASS_IDX) { + max_fill -= ctx->tls_count_c4; + while (page->free_list && filled < max_fill - 1) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + ((uint8_t*)blk)[0] = header_byte; + ctx->tls_freelist_c4[ctx->tls_count_c4++] = blk; + filled++; + } + page->used += filled; + + if (page->free_list) { + void* blk = page->free_list; + page->free_list = *(void**)blk; + page->used++; + ((uint8_t*)blk)[0] = header_byte; + return SMALL_V6_USER_FROM_BASE(blk); + } + + if (ctx->tls_count_c4 > 0) { + void* blk = ctx->tls_freelist_c4[--ctx->tls_count_c4]; + return SMALL_V6_USER_FROM_BASE(blk); + } + } + + return NULL; +} diff --git a/docs/analysis/REGIONID_V6_DESIGN.md b/docs/analysis/REGIONID_V6_DESIGN.md index 23890a6d..747a7f05 100644 Binary files a/docs/analysis/REGIONID_V6_DESIGN.md and b/docs/analysis/REGIONID_V6_DESIGN.md differ diff --git a/hakmem.d b/hakmem.d index a3fad914..f04c51b3 100644 --- a/hakmem.d +++ b/hakmem.d @@ -105,6 +105,12 @@ hakmem.o: core/hakmem.c core/hakmem.h core/hakmem_build_flags.h \ core/box/../front/../box/smallobject_hotbox_v3_env_box.h \ core/box/../front/../box/smallobject_hotbox_v4_box.h \ core/box/../front/../box/smallobject_hotbox_v5_box.h \ + core/box/../front/../box/smallobject_core_v6_box.h \ + core/box/../front/../box/smallobject_v6_env_box.h \ + core/box/../front/../box/tiny_route_env_box.h \ + core/box/../front/../box/free_dispatch_stats_box.h \ + core/box/../front/../box/smallobject_hotbox_v4_env_box.h \ + core/box/../front/../box/smallobject_v5_env_box.h \ core/box/../front/../box/tiny_c7_ultra_box.h \ core/box/../front/../box/tiny_c7_ultra_segment_box.h \ core/box/../front/../box/tiny_c6_ultra_free_box.h \ @@ -119,10 +125,6 @@ hakmem.o: core/hakmem.c core/hakmem.h core/hakmem_build_flags.h \ core/box/../front/../box/free_path_stats_box.h \ core/box/../front/../box/tiny_front_hot_box.h \ core/box/../front/../box/tiny_ptr_convert_box.h \ - core/box/../front/../box/tiny_route_env_box.h \ - core/box/../front/../box/free_dispatch_stats_box.h \ - core/box/../front/../box/smallobject_hotbox_v4_env_box.h \ - core/box/../front/../box/smallobject_v5_env_box.h \ core/box/../front/../box/tiny_front_stats_box.h \ core/box/../front/../box/free_path_stats_box.h \ core/box/../front/../box/alloc_gate_stats_box.h \ @@ -309,6 +311,12 @@ core/box/../front/../box/../superslab/superslab_inline.h: core/box/../front/../box/smallobject_hotbox_v3_env_box.h: core/box/../front/../box/smallobject_hotbox_v4_box.h: core/box/../front/../box/smallobject_hotbox_v5_box.h: +core/box/../front/../box/smallobject_core_v6_box.h: +core/box/../front/../box/smallobject_v6_env_box.h: +core/box/../front/../box/tiny_route_env_box.h: +core/box/../front/../box/free_dispatch_stats_box.h: +core/box/../front/../box/smallobject_hotbox_v4_env_box.h: +core/box/../front/../box/smallobject_v5_env_box.h: core/box/../front/../box/tiny_c7_ultra_box.h: core/box/../front/../box/tiny_c7_ultra_segment_box.h: core/box/../front/../box/tiny_c6_ultra_free_box.h: @@ -323,10 +331,6 @@ core/box/../front/../box/tiny_front_v3_env_box.h: core/box/../front/../box/free_path_stats_box.h: core/box/../front/../box/tiny_front_hot_box.h: core/box/../front/../box/tiny_ptr_convert_box.h: -core/box/../front/../box/tiny_route_env_box.h: -core/box/../front/../box/free_dispatch_stats_box.h: -core/box/../front/../box/smallobject_hotbox_v4_env_box.h: -core/box/../front/../box/smallobject_v5_env_box.h: core/box/../front/../box/tiny_front_stats_box.h: core/box/../front/../box/free_path_stats_box.h: core/box/../front/../box/alloc_gate_stats_box.h: