tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
603 Commits 2 Branches 0 Tags
71b1354d326c6d0b4013b3f65431f0a7aa492550
Commit Graph

11 Commits

Author SHA1 Message Date
Moe Charm (CI)
71b1354d32 Phase 10: FREE-TINY-FAST MONO LEGACY DIRECT (GO +1.89%) 
Results:
- A/B test: +1.89% on Mixed (10-run, clean env)
- Baseline: 51.96M ops/s
- Optimized: 52.94M ops/s
- Improvement: +984K ops/s (+1.89%)
- C6-heavy verification: +7.86% (nonlegacy_mask works correctly, no misfires)

Strategy:
- Extend Phase 9 (C0-C3 DUALHOT) to C4-C7 LEGACY DIRECT
- Fail-Fast principle: Never misclassify MID/ULTRA/V7 as LEGACY
- nonlegacy_mask: Cached at init, hot path uses single bit operation

Success factors:
1. Performance improvement: +1.89% (1.9x GO threshold)
2. Safety verified: nonlegacy_mask prevents MID v3 misfire in C6-heavy
3. Phase 9 coexistence: C0-C3 (Phase 9) + C4-C7 (Phase 10) = full LEGACY coverage
4. Minimal overhead: Single bit operation in hot path (mask & (1u<<class))

Implementation:
- Patch 1: ENV gate box (free_tiny_fast_mono_legacy_direct_env_box.h)
  - ENV: HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT=0/1 (default 0)
  - nonlegacy_mask cached (reuses free_policy_fast_v2_nonlegacy_mask())
  - Probe window: 64 (avoid bench_profile putenv race)
- Patch 2: Early-exit in free_tiny_fast() (malloc_tiny_fast.h)
  - Conditions: !nonlegacy_mask, route==LEGACY, !LARSON_FIX, done==1
  - Direct call: tiny_legacy_fallback_free_base()
- Patch 3: Visibility (free_path_stats_box.h)
  - mono_legacy_direct_hit counter (compile-out in release)
- Patch 4: cleanenv extension (run_mixed_10_cleanenv.sh)
  - ENV leak protection

Safety verification (C6-heavy):
- OFF: 19.75M ops/s
- ON: 21.30M ops/s (+7.86%)
- nonlegacy_mask correctly excludes C6 (MID v3 active)
- Improvement from C0-C5, C7 direct path acceleration

Files modified:
- core/bench_profile.h: add to MIXED_TINYV3_C7_SAFE preset
- core/front/malloc_tiny_fast.h: early-exit insertion
- core/box/free_path_stats_box.h: counter
- core/box/free_tiny_fast_mono_legacy_direct_env_box.h: NEW (ENV gate + nonlegacy_mask)
- scripts/run_mixed_10_cleanenv.sh: ENV leak protection

Health check: PASSED (all profiles)

Promotion: Added to MIXED_TINYV3_C7_SAFE preset (default ON, opt-out)

Rollback: HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT=0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-14 20:09:40 +09:00
Moe Charm (CI)
871034da1f Phase 9: FREE-TINY-FAST MONO DUALHOT (GO +2.72%) 
Results:
- A/B test: +2.72% on Mixed (10-run, clean env)
- Baseline: 48.89M ops/s
- Optimized: 50.22M ops/s
- Improvement: +1.33M ops/s (+2.72%)
- Stability: Standard deviation reduced by 60.8% (2.44M → 955K ops/s)

Strategy:
- Transplant C0-C3 "second hot" path to monolithic free_tiny_fast()
- Early-exit within monolithic (no hot/cold split)
- FastLane free now benefits from C0-C3 direct path

Success factors:
1. Performance improvement: +2.72% (2.7x GO threshold)
2. Stability improvement: 2.6x more stable (stdev 60.8% reduction)
3. Learned from Phase 7 failure:
   - Phase 7: Function split (hot/cold) → NO-GO
   - Phase 9: Early-exit within monolithic → GO
4. FastLane free compatibility: C0-C3 direct path now works with FastLane
5. Policy snapshot overhead reduction: C0-C3 (48% of Mixed) skip route lookup

Implementation:
- Patch 1: ENV gate box (free_tiny_fast_mono_dualhot_env_box.h)
  - ENV: HAKMEM_FREE_TINY_FAST_MONO_DUALHOT=0/1 (default 0)
  - Probe window: 64 (avoid bench_profile putenv race)
- Patch 2: Early-exit in free_tiny_fast() (malloc_tiny_fast.h)
  - Conditions: class_idx <= 3, !LARSON_FIX, route==LEGACY
  - Direct call: tiny_legacy_fallback_free_base()
- Patch 3: Visibility (free_path_stats_box.h)
  - mono_dualhot_hit counter (compile-out in release)
- Patch 4: cleanenv extension (run_mixed_10_cleanenv.sh)
  - ENV leak protection

Files modified:
- core/bench_profile.h: add to MIXED_TINYV3_C7_SAFE preset
- core/front/malloc_tiny_fast.h: early-exit insertion
- core/box/free_path_stats_box.h: counter
- core/box/free_tiny_fast_mono_dualhot_env_box.h: NEW (ENV gate)
- scripts/run_mixed_10_cleanenv.sh: ENV leak protection

Health check: PASSED (all profiles)

Promotion: Added to MIXED_TINYV3_C7_SAFE preset (default ON, opt-out)

Rollback: HAKMEM_FREE_TINY_FAST_MONO_DUALHOT=0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-14 19:16:49 +09:00
Moe Charm (CI)
d54893ea1d Phase 3 C3: Static Routing A/B Test ADOPT (+2.20% Mixed gain) 
Step 2 & 3 Complete:
- A/B test (Mixed 10-run): STATIC_ROUTE=0 (38.91M) → =1 (39.77M) = +2.20% avg
  - Median gain: +1.98%
  - Result: ✅ GO (exceeds +1.0% threshold)

- Decision: ✅ ADOPT into MIXED_TINYV3_C7_SAFE preset
  - bench_profile.h line 77: HAKMEM_TINY_STATIC_ROUTE=1 default
  - Learner auto-disables static route when HAKMEM_SMALL_LEARNER_V7_ENABLED=1

Implementation Summary:
- core/box/tiny_static_route_box.{h,c}: Research box (Step 1A)
- core/front/malloc_tiny_fast.h: Route lookup integration (Step 1B, lines 249-256)
- core/bench_profile.h: Bench sync + preset adoption

Cumulative Phase 2-3 Gains:
- B3 (Routing shape): +2.89%
- B4 (Wrapper split): +1.47%
- C3 (Static routing): +2.20%
- Total: ~6.8% (35.2M → ~39.8M ops/s)

Next: Phase 3 C1 (TLS Prefetch, expected +2-4%)

🤖 Generated with Claude Code

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2025-12-13 18:46:11 +09:00
Moe Charm (CI)
e95e61f0ff Phase POLICY-FAST-PATH-V2 complete + MID-V35-HOTPATH-OPT-1 design 
## Phase POLICY-FAST-PATH-V2 (FROZEN)
- Implementation complete: free_policy_fast_v2_box.h + malloc_tiny_fast.h integration
- A/B Results:
  - Mixed (ws=400): -1.6% regression ❌ (branch cost > skip benefit)
  - C6-heavy (ws=200): +5.4% improvement ✅
- Decision: Default OFF, FROZEN (ws<300 / C6-heavy research only)
- Learning: Large WS causes branch misprediction to dominate

## Phase 3-GRADUATE + ENV probe fix
- 64-probe retry for getenv() stability during bench_profile putenv()
- C6 ULTRA intrusive freelist: FROZEN (research box)

## Phase MID-V35-HOTPATH-OPT-1-DESIGN
- Design doc for next optimization target
- Target: MID v3.5 alloc/free hot path (C5-C6)
- Boxes: Stats Gate, TLS Layout, Boundary Check elimination
- Expected: +3-9% on Mixed mainline

Files:
- core/box/free_policy_fast_v2_box.h (new)
- core/box/free_path_stats_box.h/c (policy_fast_v2_skip counter)
- core/front/malloc_tiny_fast.h (fast-path integration)
- docs/analysis/MID_V35_HOTPATH_OPT_1_DESIGN.md (new)
- docs/analysis/PHASE_3_GRADUATE_*.md (new)
- CURRENT_TASK.md (phase status update)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-12-12 18:40:08 +09:00
Moe Charm (CI)
1a8652a91a Phase TLS-UNIFY-3: C6 intrusive freelist implementation (完成) 
Implement C6 ULTRA intrusive LIFO freelist with ENV gating:
- Single-linked LIFO using next pointer at USER+1 offset
- tiny_next_store/tiny_next_load for pointer access (single source of truth)
- Segment learning via ss_fast_lookup (per-class seg_base/seg_end)
- ENV gate: HAKMEM_TINY_C6_ULTRA_INTRUSIVE_FL (default OFF)
- Counters: c6_ifl_push/pop/fallback in FREE_PATH_STATS

Files:
- core/box/tiny_ultra_tls_box.h: Added c6_head field for intrusive LIFO
- core/box/tiny_ultra_tls_box.c: Pop/push with intrusive branching (case 6)
- core/box/tiny_c6_ultra_intrusive_env_box.h: ENV gate (new)
- core/box/tiny_c6_intrusive_freelist_box.h: L1 pure LIFO (new)
- core/tiny_debug_ring.h: C6_IFL events
- core/box/free_path_stats_box.h/c: c6_ifl_* counters

A/B Test Results (1M iterations, ws=200, 257-512B):
- ENV_OFF (array): 56.6 Mop/s avg
- ENV_ON (intrusive): 57.6 Mop/s avg (+1.8%, within noise)
- Counters verified: c6_ifl_push=265890, c6_ifl_pop=265815, fallback=0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2025-12-12 16:26:42 +09:00
Moe Charm (CI)
39a3c53dbc Phase v7-2: SmallObject v7 C6-only implementation with RegionIdBox integration 
- SmallSegment_v7: 2MiB segment with TLS slot and free page stack
- ColdIface_v7: Page refill/retire between HotBox and SegmentBox
- HotBox_v7: Full C6-only alloc/free with header writing (HEADER_MAGIC|class_idx)
- Free path early-exit: Check v7 route BEFORE ss_fast_lookup (separate mmap segment)
- RegionIdBox: Register v7 segment for ptr->region lookup
- Benchmark: v7 ON ~54.5M ops/s (-7% overhead vs 58.6M legacy baseline)

v7 correctly balances alloc/free counts and page lifecycle.
RegionIdBox overhead identified as primary cost driver.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2025-12-12 03:12:28 +09:00
Moe Charm (CI)
fb88725a43 Phase FREE-LEGACY-OPT-6: C4 ULTRA Implementation 
Implement C4 ULTRA free TLS cache with parasitic free+alloc pattern,
achieving 99.7-99.9% elimination of C4 legacy fallback calls.

Key Features:
- TLS cache cap=64 (tuned for L1 cache fit, smaller than C5/C6's 128)
- Segment learning via ss_fast_lookup() on first free
- Free-side cache push + alloc-side TLS pop pattern
- ENV gate: HAKMEM_TINY_C4_ULTRA_FREE_ENABLED (default OFF)
- Full FREE_PATH_STATS instrumentation

Benchmark Results:
C4-heavy (65-128B range):
  - C4 legacy: 591,583 → 1,711 (-99.7%)
  - c4_ultra cache hits: ~599k (free) + ~599k (alloc)
  - Mixed load: 340,732 → 284 C4 legacy (-99.9%)

Legacy fallback reduction:
  - C4-heavy: 589,872 fewer legacy calls (-10.9% total)
  - Mixed: 340,448 fewer C4 legacy calls (-12.8% in mixed)

Performance note: ~2% throughput cost in isolated C4-heavy case,
acceptable tradeoff for 99%+ legacy elimination per class.

Files:
  NEW: core/box/tiny_c4_ultra_free_box.h/c
  NEW: core/box/tiny_c4_ultra_free_env_box.h
  MOD: core/box/tiny_ultra_classes_box.h (added C4 macros)
  MOD: core/box/free_path_stats_box.h/c (C4 ULTRA counters)
  MOD: core/front/malloc_tiny_fast.h (C4 alloc+free integration)
  MOD: Makefile (added C4 ULTRA object)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2025-12-11 19:38:27 +09:00
Moe Charm (CI)
ea6ed1a6e4 Phase FREE-LEGACY-OPT-5-1/5-2: C5 ULTRA free+alloc integration 
Summary:
========
Implemented C5 ULTRA TLS cache pattern following the successful C6 ULTRA design:
- Phase 5-1: Free-side TLS cache + segment learning
- Phase 5-2: Alloc-side TLS pop for complete free+alloc cycle integration

Targets C5 class (129-256B) as next legacy reduction after C6 completion.

Key Changes:
============

1. NEW FILES:
   - core/box/tiny_c5_ultra_free_box.h: C5 ULTRA TLS cache structure
   - core/box/tiny_c5_ultra_free_box.c: C5 free path implementation (same pattern as C6)
   - core/box/tiny_c5_ultra_free_env_box.h: ENV gating (HAKMEM_TINY_C5_ULTRA_FREE_ENABLED)

2. MODIFIED FILES:
   - core/front/malloc_tiny_fast.h:
     * Added C5 ULTRA includes
     * Added C5 alloc-side TLS pop at lines 186-194 (integrated with C6)
     * Added C5 free path at lines 333-337 (integrated with C6)

   - core/box/tiny_ultra_classes_box.h:
     * Added TINY_CLASS_C5 constant
     * Added tiny_class_is_c5() macro
     * Extended tiny_class_is_ultra() to include C5

   - core/box/free_path_stats_box.h:
     * Added c5_ultra_free_fast counter
     * Added c5_ultra_alloc_hit counter

   - core/box/free_path_stats_box.c:
     * Updated stats dump to output C5 counters

   - Makefile:
     * Added core/box/tiny_c5_ultra_free_box.o to all object lists

3. Design Rationale:
   - Exact copy of C6 ULTRA pattern (proven effective)
   - TLS cache capacity: 128 blocks (same as C6 for consistency)
   - Segment learning on first C5 free via ss_fast_lookup()
   - Alloc-side pop integrated directly in malloc_tiny_fast.h hotpath
   - Legacy fallback unification via tiny_legacy_fallback_free_base()

4. Expected Impact:
   - C5 legacy calls: 68,871 → 0 (100% elimination)
   - Total legacy reduction: ~53% of remaining 129,623
   - Mixed workload: Minimal regression (C5 is smaller class, fewer allocations)

5. Stats Collection:
   Run with: HAKMEM_TINY_C5_ULTRA_FREE_ENABLED=1 HAKMEM_FREE_PATH_STATS=1 ./bench_allocators_hakmem

   Expected output:
   [FREE_PATH_STATS] ... c5_ultra_free=68871 c5_ultra_alloc=68871 ... legacy_fb=60752 ...
   [FREE_PATH_STATS_LEGACY_BY_CLASS] ... c5=0 ...

Status:
=======
- Code: ✅ COMPLETE (3 new files + 5 modified files)
- Compilation: ✅ Verified (no errors, only unused variable warnings unrelated to C5)
- Functionality: Ready to benchmark (ENV gating: default OFF, opt-in via ENV)

Phase Progression:
==================
✅ Phase 4-4: C6 ULTRA free+alloc (legacy C6: 137,319 → 0)
✅ Phase 5-1/5-2: C5 ULTRA free+alloc (legacy C5: 68,871 → 0 expected)
⏳ Phase 4.5: C4 ULTRA (34,727 remaining)
📋 Future: C3/C2 ULTRA if beneficial

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2025-12-11 19:26:51 +09:00
Moe Charm (CI)
9830eff6cc Phase FREE-LEGACY-OPT-4-4: C6 ULTRA free+alloc integration 
Parasitic TLS cache: alloc now pops from the TLS freelist filled by free.

Implementation:
- malloc_tiny_fast(): C6 class-specific TLS pop check before route switch
  - if (class_idx == 6 && tiny_c6_ultra_free_enabled())
  - pop from TinyC6UltraFreeTLS.freelist[--count]
  - return USER pointer (BASE + 1)

- FreePathStats: Added c6_ultra_alloc_hit counter for observability

Results (Mixed 16-1024B):
- OFF: 40.2M ops/s baseline
- ON:  42.2M ops/s (+4.9%) stable

Per-profile:
- Mixed:     +4.9% (40.2M → 42.2M)
- C6-heavy:  +7.6% (40.7M → 43.8M)

Free-alloc loop:
- free: TLS push (all C6 frees)
- alloc: TLS pop (all C6 allocs in steady state)
- Cache never fills, no legacy overflow
- C6 legacy_by_class reduced from 137K to 0 (100% elimination)

Key insight:
- Free-only TLS cache fails without alloc integration
- Once integrated, creates perfect load-balancing loop
- Alloc drains exactly what free fills

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-11 18:47:21 +09:00
Moe Charm (CI)
1b196b3ac0 Phase FREE-LEGACY-OPT-4-2/4-3: C6 ULTRA-free TLS cache + segment learning 
Phase 4-2:
- Add TinyC6UltraFreeTLS structure with 128-slot TLS freelist
- Implement tiny_c6_ultra_free_fast/slow for C6 free hot path
- Add c6_ultra_free_fast counter to FreePathStats
- ENV gate: HAKMEM_TINY_C6_ULTRA_FREE_ENABLED (default: OFF)

Phase 4-3:
- Add segment learning on first C6 free via ss_fast_lookup()
- Learn seg_base/seg_end from SuperSlab for range check
- Increase cache capacity from 32 to 128 blocks

Results:
- Segment learning works: fast path captures blocks in segment
- However, without alloc integration, cache fills up and overflows to legacy
- Net effect: +1-3% (within noise range)
- Drain strategy also tested: no benefit (equal overhead)

Conclusion:
- Free-only TLS cache is limited without alloc-side integration
- Core v6 already has alloc/free integrated TLS (but -12% slower)
- Keep as research box (ENV default OFF)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-11 18:34:27 +09:00
Moe Charm (CI)
210633117a Phase FREE-LEGACY-OPT-4-1: Legacy per-class breakdown analysis 
## 目的
Legacy fallback 49.2% の内訳を per-class で分析し、最も Legacy を使用しているクラスを特定。

## 実装内容

1. FreePathStats 構造体の拡張
   - legacy_by_class[8] フィールドを追加（C0-C7 の Legacy fallback 内訳）

2. デストラクタ出力の更新
   - [FREE_PATH_STATS_LEGACY_BY_CLASS] 行を追加し、C0-C7 の内訳を出力

3. カウンタの散布
   - free_tiny_fast() の Legacy fallback 経路で legacy_by_class[class_idx] をインクリメント
   - class_idx の範囲チェック（0-7）を実施

## 測定結果（Mixed 16-1024B）

**測定安定性**: 完全に安定（3 回とも同一の値、決定的測定）

Legacy per-class 内訳:
- C0: 0 (0.0%)
- C1: 0 (0.0%)
- C2: 8,746 (3.3% of legacy)
- C3: 17,279 (6.5% of legacy)
- C4: 34,727 (13.0% of legacy)
- C5: 68,871 (25.8% of legacy)
- C6: 137,319 (51.4% of legacy) ← 最大シェア
- C7: 0 (0.0%)

合計: 266,942 (49.2% of total free calls)

## 分析結果

**最大シェアクラス**: C6 (513-1024B) が Legacy の 51.4% を占める

**理由**:
- Mixed 16-1024B では C6 サイズのアロケーションが多い
- C7 ULTRA は C7 専用で C6 は未対応
- v3/v4 も C6 をカバーしていない
- Route 設定で C6 は Legacy に直接落ちている

## 次のアクション

Phase FREE-LEGACY-OPT-4-2 で C6 クラスに ULTRA-Free lane を実装:
- Legacy fallback を 51% 削減（C6 分）
- Legacy: 49.2% → 24-27% に改善（半減）
- Mixed 16-1024B: 44.8M → 47-48M ops/s 程度（+5-8% 改善）

## 変更ファイル

- core/box/free_path_stats_box.h: FreePathStats 構造体に legacy_by_class[8] 追加
- core/box/free_path_stats_box.c: デストラクタに per-class 出力追加
- core/front/malloc_tiny_fast.h: Legacy fallback 経路に per-class カウンタ追加
- docs/analysis/FREE_LEGACY_PATH_ANALYSIS.md: Phase 4-1 分析結果を記録

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-11 18:04:14 +09:00