hakmem/docs/analysis/PHASE6_10_CUMULATIVE_RESULTS.md

Phase 6-10: Cumulative Results & Strategic Analysis

Date: 2025-12-14 Status: ✅ COMPLETE - Major performance milestone achieved

---

Executive Summary

Phases 6-10 delivered +24.6% cumulative improvement on Mixed workload (16-1024B), representing the largest performance gain in hakmem's optimization history.

Cumulative Performance

Metric	Phase 5 Baseline	Phase 10 Final	Improvement
Throughput (Mixed)	43.04M ops/s	53.62M ops/s	+24.6% 🎉
Optimization Strategy	Per-phase incremental	Layer collapse + path consolidation	Structural

---

Phase-by-Phase Breakdown

Phase 6-1: Front FastLane (Layer Collapse) — ✅ +11.13%

Target: wrapper→gate→policy→route 層の固定費

Strategy: 単一エントリポイント (FastLane) への集約

• front_fastlane_try_malloc(size) - size→class→route→handler を 1 箇所で処理
• front_fastlane_try_free(ptr) - header validation + direct call

Result: +11.13% (hakmem 史上最大の単一改善)

Key Files:

• core/box/front_fastlane_env_box.h
• core/box/front_fastlane_box.h
• core/box/hak_wrappers.inc.h (wrapper integration)

Lesson: Consolidation at wrapper level is the winning pattern (vs micro-optimizations in hot path)

---

Phase 6-2: Front FastLane Free DeDup — ✅ +5.18%

Target: FastLane free 側の重複 header validation

Strategy: DeDup ON → free_tiny_fast() を直接呼び出し (duplicate validation を skip)

Result: +5.18% (σ: 1.00% → 0.58%, variance -42%)

Key Files:

• core/box/front_fastlane_env_box.h (DeDup gate 追加)
• core/box/front_fastlane_box.h (direct call path)

Lesson: Eliminating redundant checks provides both performance and stability gains

---

Phase 7: Front FastLane Free Hot/Cold Alignment — ❌ NO-GO (-2.16%)

Target: FastLane free を free_tiny_fast_hot() に寄せる

Result: -2.16% (FROZEN)

Root Cause:

• Hot/cold split が FastLane の軽量経路に overhead を追加
• TLS access + cold path branching
• I-cache locality 低下

Lesson: Same optimization has different effects in different contexts (wrapper vs FastLane)

---

Phase 8: FREE-STATIC-ROUTE ENV Cache Fix — ✅ +2.61%

Target: Phase 3 D1 最適化が効かない ENV cache 事故

Strategy: bench_profile putenv 後に refresh mechanism を追加

Result: +2.61% (σ: 867K → 336K, variance -61%)

Key Files:

• core/box/tiny_free_route_cache_env_box.{h,c} (refresh 追加)
• core/bench_profile.h (sync 追加)

Lesson: ENV gate synchronization is critical for ensuring optimizations actually work

---

Phase 9: FREE-TINY-FAST MONO DUALHOT — ✅ +2.72%

Target: Phase 7 の失敗を教訓に、monolithic 内 early-exit で C0-C3 direct

Strategy: free_tiny_fast() 内で C0-C3 を LEGACY に early-exit (function split なし)

Result: +2.72% (σ: 2.44M → 955K, variance -60.8%, 2.6x more stable)

Key Files:

• core/box/free_tiny_fast_mono_dualhot_env_box.h
• core/front/malloc_tiny_fast.h (early-exit 追加)

Lesson: Monolithic early-exit > function split (avoids call overhead, maintains I-cache locality)

---

Phase 10: FREE-TINY-FAST MONO LEGACY DIRECT — ✅ +1.89%

Target: Phase 9 (C0-C3) を C4-C7 に拡張

Strategy: nonlegacy_mask キャッシュで ULTRA/MID/V7 誤爆を防ぎつつ LEGACY direct

Result: +1.89%

Key Files:

• core/box/free_tiny_fast_mono_legacy_direct_env_box.h (nonlegacy_mask caching)
• core/front/malloc_tiny_fast.h (C4-C7 early-exit)

Lesson: Cached bitmask enables safe fast-path with minimal overhead

---

Phase 11: ENV Snapshot "maybe-fast" API — ❌ NO-GO (-8.35%)

Target: ENV snapshot 参照固定費 (~2-3%) を削減

Strategy: hakmem_env_snapshot_maybe_fast() API で enabled+snapshot+front_snap を 1 回に集約

Result: -8.35% (FROZEN, 設計ミス)

Root Cause:

• maybe_fast() を inline hot path 関数内で呼ぶことで、ctor_mode check が累積
• Compiler optimization 阻害 (unconditional call vs conditional branch)

Lesson: ENV gate optimization should target gate itself, not call sites

---

Technical Patterns (Across All Phases)

Winning Patterns ✅

1. Wrapper-level consolidation (Phase 6-1: +11.13%)

   • Collapse multiple layers into single entry point
   • Reduce total instruction count, improve I-cache locality

2. Deduplication (Phase 6-2: +5.18%)

   • Eliminate redundant checks at layer boundaries
   • Both performance and stability benefits

3. Monolithic early-exit (Phase 9: +2.72%, Phase 10: +1.89%)

   • Better than function split (avoids call overhead)
   • Maintains I-cache locality

4. Cached bitmask (Phase 10: nonlegacy_mask)

   • Single bit operation for complex condition
   • Computed once at init, hot path reads only

5. ENV gate synchronization (Phase 8: +2.61%)

   • Refresh mechanism ensures optimizations actually work
   • Critical for bench_profile integration

Anti-Patterns ❌

1. Function split for lightweight paths (Phase 7: -2.16%)

   • Hot/cold split adds overhead when path is already optimized
   • Context-dependent: works for wrapper, fails for FastLane

2. Call-site API changes (Phase 11: -8.35%)

   • Moving helper calls into inline hot path accumulates overhead
   • Compiler optimization inhibited

3. Branch hints without profiling (E3-4, E5-3c precedents)

   • Profile-dependent, often regresses

---

Perf Profile Analysis (Phase 10 後)

Current Hotspots (Mixed, 30M iterations)

Symbol	Self%	Classification	Next Action
front_fastlane_try_free	33.88%	集約点 (expected)	✅ Successfully consolidated
main	26.21%	Benchmark overhead	N/A (not optimizable)
malloc	23.26%	Alloc wrapper 集約点	🔍 Investigate further
tiny_header_finalize_alloc	5.33%	Header write	⚪ E5-2 tried, NEUTRAL
tiny_c7_ultra_alloc	3.75%	C7 ULTRA path	🔍 Possible target
unified_cache_push	1.61%	Cache push	⚪ Marginal ROI (~+1.0%)
hakmem_env_snapshot	0.82%	ENV snapshot	❌ Phase 11 failed
tiny_front_v3_snapshot_get	0.66%	Front snapshot	❌ Phase 11 failed

Key Observations

1. front_fastlane_try_free (33.88%) - This is EXPECTED and GOOD

   • Phase 6-1 successfully consolidated free path
   • High self% indicates consolidation worked (vs distributed overhead)
   • Not a bottleneck, just a measurement artifact

2. malloc (23.26%) - Alloc wrapper 集約点

   • Similar to front_fastlane_try_free (includes FastLane alloc)
   • May indicate alloc side has room for optimization
   • Need deeper analysis to separate wrapper vs actual hot path

3. tiny_header_finalize_alloc (5.33%) - Already optimized

   • Phase 5 E5-2 (Header Write-Once) was NEUTRAL (+0.45%)
   • Branch overhead ≈ savings
   • Further optimization unlikely to help

4. Remaining hotspots < 2% - Marginal ROI

   • unified_cache_push (1.61%): E5-3b predicted ~+1.0%
   • tiny_c7_ultra_alloc (3.75%): C7-specific, narrow scope

---

Strategic Options (Next Steps)

Option A: Continue Micro-Optimizations ⚪ (Marginal ROI)

Targets:

• unified_cache_push (1.61%) - predicted +1.0% ROI
• tiny_c7_ultra_alloc (3.75%) - C7-specific

Risk: Diminishing returns (each optimization < +1-2%)

Recommendation: Low priority unless low-hanging fruit identified

---

Option B: Alloc Side Deep Dive 🔍 (High Potential)

Observation: malloc (23.26%) suggests alloc side may have structural opportunities

Investigation Steps:

1. Perf profile with --call-graph dwarf to separate wrapper vs hot path
2. Identify if FastLane alloc has similar consolidation opportunities as FastLane free
3. Look for alloc-side deduplication opportunities

Expected ROI: +5-10% (if similar to Phase 6 free-side gains)

Recommendation: HIGH PRIORITY - Most promising next direction

---

Option C: Declare Victory 🎉 (Strategic Pause)

Achievement: +24.6% cumulative (Phase 6-10)

Context:

• Phase 5: ~+9-10% cumulative
• Phase 6-10: +24.6% cumulative
• Total (Phase 5-10): ~+30-35% cumulative

Rationale:

• Reached major performance milestone
• Remaining optimizations < +2% each (marginal)
• Risk/reward ratio decreasing

Recommendation: ⚪ NEUTRAL - Valid option depending on project goals

---

Cumulative Statistics

Performance Gains

Phase Range	Cumulative Gain	Individual Phases
Phase 5 (E4-E5)	~+9-10%	E4-1 (+3.51%), E4-2 (+21.83%), E5-1 (+3.35%)
Phase 6-10	+24.6%	6-1 (+11.13%), 6-2 (+5.18%), 8 (+2.61%), 9 (+2.72%), 10 (+1.89%)
Total (Phase 5-10)	~+30-35%	Estimated compound effect

Stability Improvements

Multiple phases delivered variance reduction:

• Phase 6-2: σ -42% (1.00% → 0.58%)
• Phase 8: σ -61% (867K → 336K)
• Phase 9: σ -60.8% (2.44M → 955K, 2.6x more stable)

Insight: Consolidation/deduplication patterns provide BOTH performance AND stability

---

Lessons Learned

1. Consolidation > Micro-Optimization

Evidence: Phase 6-1 (+11.13%) > all micro-optimizations combined

Pattern: Wrapper-level structural changes yield largest gains

2. Context Matters

Evidence: Hot/cold split works in wrapper (Phase 1), fails in FastLane (Phase 7 -2.16%)

Pattern: Same optimization technique has different effects in different contexts

3. Monolithic Early-Exit > Function Split

Evidence: Phase 9 (+2.72%) succeeded where Phase 7 (-2.16%) failed

Pattern: Avoid function call overhead, maintain I-cache locality

4. Inline Hot Path = Danger Zone

Evidence: Phase 11 (-8.35%) - API call in inline function accumulated overhead

Pattern: Even 2-3 instructions are expensive at high call frequency

5. ENV Gate Optimization Should Target Gate Itself

Evidence: Phase 11 (-8.35%) - call-site changes inhibited compiler optimization

Pattern: Optimize the gate (caching, probe window), not the call sites

6. Variance Reduction = Hidden Value

Evidence: Multiple phases delivered 40-60% variance reduction

Pattern: Consolidation/deduplication improves both mean AND stability

---

Recommendations

Immediate Next Steps (Priority Order)

1. HIGH PRIORITY: Alloc side deep dive (Option B)

   • Perf profile with call-graph to identify opportunities
   • Look for FastLane alloc equivalent optimizations
   • Expected ROI: +5-10%

2. MEDIUM PRIORITY: Strategic pause (Option C)

   • Document Phase 6-10 achievements
   • Re-evaluate project goals and priorities
   • Consider if +24.6% is "good enough" milestone

3. LOW PRIORITY: Micro-optimizations (Option A)

   • Only pursue if low-hanging fruit identified
   • Each remaining optimization < +2% ROI

Long-Term Strategic Questions

1. What is the target performance?

   • mimalloc competitive? (need benchmark comparison)
   • Absolute threshold? (e.g., 60M ops/s on Mixed)

2. What is the risk tolerance?

   • Each optimization has ~20-30% chance of NO-GO
   • Diminishing returns as we optimize further

3. What is the maintenance cost?

   • Each new optimization adds complexity
   • ENV gates increase test matrix

---

Conclusion

Phase 6-10 delivered +24.6% cumulative improvement, representing hakmem's most successful optimization sequence. The winning patterns (consolidation, deduplication, monolithic early-exit) are now well-established and can guide future work.

Status: ✅ MAJOR MILESTONE ACHIEVED

Next: Recommend alloc side deep dive as highest-ROI next direction, or strategic pause to consolidate gains.

---

Implementation: Claude Code Date: 2025-12-14 Phases: 6-10 (with Phase 11 NO-GO documented) Commits: ea221d057, f301ee4df, dcc1d42e7, be723ca05, 871034da1, 71b1354d3, ad73ca554, 16c7bce2d