tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e51231471be93b210d6e6b94f1fb13a345a18ba5
hakmem/docs/archive
History
Moe Charm (CI) 984cca41ef P0 Optimization: Shared Pool fast path with O(1) metadata lookup 
Performance Results:
- Throughput: 2.66M ops/s → 3.8M ops/s (+43% improvement)
- sp_meta_find_or_create: O(N) linear scan → O(1) direct pointer
- Stage 2 metadata scan: 100% → 10-20% (80-90% reduction via hints)

Core Optimizations:

1. O(1) Metadata Lookup (superslab_types.h)
   - Added `shared_meta` pointer field to SuperSlab struct
   - Eliminates O(N) linear search through ss_metadata[] array
   - First access: O(N) scan + cache | Subsequent: O(1) direct return

2. sp_meta_find_or_create Fast Path (hakmem_shared_pool.c)
   - Check cached ss->shared_meta first before linear scan
   - Cache pointer after successful linear scan for future lookups
   - Reduces 7.8% CPU hotspot to near-zero for hot paths

3. Stage 2 Class Hints Fast Path (hakmem_shared_pool_acquire.c)
   - Try class_hints[class_idx] FIRST before full metadata scan
   - Uses O(1) ss->shared_meta lookup for hint validation
   - __builtin_expect() for branch prediction optimization
   - 80-90% of acquire calls now skip full metadata scan

4. Proper Initialization (ss_allocation_box.c)
   - Initialize shared_meta = NULL in superslab_allocate()
   - Ensures correct NULL-check semantics for new SuperSlabs

Additional Improvements:
- Updated ptr_trace and debug ring for release build efficiency
- Enhanced ENV variable documentation and analysis
- Added learner_env_box.h for configuration management
- Various Box optimizations for reduced overhead

Thread Safety:
- All atomic operations use correct memory ordering
- shared_meta cached under mutex protection
- Lock-free Stage 2 uses proper CAS with acquire/release semantics

Testing:
- Benchmark: 1M iterations, 3.8M ops/s stable
- Build: Clean compile RELEASE=0 and RELEASE=1
- No crashes, memory leaks, or correctness issues

Next Optimization Candidates:
- P1: Per-SuperSlab free slot bitmap for O(1) slot claiming
- P2: Reduce Stage 2 critical section size
- P3: Page pre-faulting (MAP_POPULATE)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 16:21:54 +09:00
..
ACE_AND_SUPERSLAB.md
WIP: Add TLS SLL validation and SuperSlab registry fallback
2025-12-03 20:42:28 +09:00
ACE_PHASE1_IMPLEMENTATION_TODO.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ACE_PHASE1_PROGRESS.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ACE_PHASE1_TEST_RESULTS.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ALIGNMENT_FIX_VERIFICATION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
ALLOCATION_MODEL_COMPARISON.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
ATOMIC_FREELIST_IMPLEMENTATION_STRATEGY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ATOMIC_FREELIST_INDEX.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ATOMIC_FREELIST_QUICK_START.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
ATOMIC_FREELIST_SITE_BY_SITE_GUIDE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
ATOMIC_FREELIST_SUMMARY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
BENCH_MODE.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
BENCHMARKING_CHECKLIST.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
BITMAP_VS_FREELIST_TRADEOFFS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
BOX3_REFACTORING.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
BRANCH_OPTIMIZATION_QUICK_START.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
BRANCH_PREDICTION_OPTIMIZATION_REPORT.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
BUG_FLOW_DIAGRAM.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
BUILD_README.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
CLAUDE_VERIFICATION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
claude.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
CLAUDE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
CODE_CLEANUP_2025_10_26.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
CURRENT_STATUS_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
CURRENT_TASK.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
DEBUG_100PCT_STABILITY.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
debug_analysis_final_$(date +%Y%m%d_%H%M%S).md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
DEBUG_LOGGING_POLICY.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
debug_logs_$(date +%Y%m%d_%H%M%S).md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
debug_logs_round2_$(date +%Y%m%d_%H%M%S).md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
dynamic_pool_ultrathink.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
ENV_VARS_2025-10-24.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
FEATURE_AUDIT_REMOVE_LIST.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
FINAL_RESULTS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
final_verdict.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
FOLDER_REORGANIZATION_2025_11_01.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
FREE_INC_SUMMARY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
FREE_TO_SS_TECHNICAL_DEEPDIVE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
HISTORY.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
L1D_CACHE_MISS_HOTSPOT_DIAGRAM.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
L1D_OPTIMIZATION_QUICK_START_GUIDE.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
LARGE_FILES_QUICK_REFERENCE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
LD_PRELOAD_BENCHMARK_BAN.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
LEARNING_SYSTEM_BUGS_P0.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
MAINLINE_INTEGRATION.md
Docs: Update ENV variable documentation after Ultra HEAP deletion
2025-11-27 04:51:59 +09:00
MALLOC_FALLBACK_REMOVAL_REPORT.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
MIGRATION_GUIDE.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
MIMALLOC_KEY_FINDINGS.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
NEXT_STEPS_2025_10_22.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
optimization_analysis.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
OPTIMIZATION_NEXT_STEPS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
OPTIMIZATION_REPORT_2025_11_12.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
OPTIMIZATION_SUMMARY_2025_10_22.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PAPER_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PERFORMANCE_INVESTIGATION_2025_10_22.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE4_IMPROVEMENT_ROADMAP.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
phase_7_7_battle_results.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
phase_8_1_results.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
phase_8_strategy_ultrathink.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6_SACS_OVERVIEW.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.3_MADVISE_BATCHING.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.6_ELO_CONTROL_FLOW_FIX.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.6_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.7_INDEX.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.7_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.8_CONFIG_CLEANUP.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.8_PROGRESS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.10_SITE_RULES.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.10.1_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11_DYNAMIC_THRESHOLDS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11.1_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11.2_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11.3_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.11.4_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.12_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.12_TINY_POOL.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.12.1_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.12.1_STEP2_RESTORATION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.13_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.13_INITIAL_RESULTS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.13_MIMALLOC_BENCH.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.14_COMPLETION_REPORT.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_P0.1_COMPLETION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_P0.2_INVESTIGATION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_P0.3_COMPLETION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_P1_BENCHMARK_RESULTS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_P1.5_COMPLETION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_QUICK_REF.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_STEP1_COMPLETION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_STEP2_COMPLETION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_6.15_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_7_6_PROGRESS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_7_6_STATUS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_7_8_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PHASE_REGISTRY_RESULTS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
POINTER_FIX_SUMMARY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
POOL_FULL_FIX_EVALUATION.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
POOL_HOT_PATH_BOTTLENECK.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
POOL_IMPLEMENTATION_CHECKLIST.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
POOL_TLS_PHASE1_5A_FIX.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
POOL_TLS_QUICKSTART.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
PROFILES.md
P0 Optimization: Shared Pool fast path with O(1) metadata lookup
2025-12-04 16:21:54 +09:00
PROFILING_GUIDE.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
PROFILING_RESULTS_2025_10_22.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
QUICK_REFERENCE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
RANDOM_MIXED_SUMMARY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
README_CLEAN.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
REFACTOR_QUICK_START.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
REMOVE_MALLOC_FALLBACK_TASK.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
RING_CACHE_ACTIVATION_GUIDE.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
SACS_3_OVERVIEW.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
SANITIZER_PHASE1_RESULTS.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
SESSION_SUMMARY_2025-10-26.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
SETUP_GUIDE.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
SOURCE_MAP.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
SPLIT_DETAILS.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
STABILITY_POLICY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
SUPERSLAB_BOX_REFACTORING_COMPLETE.md
ENV cleanup: Remove BG/HotMag vars & guard fprintf (Larson 52.3M ops/s)
2025-11-26 14:45:26 +09:00
SUPERSLAB_INVESTIGATION_RESULTS.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
SUPERSLAB_REFILL_BREAKDOWN.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
THREAD_SAFETY_SOLUTION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
TINY_DRAIN_INTERVAL_AB_REPORT.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
TINY_HEAP_V2_TASK_SPEC.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
TINY_LEARNING_LAYER.md
Docs: Update ENV variable documentation after Ultra HEAP deletion
2025-11-27 04:51:59 +09:00
TINY_POOL_OPTIMIZATION_ROADMAP.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
TINY_POOL_OPTIMIZATION_STRATEGY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
TLS_DECISION_TREE.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
ULTRATHINK_SUMMARY.md
Wrap debug fprintf in !HAKMEM_BUILD_RELEASE guards (Release build optimization)
2025-11-26 13:14:18 +09:00
VERIFICATION_EXECUTIVE_SUMMARY.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
VERIFICATION_SUMMARY_JP.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
WARMUP_ZERO_EFFECT_INVESTIGATION.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00
WHAT_IS_DYNAMIC.md
Debug Counters Implementation - Clean History
2025-11-05 12:31:14 +09:00