c2716f5c01
Implement Phase 2: Headerless Allocator Support (Partial)
...
- Feature: Added HAKMEM_TINY_HEADERLESS toggle (A/B testing)
- Feature: Implemented Headerless layout logic (Offset=0)
- Refactor: Centralized layout definitions in tiny_layout_box.h
- Refactor: Abstracted pointer arithmetic in free path via ptr_conversion_box.h
- Verification: sh8bench passes in Headerless mode (No TLS_SLL_HDR_RESET)
- Known Issue: Regression in Phase 1 mode due to blind pointer conversion logic
2025-12-03 12:11:27 +09:00
195c74756c
Fix mid free routing and relax mid W_MAX
2025-12-01 22:06:10 +09:00
04186341c1
Phase 4-Step2: Add Hot/Cold Path Box (+7.3% performance)
...
Implemented Hot/Cold Path separation using Box pattern for Tiny allocations:
Performance Improvement (without PGO):
- Baseline (Phase 26-A): 53.3 M ops/s
- Hot/Cold Box (Phase 4-Step2): 57.2 M ops/s
- Gain: +7.3% (+3.9 M ops/s)
Implementation:
1. core/box/tiny_front_hot_box.h - Ultra-fast hot path (1 branch)
- Removed range check (caller guarantees valid class_idx)
- Inline cache hit path with branch prediction hints
- Debug metrics with zero overhead in Release builds
2. core/box/tiny_front_cold_box.h - Slow cold path (noinline, cold)
- Refill logic (batch allocation from SuperSlab)
- Drain logic (batch free to SuperSlab)
- Error reporting and diagnostics
3. core/front/malloc_tiny_fast.h - Updated to use Hot/Cold Boxes
- Hot path: tiny_hot_alloc_fast() (1 branch: cache empty check)
- Cold path: tiny_cold_refill_and_alloc() (noinline, cold attribute)
- Clear separation improves i-cache locality
Branch Analysis:
- Baseline: 4-5 branches in hot path (range check + cache check + refill logic mixed)
- Hot/Cold Box: 1 branch in hot path (cache empty check only)
- Reduction: 3-4 branches eliminated from hot path
Design Principles (Box Pattern):
✅ Single Responsibility: Hot path = cache hit only, Cold path = refill/errors
✅ Clear Contract: Hot returns NULL on miss, Cold handles miss
✅ Observable: Debug metrics (TINY_HOT_METRICS_*) gated by NDEBUG
✅ Safe: Branch prediction hints (TINY_HOT_LIKELY/UNLIKELY)
✅ Testable: Isolated hot/cold paths, easy A/B testing
PGO Status:
- Temporarily disabled (build issues with __gcov_merge_time_profile)
- Will re-enable PGO in future commit after resolving gcc/lto issues
- Current benchmarks are without PGO (fair A/B comparison)
Other Changes:
- .gitignore: Added *.d files (dependency files, auto-generated)
- Makefile: PGO targets temporarily disabled (show informational message)
- build_pgo.sh: Temporarily disabled (show "PGO paused" message)
Next: Phase 4-Step3 (Front Config Box, target +5-8%)
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-29 11:58:37 +09:00
64ed3d8d8c
Add ss_fast_lookup() for O(1) SuperSlab lookup via mask
...
Replaces expensive hak_super_lookup() (registry hash lookup, 50-100 cycles)
with fast mask-based lookup (~5-10 cycles) in free hot paths.
Algorithm:
1. Mask pointer with SUPERSLAB_SIZE_MIN (1MB) - works for both 1MB and 2MB SS
2. Validate magic (SUPERSLAB_MAGIC)
3. Range check using ss->lg_size
Applied to:
- tiny_free_fast.inc.h: tiny_free_fast() SuperSlab path
- tiny_free_fast_v2.inc.h: LARSON_FIX cross-thread check
- front/malloc_tiny_fast.h: free_tiny_fast() LARSON_FIX path
Note: Performance impact minimal with LARSON_FIX=OFF (default) since
SuperSlab lookup is skipped entirely in that case. Optimization benefits
LARSON_FIX=ON path for safe multi-threaded operation.
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-27 12:47:10 +09:00
d8e3971dc2
Fix cross-thread ownership check: Use bits 8-15 for owner_tid_low
...
Problem:
- TLS_SLL_PUSH_DUP crash in Larson multi-threaded benchmark
- Cross-thread frees incorrectly routed to same-thread TLS path
- Root cause: pthread_t on glibc is 256-byte aligned (TCB base)
so lower 8 bits are ALWAYS 0x00 for ALL threads
Fix:
- Change owner_tid_low from (tid & 0xFF) to ((tid >> 8) & 0xFF)
- Bits 8-15 actually vary between threads, enabling correct detection
- Applied consistently across all ownership check locations:
- superslab_inline.h: ss_owner_try_acquire/release/is_mine
- slab_handle.h: slab_try_acquire
- tiny_free_fast.inc.h: tiny_free_is_same_thread_ss
- tiny_free_fast_v2.inc.h: cross-thread detection
- tiny_superslab_free.inc.h: same-thread check
- ss_allocation_box.c: slab initialization
- hakmem_tiny_superslab.c: ownership handling
Also added:
- Address watcher debug infrastructure (tiny_region_id.h)
- Cross-thread detection in malloc_tiny_fast.h Front Gate
Test results:
- Larson 1T/2T/4T: PASS (no TLS_SLL_PUSH_DUP crash)
- random_mixed: PASS
- Performance: ~20M ops/s (regression from 48M, needs optimization)
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-27 11:52:11 +09:00
5c9fe34b40
Enable performance optimizations by default (+557% improvement)
...
## Performance Impact
**Before** (optimizations OFF):
- Random Mixed 256B: 9.4M ops/s
- System malloc ratio: 10.6% (9.5x slower)
**After** (optimizations ON):
- Random Mixed 256B: 61.8M ops/s (+557%)
- System malloc ratio: 70.0% (1.43x slower) ✅
- 3-run average: 60.1M - 62.8M ops/s (±2.2% variance)
## Changes
Enabled 3 critical optimizations by default:
### 1. HAKMEM_SS_EMPTY_REUSE (hakmem_shared_pool.c:810)
```c
// BEFORE: default OFF
empty_reuse_enabled = (e && *e && *e != '0') ? 1 : 0;
// AFTER: default ON
empty_reuse_enabled = (e && *e && *e == '0') ? 0 : 1;
```
**Impact**: Reuse empty slabs before mmap, reduces syscall overhead
### 2. HAKMEM_TINY_UNIFIED_CACHE (tiny_unified_cache.h:69)
```c
// BEFORE: default OFF
g_enable = (e && *e && *e != '0') ? 1 : 0;
// AFTER: default ON
g_enable = (e && *e && *e == '0') ? 0 : 1;
```
**Impact**: Unified TLS cache improves hit rate
### 3. HAKMEM_FRONT_GATE_UNIFIED (malloc_tiny_fast.h:42)
```c
// BEFORE: default OFF
g_enable = (e && *e && *e != '0') ? 1 : 0;
// AFTER: default ON
g_enable = (e && *e && *e == '0') ? 0 : 1;
```
**Impact**: Unified front gate reduces dispatch overhead
## ENV Override
Users can still disable optimizations if needed:
```bash
export HAKMEM_SS_EMPTY_REUSE=0 # Disable empty slab reuse
export HAKMEM_TINY_UNIFIED_CACHE=0 # Disable unified cache
export HAKMEM_FRONT_GATE_UNIFIED=0 # Disable unified front gate
```
## Comparison to Competitors
```
mimalloc: 113.34M ops/s (1.83x faster than HAKMEM)
System malloc: 88.20M ops/s (1.43x faster than HAKMEM)
HAKMEM: 61.80M ops/s ✅ Competitive performance
```
## Files Modified
- core/hakmem_shared_pool.c - EMPTY_REUSE default ON
- core/front/tiny_unified_cache.h - UNIFIED_CACHE default ON
- core/front/malloc_tiny_fast.h - FRONT_GATE_UNIFIED default ON
🚀 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-22 01:29:05 +09:00
5b36c1c908
Phase 26: Front Gate Unification - Tiny allocator fast path (+12.9%)
...
Implementation:
- New single-layer malloc/free path for Tiny (≤1024B) allocations
- Bypasses 3-layer overhead: malloc → hak_alloc_at (236 lines) → wrapper → tiny_alloc_fast
- Leverages Phase 23 Unified Cache (tcache-style, 2-3 cache misses)
- Safe fallback to normal path on Unified Cache miss
Performance (Random Mixed 256B, 100K iterations):
- Baseline (Phase 26 OFF): 11.33M ops/s
- Phase 26 ON: 12.79M ops/s (+12.9%)
- Prediction (ChatGPT): +10-15% → Actual: +12.9% (perfect match!)
Bug fixes:
- Initialization bug: Added hak_init() call before fast path
- Page boundary SEGV: Added guard for offset_in_page == 0
Also includes Phase 23 debug log fixes:
- Guard C2_CARVE logs with #if !HAKMEM_BUILD_RELEASE
- Guard prewarm logs with #if !HAKMEM_BUILD_RELEASE
- Set Hot_2048 as default capacity (C2/C3=2048, others=64)
Files:
- core/front/malloc_tiny_fast.h: Phase 26 implementation (145 lines)
- core/box/hak_wrappers.inc.h: Fast path integration (+28 lines)
- core/front/tiny_unified_cache.h: Hot_2048 default
- core/tiny_refill_opt.h: C2_CARVE log guard
- core/box/ss_hot_prewarm_box.c: Prewarm log guard
- CURRENT_TASK.md: Phase 26 completion documentation
ENV variables:
- HAKMEM_FRONT_GATE_UNIFIED=1 (enable Phase 26, default: OFF)
- HAKMEM_TINY_UNIFIED_CACHE=1 (Phase 23, required)
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-11-17 05:29:08 +09:00
