tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
51 Commits 2 Branches 0 Tags
ef2d1caa2a2c5f2271a90b43a813c1d70afa3387
Commit Graph

8 Commits

Author SHA1 Message Date
Moe Charm (CI)
ef2d1caa2a Phase 7-1.3: Simplify HAK_RET_ALLOC macro definition (-35% LOC, -100% #undef) 
Problem:
- Phase 7-1.3 working code had complex #ifndef/#undef pattern
- Bidirectional dependency between hakmem_tiny.c and tiny_alloc_fast.inc.h
- Dangerous #undef usage masking real errors
- 3 levels of #ifdef nesting, hard to understand control flow

Solution:
- Single definition point in core/hakmem_tiny.c (lines 116-152)
- Clear feature flag based selection: #if HAKMEM_TINY_HEADER_CLASSIDX
- Removed duplicate definition and #undef from tiny_alloc_fast.inc.h
- Added clear comment pointing to single definition point

Results:
- -35% lines of code (7 lines deleted)
- -100% #undef usage (eliminated dangerous pattern)
- -33% nesting depth (3 levels → 2 levels)
- Much clearer control flow (single decision point)
- Same performance: 2.63M ops/s Larson, 17.7M ops/s bench_random_mixed

Implementation:
1. core/hakmem_tiny.c: Replaced #ifndef/#undef with #if HAKMEM_TINY_HEADER_CLASSIDX
2. core/tiny_alloc_fast.inc.h: Deleted duplicate macro, added pointer comment

Testing:
- Larson 1T: 2.63M ops/s (expected ~2.73M, within variance)
- bench_random_mixed (128B): 17.7M ops/s (better than before!)
- All builds clean with HEADER_CLASSIDX=1

Recommendation from Task Agent Ultrathink (Option A - Single Definition):
https://github.com/anthropics/claude-code/issues/...

Phase: 7-1.3 (Ifdef Simplification)
Date: 2025-11-08
 2025-11-08 11:49:21 +09:00
Moe Charm (CI)
4983352812 Perf: Phase 7-1.3 - Hybrid mincore + Macro fix (+194-333%) 
## Summary
Fixed CRITICAL bottleneck (mincore overhead) and macro definition bug.
Result: 2-3x performance improvement across all benchmarks.

## Performance Results
- Larson 1T: 631K → 2.73M ops/s (+333%) 🚀
- bench_random_mixed (128B): 768K → 2.26M ops/s (+194%) 🚀
- bench_random_mixed (512B): → 1.43M ops/s (new)
- [HEADER_INVALID] messages: Many → ~Zero ✅

## Changes

### 1. Hybrid mincore Optimization (317-634x faster)
**Problem**: `hak_is_memory_readable()` calls mincore() syscall on EVERY free
- Cost: 634 cycles/call
- Impact: 40x slower than System malloc

**Solution**: Check alignment BEFORE calling mincore()
- Step 1 (1-byte header): `if ((ptr & 0xFFF) == 0)` → only 0.1% call mincore
- Step 2 (16-byte header): `if ((ptr & 0xFFF) < HEADER_SIZE)` → only 0.4% call mincore
- Result: 634 → 1-2 cycles effective (99.6% skip mincore)

**Files**:
- core/tiny_free_fast_v2.inc.h:53-71 - Step 1 hybrid check
- core/box/hak_free_api.inc.h:94-107 - Step 2 hybrid check
- core/hakmem_internal.h:281-312 - Performance warning added

### 2. HAK_RET_ALLOC Macro Fix (CRITICAL BUG)
**Problem**: Macro definition order prevented Phase 7 header write
- hakmem_tiny.c:130 defined legacy macro (no header write)
- tiny_alloc_fast.inc.h:67 had `#ifndef` guard → skipped!
- Result: Headers NEVER written → All frees failed → Slow path

**Solution**: Force Phase 7 macro to override legacy
- hakmem_tiny.c:119 - Added `#ifndef HAK_RET_ALLOC` guard
- tiny_alloc_fast.inc.h:69-72 - Added `#undef` before redefine

### 3. Magic Byte Fix
**Problem**: Release builds don't write magic byte, but free ALWAYS checks it
- Result: All headers marked as invalid

**Solution**: ALWAYS write magic byte (same 1-byte write, no overhead)
- tiny_region_id.h:50-54 - Removed `#if !HAKMEM_BUILD_RELEASE` guard

## Technical Details

### Hybrid mincore Effectiveness
| Case | Frequency | Cost | Weighted |
|------|-----------|------|----------|
| Normal (Step 1) | 99.9% | 1-2 cycles | 1-2 |
| Page boundary | 0.1% | 634 cycles | 0.6 |
| **Total** | - | - | **1.6-2.6 cycles** |

**Improvement**: 634 → 1.6 cycles = **317-396x faster!**

### Macro Fix Impact
**Before**: HAK_RET_ALLOC(cls, ptr) → return (ptr)  // No header write
**After**: HAK_RET_ALLOC(cls, ptr) → return tiny_region_id_write_header((ptr), (cls))

**Result**: Headers properly written → Fast path works → +194-333% performance

## Investigation
Task Agent Ultrathink analysis identified:
1. mincore() syscall overhead (634 cycles)
2. Macro definition order conflict
3. Release/Debug build mismatch (magic byte)

Full report: PHASE7_DESIGN_REVIEW.md (23KB, 758 lines)

## Related
- Phase 7-1.0: PoC implementation (+39%~+436%)
- Phase 7-1.1: Dual-header dispatch (Task Agent)
- Phase 7-1.2: Page boundary SEGV fix (100% crash-free)
- Phase 7-1.3: Hybrid mincore + Macro fix (this commit)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-08 04:50:41 +09:00
Moe Charm (CI)
6b1382959c Phase 7-1 PoC: Region-ID Direct Lookup (+39%~+436% improvement!) 
Implemented ultra-fast header-based free path that eliminates SuperSlab
lookup bottleneck (100+ cycles → 5-10 cycles).

## Key Changes

1. **Smart Headers** (core/tiny_region_id.h):
   - 1-byte header before each allocation stores class_idx
   - Memory layout: [Header: 1B] [User data: N-1B]
   - Overhead: <2% average (0% for Slab[0] using wasted padding)

2. **Ultra-Fast Allocation** (core/tiny_alloc_fast.inc.h):
   - Write header at base: *base = class_idx
   - Return user pointer: base + 1

3. **Ultra-Fast Free** (core/tiny_free_fast_v2.inc.h):
   - Read class_idx from header (ptr-1): 2-3 cycles
   - Push base (ptr-1) to TLS freelist: 3-5 cycles
   - Total: 5-10 cycles (vs 500+ cycles current!)

4. **Free Path Integration** (core/box/hak_free_api.inc.h):
   - Removed SuperSlab lookup from fast path
   - Direct header validation (no lookup needed!)

5. **Size Class Adjustment** (core/hakmem_tiny.h):
   - Max tiny size: 1023B (was 1024B)
   - 1024B requests → Mid allocator fallback

## Performance Results

| Size | Baseline | Phase 7 | Improvement |
|------|----------|---------|-------------|
| 128B | 1.22M | 6.54M | **+436%** 🚀 |
| 512B | 1.22M | 1.70M | **+39%** |
| 1023B | 1.22M | 1.92M | **+57%** |

## Build & Test

Enable Phase 7:
  make HEADER_CLASSIDX=1 bench_random_mixed_hakmem

Run benchmark:
  HAKMEM_TINY_USE_SUPERSLAB=1 ./bench_random_mixed_hakmem 10000 128 1234567

## Known Issues

- 1024B requests fallback to Mid allocator (by design)
- Target 40-60M ops/s not yet reached (current: 1.7-6.5M)
- Further optimization needed (TLS capacity tuning, refill optimization)

## Credits

Design: ChatGPT Pro Ultrathink, Claude Code
Implementation: Claude Code with Task Agent Ultrathink support

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-08 03:18:17 +09:00
Moe Charm (CI)
b7021061b8 Fix: CRITICAL double-allocation bug in trc_linear_carve() 
Root Cause:
trc_linear_carve() used meta->used as cursor, but meta->used decrements
on free, causing already-allocated blocks to be re-carved.

Evidence:
- [LINEAR_CARVE] used=61 batch=1 → block 61 created
- (blocks freed, used decrements 62→59)
- [LINEAR_CARVE] used=59 batch=3 → blocks 59,60,61 RE-CREATED!
- Result: double-allocation → memory corruption → SEGV

Fix Implementation:
1. Added TinySlabMeta.carved (monotonic counter, never decrements)
2. Changed trc_linear_carve() to use carved instead of used
3. carved tracks carve progress, used tracks active count

Files Modified:
- core/superslab/superslab_types.h: Add carved field
- core/tiny_refill_opt.h: Use carved in trc_linear_carve()
- core/hakmem_tiny_superslab.c: Initialize carved=0
- core/tiny_alloc_fast.inc.h: Add next pointer validation
- core/hakmem_tiny_free.inc: Add drain/free validation

Test Results:
✅ bench_random_mixed: 950,037 ops/s (no crash)
✅ Fail-fast mode: 651,627 ops/s (with diagnostic logs)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-08 01:18:37 +09:00
Moe Charm (CI)
1da8754d45 CRITICAL FIX: TLS 未初期化による 4T SEGV を完全解消 
**問題:**
- Larson 4T で 100% SEGV (1T は 2.09M ops/s で完走)
- System/mimalloc は 4T で 33.52M ops/s 正常動作
- SS OFF + Remote OFF でも 4T で SEGV

**根本原因: (Task agent ultrathink 調査結果)**
```
CRASH: mov (%r15),%r13
R15 = 0x6261  ← ASCII "ba" (ゴミ値、未初期化TLS)
```

Worker スレッドの TLS 変数が未初期化:
- `__thread void* g_tls_sll_head[TINY_NUM_CLASSES];`  ← 初期化なし
- pthread_create() で生成されたスレッドでゼロ初期化されない
- NULL チェックが通過 (0x6261 != NULL) → dereference → SEGV

**修正内容:**
全 TLS 配列に明示的初期化子 `= {0}` を追加:

1. **core/hakmem_tiny.c:**
   - `g_tls_sll_head[TINY_NUM_CLASSES] = {0}`
   - `g_tls_sll_count[TINY_NUM_CLASSES] = {0}`
   - `g_tls_live_ss[TINY_NUM_CLASSES] = {0}`
   - `g_tls_bcur[TINY_NUM_CLASSES] = {0}`
   - `g_tls_bend[TINY_NUM_CLASSES] = {0}`

2. **core/tiny_fastcache.c:**
   - `g_tiny_fast_cache[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_count[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_free_head[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_free_count[TINY_FAST_CLASS_COUNT] = {0}`

3. **core/hakmem_tiny_magazine.c:**
   - `g_tls_mags[TINY_NUM_CLASSES] = {0}`

4. **core/tiny_sticky.c:**
   - `g_tls_sticky_ss[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}`
   - `g_tls_sticky_idx[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}`
   - `g_tls_sticky_pos[TINY_NUM_CLASSES] = {0}`

**効果:**
```
Before: 1T: 2.09M ✅  |  4T: SEGV 💀
After:  1T: 2.41M ✅  |  4T: 4.19M ✅  (+15% 1T, SEGV解消)
```

**テスト:**
```bash
# 1 thread: 完走
./larson_hakmem 2 8 128 1024 1 12345 1
→ Throughput = 2,407,597 ops/s ✅

# 4 threads: 完走（以前は SEGV）
./larson_hakmem 2 8 128 1024 1 12345 4
→ Throughput = 4,192,155 ops/s ✅
```

**調査協力:** Task agent (ultrathink mode) による完璧な根本原因特定

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-07 01:27:04 +09:00
Moe Charm (CI)
5ea6c1237b Tiny: add per-class refill count tuning infrastructure (ChatGPT) 
External AI (ChatGPT Pro) implemented hierarchical refill count tuning:
- Move getenv() from hot path to init (performance hygiene)
- Add per-class granularity: global → hot/mid → per-class precedence
- Environment variables:
  * HAKMEM_TINY_REFILL_COUNT (global default)
  * HAKMEM_TINY_REFILL_COUNT_HOT (classes 0-3)
  * HAKMEM_TINY_REFILL_COUNT_MID (classes 4-7)
  * HAKMEM_TINY_REFILL_COUNT_C{0..7} (per-class override)

Performance impact: Neutral (no tuning applied yet, default=16)
- Larson 4-thread: 4.19M ops/s (unchanged)
- No measurable overhead from init-time parsing

Code quality improvement:
- Better separation: hot path reads plain ints (no syscalls)
- Future-proof: enables A/B testing per size class
- Documentation: ENV_VARS.md updated

Note: Per Ultrathink's advice, further tuning deferred until bottleneck
visualization (superslab_refill branch analysis) is complete.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: ChatGPT <external-ai@openai.com>
 2025-11-05 17:45:11 +09:00
Claude
af938fe378 Add RDTSC profiling - Identify refill bottleneck 
Profiling Results:
- Fast path: 143 cycles (10.4% of time) ✅ Good
- Refill: 19,624 cycles (89.6% of time) 🚨 Bottleneck!

Refill is 137x slower than fast path and dominates total cost.
Only happens 6.3% of the time but takes 90% of execution time.

Next: Optimize sll_refill_small_from_ss() backend.
 2025-11-05 06:35:03 +00:00
Moe Charm (CI)
52386401b3 Debug Counters Implementation - Clean History 
Major Features:
- Debug counter infrastructure for Refill Stage tracking
- Free Pipeline counters (ss_local, ss_remote, tls_sll)
- Diagnostic counters for early return analysis
- Unified larson.sh benchmark runner with profiles
- Phase 6-3 regression analysis documentation

Bug Fixes:
- Fix SuperSlab disabled by default (HAKMEM_TINY_USE_SUPERSLAB)
- Fix profile variable naming consistency
- Add .gitignore patterns for large files

Performance:
- Phase 6-3: 4.79 M ops/s (has OOM risk)
- With SuperSlab: 3.13 M ops/s (+19% improvement)

This is a clean repository without large log files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-05 12:31:14 +09:00