tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
340 Commits 2 Branches 0 Tags
2bd8da926762d4cd00148444a5eec49959cd7612
Commit Graph

57 Commits

Author SHA1 Message Date
Moe Charm (CI)
382980d450 Phase 6-2.4: Fix SuperSlab free SEGV: remove guess loop and add memory readability check; add registry atomic consistency (base as _Atomic uintptr_t with acq/rel); add debug toggles (SUPER_REG_DEBUG/REQTRACE); update CURRENT_TASK with results and next steps; capture suite results. 2025-11-07 18:07:48 +09:00
Moe Charm (CI)
b6d9c92f71 Fix: SuperSlab guess loop & header magic SEGV (random_mixed/mid_large_mt) 
## Problem
bench_random_mixed_hakmem and bench_mid_large_mt_hakmem crashed with SEGV:
- random_mixed: Exit 139 (SEGV) ❌
- mid_large_mt: Exit 139 (SEGV) ❌
- Larson: 838K ops/s ✅ (worked fine)

Error: Unmapped memory dereference in free path

## Root Causes (2 bugs found by Ultrathink Task)

### Bug 1: Guess Loop (core/box/hak_free_api.inc.h:92-95)
```c
for (int lg=21; lg>=20; lg--) {
    SuperSlab* guess=(SuperSlab*)((uintptr_t)ptr & ~mask);
    if (guess && guess->magic==SUPERSLAB_MAGIC) {  // ← SEGV
        // Dereferences unmapped memory
    }
}
```

### Bug 2: Header Magic Check (core/box/hak_free_api.inc.h:115)
```c
void* raw = (char*)ptr - HEADER_SIZE;
AllocHeader* hdr = (AllocHeader*)raw;
if (hdr->magic != HAKMEM_MAGIC) {  // ← SEGV
    // Dereferences unmapped memory if ptr has no header
}
```

**Why SEGV:**
- Registry lookup fails (allocation not from SuperSlab)
- Guess loop calculates 1MB/2MB aligned address
- No memory mapping validation
- Dereferences unmapped memory → SEGV

**Why Larson worked but random_mixed failed:**
- Larson: All from SuperSlab → registry hit → never reaches guess loop
- random_mixed: Diverse sizes (8-4096B) → registry miss → enters buggy paths

**Why LD_PRELOAD worked:**
- hak_core_init.inc.h:119-121 disables SuperSlab by default
- → SS-first path skipped → buggy code never executed

## Fix (2-part)

### Part 1: Remove Guess Loop
File: core/box/hak_free_api.inc.h:92-95
- Deleted unsafe guess loop (4 lines)
- If registry lookup fails, allocation is not from SuperSlab

### Part 2: Add Memory Safety Check
File: core/hakmem_internal.h:277-294
```c
static inline int hak_is_memory_readable(void* addr) {
    unsigned char vec;
    return mincore(addr, 1, &vec) == 0;  // Check if mapped
}
```

File: core/box/hak_free_api.inc.h:115-131
```c
if (!hak_is_memory_readable(raw)) {
    // Not accessible → route to appropriate handler
    // Prevents SEGV on unmapped memory
    goto done;
}
// Safe to dereference now
AllocHeader* hdr = (AllocHeader*)raw;
```

## Verification

| Test | Before | After | Result |
|------|--------|-------|--------|
| random_mixed (2KB) | ❌ SEGV | ✅ 2.22M ops/s | 🎉 Fixed |
| random_mixed (4KB) | ❌ SEGV | ✅ 2.58M ops/s | 🎉 Fixed |
| Larson 4T | ✅ 838K | ✅ 838K ops/s | ✅ No regression |

**Performance Impact:** 0% (mincore only on fallback path)

## Investigation

- Complete analysis: SEGV_ROOT_CAUSE_COMPLETE.md
- Fix report: SEGV_FIX_REPORT.md
- Previous investigation: SEGFAULT_INVESTIGATION_REPORT.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-07 17:34:24 +09:00
Moe Charm (CI)
f6b06a0311 Fix: Active counter double-decrement in P0 batch refill (4T crash → stable) 
## Problem
HAKMEM 4T crashed with "free(): invalid pointer" on startup:
- System/mimalloc: 3.3M ops/s ✅
- HAKMEM 1T: 838K ops/s (-75%) ⚠️
- HAKMEM 4T: Crash (Exit 134) ❌

Error: superslab_refill returned NULL (OOM), active=0, bitmap=0x00000000

## Root Cause (Ultrathink Task Agent Investigation)
Active counter double-decrement when re-allocating from freelist:

1. Free → counter-- ✅
2. Remote drain → add to freelist (no counter change) ✅
3. P0 batch refill → move to TLS cache (forgot counter++) ❌ BUG!
4. Next free → counter-- ❌ Double decrement!

Result: Counter underflow → SuperSlab appears "full" → OOM → crash

## Fix (1 line)
File: core/hakmem_tiny_refill_p0.inc.h:103

+ss_active_add(tls->ss, from_freelist);

Reason: Freelist re-allocation moves block from "free" to "allocated" state,
so active counter MUST increment.

## Verification
| Setting        | Before  | After          | Result       |
|----------------|---------|----------------|--------------|
| 4T default     | ❌ Crash | ✅ 838,445 ops/s | 🎉 Stable    |
| Stability (2x) | -       | ✅ Same score   | Reproducible |

## Remaining Issue
❌ HAKMEM_TINY_REFILL_COUNT_HOT=64 triggers crash (class=4 OOM)
- Suspected: TLS cache over-accumulation or memory leak
- Next: Investigate HAKMEM_TINY_FAST_CAP interaction

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-07 12:37:23 +09:00
Moe Charm (CI)
8f3095fb85 CI-safe debug runners: add ASan LD_PRELOAD + UBSan mailbox targets; add asan_preload script; document sanitizer-safe workflows and results in CURRENT_TASK.md (debug complete). 2025-11-07 12:09:28 +09:00
Moe Charm (CI)
1da8754d45 CRITICAL FIX: TLS 未初期化による 4T SEGV を完全解消 
**問題:**
- Larson 4T で 100% SEGV (1T は 2.09M ops/s で完走)
- System/mimalloc は 4T で 33.52M ops/s 正常動作
- SS OFF + Remote OFF でも 4T で SEGV

**根本原因: (Task agent ultrathink 調査結果)**
```
CRASH: mov (%r15),%r13
R15 = 0x6261  ← ASCII "ba" (ゴミ値、未初期化TLS)
```

Worker スレッドの TLS 変数が未初期化:
- `__thread void* g_tls_sll_head[TINY_NUM_CLASSES];`  ← 初期化なし
- pthread_create() で生成されたスレッドでゼロ初期化されない
- NULL チェックが通過 (0x6261 != NULL) → dereference → SEGV

**修正内容:**
全 TLS 配列に明示的初期化子 `= {0}` を追加:

1. **core/hakmem_tiny.c:**
   - `g_tls_sll_head[TINY_NUM_CLASSES] = {0}`
   - `g_tls_sll_count[TINY_NUM_CLASSES] = {0}`
   - `g_tls_live_ss[TINY_NUM_CLASSES] = {0}`
   - `g_tls_bcur[TINY_NUM_CLASSES] = {0}`
   - `g_tls_bend[TINY_NUM_CLASSES] = {0}`

2. **core/tiny_fastcache.c:**
   - `g_tiny_fast_cache[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_count[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_free_head[TINY_FAST_CLASS_COUNT] = {0}`
   - `g_tiny_fast_free_count[TINY_FAST_CLASS_COUNT] = {0}`

3. **core/hakmem_tiny_magazine.c:**
   - `g_tls_mags[TINY_NUM_CLASSES] = {0}`

4. **core/tiny_sticky.c:**
   - `g_tls_sticky_ss[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}`
   - `g_tls_sticky_idx[TINY_NUM_CLASSES][TINY_STICKY_RING] = {0}`
   - `g_tls_sticky_pos[TINY_NUM_CLASSES] = {0}`

**効果:**
```
Before: 1T: 2.09M ✅  |  4T: SEGV 💀
After:  1T: 2.41M ✅  |  4T: 4.19M ✅  (+15% 1T, SEGV解消)
```

**テスト:**
```bash
# 1 thread: 完走
./larson_hakmem 2 8 128 1024 1 12345 1
→ Throughput = 2,407,597 ops/s ✅

# 4 threads: 完走（以前は SEGV）
./larson_hakmem 2 8 128 1024 1 12345 4
→ Throughput = 4,192,155 ops/s ✅
```

**調査協力:** Task agent (ultrathink mode) による完璧な根本原因特定

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-07 01:27:04 +09:00
Moe Charm (CI)
582ebdfd4f CURRENT_TASK: Registry 線形スキャン ボトルネック特定 (2025-11-05) 
- perf 分析で superslab_refill が 28.51% CPU を消費
- Root cause: 262,144 エントリの線形スキャン (97.65% の hot instructions)
- 解決策: per-class registry (8×4096 = 32K entries)
- 期待効果: +200-300% (2.59M → 7.8-10.4M ops/s)
- Box Refactor は既に動いている (+463% ST, +131% MT)

次のアクション: Phase 1 実装 (per-class registry 変更)

詳細: PERF_ANALYSIS_2025_11_05.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-05 16:47:04 +09:00
Moe Charm (CI)
52386401b3 Debug Counters Implementation - Clean History 
Major Features:
- Debug counter infrastructure for Refill Stage tracking
- Free Pipeline counters (ss_local, ss_remote, tls_sll)
- Diagnostic counters for early return analysis
- Unified larson.sh benchmark runner with profiles
- Phase 6-3 regression analysis documentation

Bug Fixes:
- Fix SuperSlab disabled by default (HAKMEM_TINY_USE_SUPERSLAB)
- Fix profile variable naming consistency
- Add .gitignore patterns for large files

Performance:
- Phase 6-3: 4.79 M ops/s (has OOM risk)
- With SuperSlab: 3.13 M ops/s (+19% improvement)

This is a clean repository without large log files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
 2025-11-05 12:31:14 +09:00