hakmem/docs/CHATGPT_HANDOFF_TLS_DIAGNOSIS.md

ChatGPT Task: TLS SLL Header Corruption Diagnosis & Fix

Status: BLOCKING - System instability detected in baseline configuration Priority: CRITICAL Assigned to: Claude (ChatGPT model) Expected Duration: 4-8 hours

---

Executive Summary

The hakmem memory allocator baseline configuration crashes with a critical header corruption error:

[TLS_SLL_HDR_RESET] cls=1 base=0x... got=0x31 expect=0xa1

This occurs in shared code paths (not Phase 1 specific), blocking all further development and validation.

Your Task: Diagnose and fix this issue using the comprehensive diagnostic guide.

---

What You Need to Know

Context

• Project: hakmem - custom memory allocator with "Box Theory" architecture
• Language: C
• Current Phase: Phase 1 implementation + Phase 2 (Headerless) planning
• Problem: Baseline test crashes before completing benchmarks
• Error Location: core/box/tls_sll_box.h - header validation during TLS SLL pop

The Error

When a block is popped from the TLS SLL (Thread-Local Single-Linked List), the header validation checks:

uint8_t got = *b;              // Read byte at offset 0 of base pointer
uint8_t expected = 0xa0 | class_idx;  // For class 1: 0xa1

if (got != expected) {
    // ERROR DETECTED - got 0x31 instead of 0xa1
}

The header byte contains user data (0x31 = '1' character) instead of the expected magic value (0xa1).

This means: Either:

1. Wrong pointer was stored in TLS SLL
2. Header was not written before pushing to TLS SLL
3. Header was overwritten after pushing
4. Offset calculation is wrong

---

Your Step-by-Step Task

Step 1: Read the Comprehensive Diagnostic Document

File: /mnt/workdisk/public_share/hakmem/docs/TLS_SLL_HEADER_CORRUPTION_DIAGNOSIS.md

This 1,150+ line document contains:

• 6 detailed root cause patterns with code examples
• Minimal test case template (test_tls_sll_minimal.c)
• Diagnostic logging instrumentation points
• Fix patterns with code snippets
• 7-step validation procedure

Action: Read the entire document and understand the investigation methodology.

---

Step 2: Reproduce the Error with Minimal Test Case

Create /mnt/workdisk/public_share/hakmem/tests/test_tls_sll_minimal.c based on template in the diagnostic document.

cd /mnt/workdisk/public_share/hakmem

# Build minimal test
gcc -g -O1 -I./core -I./core/box \
    tests/test_tls_sll_minimal.c \
    -L. -lhakmem -lpthread -o test_minimal

# Run (should crash with TLS_SLL_HDR_RESET error)
./test_minimal 2>&1 | grep -E "TLS_SLL_HDR_RESET|Segmentation"

Expected Output: Should reproduce the header corruption within first 100-1000 allocations.

---

Step 3: Add Diagnostic Logging

Instrument the following locations to capture when header corruption occurs:

Location A: core/hakmem_tiny_free.inc - Header write before TLS SLL push

// Around line 550: Before tls_sll_push()
// ADD LOGGING:
fprintf(stderr, "[HEADER_WRITE] base=%p, offset=%zu, writing 0x%02x\n",
        base, offset, (HEADER_MAGIC | (class_idx & HEADER_CLASS_MASK)));

Location B: core/box/tls_sll_box.h - Header read during pop

// Around line 282-303: In tls_sll_pop_impl()
// ADD LOGGING:
fprintf(stderr, "[HEADER_READ] base=%p, got=0x%02x, expected=0x%02x\n",
        raw_base, got, expected);

Location C: core/hakmem_tiny_refill.inc.h - Magazine spill

// Around line 228: Before/after tls_sll_push()
// ADD LOGGING:
fprintf(stderr, "[SPILL] class=%d, ptr=%p (wrapping to base)\n", class_idx, p);

Action: Add detailed logging to identify which allocation/free cycle causes corruption.

---

Step 4: Run Diagnostic Test with Logging

# Rebuild with logging enabled
make clean
make shared -j8 EXTRA_CFLAGS="-g -O1 -DHAKMEM_TINY_DEBUG_LOGGING=1"

# Run minimal test and capture log
./test_minimal 2>&1 | tee diagnostic_output.txt

# Analyze log to find last successful write before corruption
grep HEADER_WRITE diagnostic_output.txt | tail -10
grep HEADER_READ diagnostic_output.txt | grep -A1 -B1 "0x31"

Expected Result: Log will show exact allocation/free sequence leading to corruption.

---

Step 5: Identify Root Cause (One of Six Patterns)

Based on diagnostic logs, match against these patterns from the diagnostic document:

1. RAW Pointer vs BASE Pointer: Wrong pointer type passed to tls_sll_push()
2. Header Offset Mismatch: Writing at offset 1, reading at offset 0
3. Atomic Fence Missing: Compiler reordering causing write-after-push
4. Adjacent Block Overflow: User data from preceding block overwrites header
5. Class Index Mismatch: Push with class_idx A, pop as class_idx B
6. Headerless Mode Interference: Mixed header/headerless logic

Action: Determine which pattern applies to your findings.

---

Step 6: Implement Surgical Fix

Once root cause is identified, apply a minimal fix (typically 1-5 lines):

Example fixes (from diagnostic document):

// Pattern 1 - RAW vs BASE pointer:
// WRONG:
tls_sll_push(class_idx, p, size);  // p is RAW pointer
// FIXED:
hak_base_ptr_t base = HAK_BASE_FROM_RAW(p);
tls_sll_push(class_idx, base, size);

// Pattern 2 - Offset mismatch:
// WRONG:
*(uint8_t*)((char*)base + 1) = header;  // Writing at offset 1
// In pop: uint8_t h = *((uint8_t*)base);  // Reading at offset 0
// FIXED:
*(uint8_t*)base = header;  // Consistent offset

// Pattern 3 - Atomic fence missing:
// WRONG:
*hdr = magic;
tls_sll_push(...);
// FIXED:
*hdr = magic;
atomic_thread_fence(memory_order_release);  // Prevent reordering
tls_sll_push(...);

Action: Apply fix to source code and rebuild.

---

Step 7: Validate Fix

# Step 7a: Run minimal test
./test_minimal 2>&1 | grep -E "TLS_SLL_HDR_RESET|passed|failed"

# Step 7b: Run baseline benchmark
make clean
make shared -j8
LD_PRELOAD=./libhakmem.so ./mimalloc-bench/out/bench/sh8bench 2>&1 | \
  grep -E "TLS_SLL_HDR_RESET|Total|PASSED|FAILED"

# Step 7c: Run cfrac (memory intensive)
LD_PRELOAD=./libhakmem.so ./mimalloc-bench/out/bench/cfrac 2>&1 | \
  grep -E "error|TLS_SLL_HDR_RESET|Total"

# Step 7d: Check for regressions
make test -j8 FILTER="tls_sll"

Success Criteria:

• ✅ Minimal test completes without TLS_SLL_HDR_RESET
• ✅ sh8bench runs to completion (several minutes)
• ✅ cfrac completes without errors
• ✅ All unit tests pass
• ✅ No performance regression (< 5%)

---

Commit & Documentation

Once validated, commit with detailed message:

git add -A
git commit -m "Fix TLS SLL header corruption in [Component]

Root Cause:
[Brief 1-2 sentence explanation of what was wrong]

Pattern Affected:
[Which of the 6 patterns this was]

Fix Applied:
[Minimal description of the fix]

Validation:
- [Test case] passed
- [Benchmark] completed without TLS_SLL_HDR_RESET
- No performance regression

Related Issues:
- TLS SLL baseline instability
- Required for Phase 1/2 validation"

---

Reference Files

File	Purpose
docs/TLS_SLL_HEADER_CORRUPTION_DIAGNOSIS.md	Complete diagnostic guide - READ FIRST
core/box/tls_sll_box.h	TLS SLL implementation (header validation at lines 282-303)
core/hakmem_tiny_free.inc	Free path (header write before push, lines ~550)
core/hakmem_tiny_refill.inc.h	Magazine spill (lines ~228)
docs/HEADERLESS_STABILITY_DEBUG_INSTRUCTIONS.md	Test environment setup
debug_artifacts/headerless/	Benchmark results showing error

---

Communication Plan

Status Updates: After each step, provide brief status:

• Step 2: "Reproducer created - X allocations before crash"
• Step 3: "Logging added to [X locations]"
• Step 4: "Log analysis complete - [pattern identified]"
• Step 5: "Root cause identified: Pattern #[N]"
• Step 6: "Fix applied - [brief description]"
• Step 7: "Validation complete - [test results]"

---

Post-Fix: Unblocking Next Phases

Once this issue is fixed, the following can proceed:

1. Phase 1 Completion: TLS Hint Box performance optimization (currently showing 2.3% improvement vs target 15-20%)
2. Phase 2 Validation: Test Headerless mode (ON/OFF configurations)
3. Performance Benchmarking: Full multi-test suite (TC1, TC2, TC3)
4. Future Phases: Phase 102 (MemApi bridge), production optimization

---

Success Metric

GOAL: TC1 baseline test completes successfully with zero TLS_SLL_HDR_RESET errors.

Current Status: ❌ FAILING (crashes at ~22 seconds) Target Status: ✅ PASSING (completion in 4-6 minutes)

---

Questions? Refer to the diagnostic document for detailed explanations of each pattern and debugging technique.

Ready to start? Begin with Step 1: Read the full diagnostic guide.

🚀 Your investigation begins now!