tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b5be708b6a70bc755c5ee8a059349d0109bbd0c2
hakmem/hakmem.d

204 lines
8.2 KiB
D
Raw Normal View History

Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
hakmem.o: core/hakmem.c core/hakmem.h core/hakmem_build_flags.h \
core/hakmem_config.h core/hakmem_features.h core/hakmem_internal.h \
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/hakmem_sys.h core/hakmem_whale.h core/box/ptr_type_box.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/hakmem_bigcache.h core/hakmem_pool.h \
core/box/hak_lane_classify.inc.h core/hakmem_l25_pool.h \
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/hakmem_policy.h core/hakmem_learner.h core/hakmem_size_hist.h \
core/hakmem_ace.h core/hakmem_site_rules.h core/hakmem_tiny.h \
core/hakmem_trace.h core/hakmem_tiny_mini_mag.h \
core/hakmem_tiny_superslab.h core/superslab/superslab_types.h \
core/hakmem_tiny_superslab_constants.h core/superslab/superslab_inline.h \
core/superslab/superslab_types.h core/superslab/../tiny_box_geometry.h \
Remove dead code: UltraHot, RingCache, FrontC23, Class5 Hotpath (cherry-picked from 225b6fcc7, conflicts resolved)
2025-11-26 12:33:49 +09:00
core/superslab/../hakmem_tiny_superslab_constants.h \
core/superslab/../hakmem_tiny_config.h core/tiny_debug_ring.h \
core/tiny_remote.h core/hakmem_tiny_superslab_constants.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/tiny_fastcache.h core/hakmem_env_cache.h \
core/box/tiny_next_ptr_box.h core/hakmem_tiny_config.h \
core/tiny_nextptr.h core/tiny_region_id.h core/tiny_box_geometry.h \
core/ptr_track.h core/hakmem_super_registry.h core/box/ss_addr_map_box.h \
core/box/../hakmem_build_flags.h core/tiny_debug_api.h core/hakmem_elo.h \
Cleanup: Add tiny_debug_api.h to eliminate guard/failfast implicit warnings Created central header for debug instrumentation API to fix implicit function declaration warnings across the codebase. Changes: 1. Created core/tiny_debug_api.h - Declares guard system API (3 functions) - Declares failfast debugging API (3 functions) - Uses forward declarations for SuperSlab/TinySlabMeta 2. Updated 3 files to include tiny_debug_api.h: - core/tiny_region_id.h (removed inline externs) - core/hakmem_tiny_tls_ops.h - core/tiny_superslab_alloc.inc.h Warnings eliminated (6 of 11 total): ✅ tiny_guard_is_enabled() ✅ tiny_guard_on_alloc() ✅ tiny_guard_on_invalid() ✅ tiny_failfast_log() ✅ tiny_failfast_abort_ptr() ✅ tiny_refill_failfast_level() Remaining warnings (deferred to P1): - ss_active_add (2 occurrences) - expand_superslab_head - hkm_ace_set_tls_capacity - smallmid_backend_free Impact: - Cleaner build output - Better type safety for debug functions - No behavior changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 06:47:13 +09:00
core/hakmem_ace_stats.h core/hakmem_batch.h core/hakmem_evo.h \
core/hakmem_debug.h core/hakmem_prof.h core/hakmem_syscall.h \
core/hakmem_ace_controller.h core/hakmem_ace_metrics.h \
core/hakmem_ace_ucb1.h core/box/bench_fast_box.h core/ptr_trace.h \
core/box/hak_exit_debug.inc.h core/box/hak_kpi_util.inc.h \
core/box/hak_core_init.inc.h core/hakmem_phase7_config.h \
core/box/ss_hot_prewarm_box.h core/box/hak_alloc_api.inc.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/../hakmem_tiny.h core/box/../hakmem_pool.h \
core/box/../hakmem_smallmid.h core/box/mid_large_config_box.h \
core/box/../hakmem_config.h core/box/../hakmem_features.h \
core/box/hak_free_api.inc.h core/hakmem_tiny_superslab.h \
core/box/../tiny_free_fast_v2.inc.h core/box/../tiny_region_id.h \
core/box/../hakmem_build_flags.h core/box/../hakmem_tiny_config.h \
core/box/../box/tls_sll_box.h core/box/../box/../hakmem_internal.h \
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../box/../hakmem_tiny_config.h \
Cleanup: Consolidate debug ENV vars to HAKMEM_DEBUG_LEVEL Integrated 4 new debug environment variables added during bug fixes into the existing unified HAKMEM_DEBUG_LEVEL system (expanded to 0-5 levels). Changes: 1. Expanded HAKMEM_DEBUG_LEVEL from 0-3 to 0-5 levels: - 0 = OFF (production) - 1 = ERROR (critical errors) - 2 = WARN (warnings) - 3 = INFO (allocation paths, header validation, stats) - 4 = DEBUG (guard instrumentation, failfast) - 5 = TRACE (verbose tracing) 2. Integrated 4 environment variables: - HAKMEM_ALLOC_PATH_TRACE → HAKMEM_DEBUG_LEVEL >= 3 (INFO) - HAKMEM_TINY_SLL_VALIDATE_HDR → HAKMEM_DEBUG_LEVEL >= 3 (INFO) - HAKMEM_TINY_REFILL_FAILFAST → HAKMEM_DEBUG_LEVEL >= 4 (DEBUG) - HAKMEM_TINY_GUARD → HAKMEM_DEBUG_LEVEL >= 4 (DEBUG) 3. Kept 2 special-purpose variables (fine-grained control): - HAKMEM_TINY_GUARD_CLASS (target class for guard) - HAKMEM_TINY_GUARD_MAX (max guard events) 4. Backward compatibility: - Legacy ENV vars still work via hak_debug_check_level() - New code uses unified system - No behavior changes for existing users Updated files: - core/hakmem_debug_master.h (level 0-5 expansion) - core/hakmem_tiny_superslab_internal.h (alloc path trace) - core/box/tls_sll_box.h (header validation) - core/tiny_failfast.c (failfast level) - core/tiny_refill_opt.h (failfast guard) - core/hakmem_tiny_ace_guard_box.inc (guard enable) - core/hakmem_tiny.c (include hakmem_debug_master.h) Impact: - Simpler debug control: HAKMEM_DEBUG_LEVEL=3 instead of 4 separate ENVs - Easier to discover/use - Consistent debug levels across codebase - Reduces ENV variable proliferation (43+ vars surveyed) Future work: - Consolidate remaining 39+ debug variables (documented in survey) - Gradual migration over 2-3 releases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 06:57:03 +09:00
core/box/../box/../hakmem_build_flags.h \
core/box/../box/../hakmem_debug_master.h \
core/box/../box/../tiny_remote.h core/box/../box/../tiny_region_id.h \
Add Box I (Integrity), Box E (Expansion), and comprehensive P0 debugging infrastructure ## Major Additions ### 1. Box I: Integrity Verification System (NEW - 703 lines) - Files: core/box/integrity_box.h (267 lines), core/box/integrity_box.c (436 lines) - Purpose: Unified integrity checking across all HAKMEM subsystems - Features: * 4-level integrity checking (0-4, compile-time controlled) * Priority 1: TLS array bounds validation * Priority 2: Freelist pointer validation * Priority 3: TLS canary monitoring * Priority ALPHA: Slab metadata invariant checking (5 invariants) * Atomic statistics tracking (thread-safe) * Beautiful BOX_BOUNDARY design pattern ### 2. Box E: SuperSlab Expansion System (COMPLETE) - Files: core/box/superslab_expansion_box.h, core/box/superslab_expansion_box.c - Purpose: Safe SuperSlab expansion with TLS state guarantee - Features: * Immediate slab 0 binding after expansion * TLS state snapshot and restoration * Design by Contract (pre/post-conditions, invariants) * Thread-safe with mutex protection ### 3. Comprehensive Integrity Checking System - File: core/hakmem_tiny_integrity.h (NEW) - Unified validation functions for all allocator subsystems - Uninitialized memory pattern detection (0xa2, 0xcc, 0xdd, 0xfe) - Pointer range validation (null-page, kernel-space) ### 4. P0 Bug Investigation - Root Cause Identified **Bug**: SEGV at iteration 28440 (deterministic with seed 42) **Pattern**: 0xa2a2a2a2a2a2a2a2 (uninitialized/ASan poisoning) **Location**: TLS SLL (Single-Linked List) cache layer **Root Cause**: Race condition or use-after-free in TLS list management (class 0) **Detection**: Box I successfully caught invalid pointer at exact crash point ### 5. Defensive Improvements - Defensive memset in SuperSlab allocation (all metadata arrays) - Enhanced pointer validation with pattern detection - BOX_BOUNDARY markers throughout codebase (beautiful modular design) - 5 metadata invariant checks in allocation/free/refill paths ## Integration Points - Modified 13 files with Box I/E integration - Added 10+ BOX_BOUNDARY markers - 5 critical integrity check points in P0 refill path ## Test Results (100K iterations) - Baseline: 7.22M ops/s - Hotpath ON: 8.98M ops/s (+24% improvement ✓) - P0 Bug: Still crashes at 28440 iterations (TLS SLL race condition) - Root cause: Identified but not yet fixed (requires deeper investigation) ## Performance - Box I overhead: Zero in release builds (HAKMEM_INTEGRITY_LEVEL=0) - Debug builds: Full validation enabled (HAKMEM_INTEGRITY_LEVEL=4) - Beautiful modular design maintains clean separation of concerns ## Known Issues - P0 Bug at 28440 iterations: Race condition in TLS SLL cache (class 0) - Cause: Use-after-free or race in remote free draining - Next step: Valgrind investigation to pinpoint exact corruption location ## Code Quality - Total new code: ~1400 lines (Box I + Box E + integrity system) - Design: Beautiful Box Theory with clear boundaries - Modularity: Complete separation of concerns - Documentation: Comprehensive inline comments and BOX_BOUNDARY markers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-12 02:45:00 +09:00
core/box/../box/../hakmem_tiny_integrity.h \
Phase E3-FINAL: Fix Box API offset bugs - ALL classes now use correct offsets ## Root Cause Analysis (GPT5) **Physical Layout Constraints**: - Class 0: 8B = [1B header][7B payload] → offset 1 = 9B needed = ❌ IMPOSSIBLE - Class 1-6: >=16B = [1B header][15B+ payload] → offset 1 = ✅ POSSIBLE - Class 7: 1KB → offset 0 (compatibility) **Correct Specification**: - HAKMEM_TINY_HEADER_CLASSIDX != 0: - Class 0, 7: next at offset 0 (overwrites header when on freelist) - Class 1-6: next at offset 1 (after header) - HAKMEM_TINY_HEADER_CLASSIDX == 0: - All classes: next at offset 0 **Previous Bug**: - Attempted "ALL classes offset 1" unification - Class 0 with offset 1 caused immediate SEGV (9B > 8B block size) - Mixed 2-arg/3-arg API caused confusion ## Fixes Applied ### 1. Restored 3-Argument Box API (core/box/tiny_next_ptr_box.h) ```c // Correct signatures void tiny_next_write(int class_idx, void* base, void* next_value) void* tiny_next_read(int class_idx, const void* base) // Correct offset calculation size_t offset = (class_idx == 0 || class_idx == 7) ? 0 : 1; ``` ### 2. Updated 123+ Call Sites Across 34 Files - hakmem_tiny_hot_pop_v4.inc.h (4 locations) - hakmem_tiny_fastcache.inc.h (3 locations) - hakmem_tiny_tls_list.h (12 locations) - superslab_inline.h (5 locations) - tiny_fastcache.h (3 locations) - ptr_trace.h (macro definitions) - tls_sll_box.h (2 locations) - + 27 additional files Pattern: `tiny_next_read(base)` → `tiny_next_read(class_idx, base)` Pattern: `tiny_next_write(base, next)` → `tiny_next_write(class_idx, base, next)` ### 3. Added Sentinel Detection Guards - tiny_fast_push(): Block nodes with sentinel in ptr or ptr->next - tls_list_push(): Block nodes with sentinel in ptr or ptr->next - Defense-in-depth against remote free sentinel leakage ## Verification (GPT5 Report) **Test Command**: `./out/release/bench_random_mixed_hakmem --iterations=70000` **Results**: - ✅ Main loop completed successfully - ✅ Drain phase completed successfully - ✅ NO SEGV (previous crash at iteration 66151 is FIXED) - ℹ️ Final log: "tiny_alloc(1024) failed" is normal fallback to Mid/ACE layers **Analysis**: - Class 0 immediate SEGV: ✅ RESOLVED (correct offset 0 now used) - 66K iteration crash: ✅ RESOLVED (offset consistency fixed) - Box API conflicts: ✅ RESOLVED (unified 3-arg API) ## Technical Details ### Offset Logic Justification ``` Class 0: 8B block → next pointer (8B) fits ONLY at offset 0 Class 1: 16B block → next pointer (8B) fits at offset 1 (after 1B header) Class 2: 32B block → next pointer (8B) fits at offset 1 ... Class 6: 512B block → next pointer (8B) fits at offset 1 Class 7: 1024B block → offset 0 for legacy compatibility ``` ### Files Modified (Summary) - Core API: `box/tiny_next_ptr_box.h` - Hot paths: `hakmem_tiny_hot_pop*.inc.h`, `tiny_fastcache.h` - TLS layers: `hakmem_tiny_tls_list.h`, `hakmem_tiny_tls_ops.h` - SuperSlab: `superslab_inline.h`, `tiny_superslab_*.inc.h` - Refill: `hakmem_tiny_refill.inc.h`, `tiny_refill_opt.h` - Free paths: `tiny_free_magazine.inc.h`, `tiny_superslab_free.inc.h` - Documentation: Multiple Phase E3 reports ## Remaining Work None for Box API offset bugs - all structural issues resolved. Future enhancements (non-critical): - Periodic `grep -R '*(void**)' core/` to detect direct pointer access violations - Enforce Box API usage via static analysis - Document offset rationale in architecture docs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 06:50:20 +09:00
core/box/../box/../hakmem_tiny.h core/box/../box/../ptr_track.h \
Code Cleanup: Remove false positives, redundant validations, and reduce verbose logging Following the C7 stride upgrade fix (commit 23c0d9541), this commit performs comprehensive cleanup to improve code quality and reduce debug noise. ## Changes ### 1. Disable False Positive Checks (tiny_nextptr.h) - **Disabled**: NXT_MISALIGN validation block with `#if 0` - **Reason**: Produces false positives due to slab base offsets (2048, 65536) not being stride-aligned, causing all blocks to appear "misaligned" - **TODO**: Reimplement to check stride DISTANCE between consecutive blocks instead of absolute alignment to stride boundaries ### 2. Remove Redundant Geometry Validations **hakmem_tiny_refill_p0.inc.h (P0 batch refill)** - Removed 25-line CARVE_GEOMETRY_FIX validation block - Replaced with NOTE explaining redundancy - **Reason**: Stride table is now correct in tiny_block_stride_for_class(), defense-in-depth validation adds overhead without benefit **ss_legacy_backend_box.c (legacy backend)** - Removed 18-line LEGACY_FIX_GEOMETRY validation block - Replaced with NOTE explaining redundancy - **Reason**: Shared_pool validates geometry at acquisition time ### 3. Reduce Verbose Logging **hakmem_shared_pool.c (sp_fix_geometry_if_needed)** - Made SP_FIX_GEOMETRY logging conditional on `!HAKMEM_BUILD_RELEASE` - **Reason**: Geometry fixes are expected during stride upgrades, no need to log in release builds ### 4. Verification - Build: ✅ Successful (LTO warnings expected) - Test: ✅ 10K iterations (1.87M ops/s, no crashes) - NXT_MISALIGN false positives: ✅ Eliminated ## Files Modified - core/tiny_nextptr.h - Disabled false positive NXT_MISALIGN check - core/hakmem_tiny_refill_p0.inc.h - Removed redundant CARVE validation - core/box/ss_legacy_backend_box.c - Removed redundant LEGACY validation - core/hakmem_shared_pool.c - Made SP_FIX_GEOMETRY logging debug-only ## Impact - **Code clarity**: Removed 43 lines of redundant validation code - **Debug noise**: Reduced false positive diagnostics - **Performance**: Eliminated overhead from redundant geometry checks - **Maintainability**: Single source of truth for geometry validation 🧹 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:00:24 +09:00
core/box/../box/../tiny_debug_ring.h \
core/box/../box/../superslab/superslab_inline.h \
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../box/tiny_header_box.h core/box/../box/../tiny_nextptr.h \
Code Cleanup: Remove false positives, redundant validations, and reduce verbose logging Following the C7 stride upgrade fix (commit 23c0d9541), this commit performs comprehensive cleanup to improve code quality and reduce debug noise. ## Changes ### 1. Disable False Positive Checks (tiny_nextptr.h) - **Disabled**: NXT_MISALIGN validation block with `#if 0` - **Reason**: Produces false positives due to slab base offsets (2048, 65536) not being stride-aligned, causing all blocks to appear "misaligned" - **TODO**: Reimplement to check stride DISTANCE between consecutive blocks instead of absolute alignment to stride boundaries ### 2. Remove Redundant Geometry Validations **hakmem_tiny_refill_p0.inc.h (P0 batch refill)** - Removed 25-line CARVE_GEOMETRY_FIX validation block - Replaced with NOTE explaining redundancy - **Reason**: Stride table is now correct in tiny_block_stride_for_class(), defense-in-depth validation adds overhead without benefit **ss_legacy_backend_box.c (legacy backend)** - Removed 18-line LEGACY_FIX_GEOMETRY validation block - Replaced with NOTE explaining redundancy - **Reason**: Shared_pool validates geometry at acquisition time ### 3. Reduce Verbose Logging **hakmem_shared_pool.c (sp_fix_geometry_if_needed)** - Made SP_FIX_GEOMETRY logging conditional on `!HAKMEM_BUILD_RELEASE` - **Reason**: Geometry fixes are expected during stride upgrades, no need to log in release builds ### 4. Verification - Build: ✅ Successful (LTO warnings expected) - Test: ✅ 10K iterations (1.87M ops/s, no crashes) - NXT_MISALIGN false positives: ✅ Eliminated ## Files Modified - core/tiny_nextptr.h - Disabled false positive NXT_MISALIGN check - core/hakmem_tiny_refill_p0.inc.h - Removed redundant CARVE validation - core/box/ss_legacy_backend_box.c - Removed redundant LEGACY validation - core/hakmem_shared_pool.c - Made SP_FIX_GEOMETRY logging debug-only ## Impact - **Code clarity**: Removed 43 lines of redundant validation code - **Debug noise**: Reduced false positive diagnostics - **Performance**: Eliminated overhead from redundant geometry checks - **Maintainability**: Single source of truth for geometry validation 🧹 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:00:24 +09:00
core/box/../box/tls_sll_drain_box.h core/box/../box/tls_sll_box.h \
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../box/slab_recycling_box.h \
core/box/../box/../hakmem_tiny_superslab.h \
core/box/../box/../hakmem_shared_pool.h \
core/box/../box/../superslab/superslab_types.h \
core/box/../box/ss_hot_cold_box.h \
core/box/../box/../superslab/superslab_types.h \
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/box/../box/free_local_box.h core/box/../box/ptr_type_box.h \
core/box/../box/free_publish_box.h core/hakmem_tiny.h \
core/tiny_region_id.h core/box/../hakmem_tiny_integrity.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/../hakmem_env_cache.h core/box/../superslab/superslab_inline.h \
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../box/ss_slab_meta_box.h \
Remove dead code: UltraHot, RingCache, FrontC23, Class5 Hotpath (cherry-picked from 225b6fcc7, conflicts resolved)
2025-11-26 12:33:49 +09:00
core/box/../box/slab_freelist_atomic.h core/box/../box/free_remote_box.h \
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/hakmem_tiny_integrity.h core/box/front_gate_v2.h \
core/box/external_guard_box.h core/box/ss_slab_meta_box.h \
core/box/fg_tiny_gate_box.h core/box/hak_wrappers.inc.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/front_gate_classifier.h core/box/../front/malloc_tiny_fast.h \
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../front/../hakmem_build_flags.h \
core/box/../front/../hakmem_tiny_config.h \
ENV Cleanup Step 8: Gate HAKMEM_SUPER_LOOKUP_DEBUG in header Gate HAKMEM_SUPER_LOOKUP_DEBUG environment variable behind #if !HAKMEM_BUILD_RELEASE in hakmem_super_registry.h inline function. Changes: - Wrap s_dbg initialization in conditional compilation - Release builds use constant s_dbg = 0 for complete elimination - Debug logging in hak_super_lookup() now fully compiled out in release Performance: 30.3M ops/s Larson (stable, no regression) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 01:45:45 +09:00
core/box/../front/../superslab/superslab_inline.h \
core/box/../front/../box/ss_slab_meta_box.h \
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../front/tiny_unified_cache.h \
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../front/../box/tiny_front_config_box.h \
core/box/../front/../box/../hakmem_build_flags.h \
core/box/../front/../tiny_region_id.h core/box/../front/../hakmem_tiny.h \
core/box/../front/../box/tiny_front_hot_box.h \
core/box/../front/../box/../hakmem_tiny_config.h \
core/box/../front/../box/../tiny_region_id.h \
core/box/../front/../box/../front/tiny_unified_cache.h \
core/box/../front/../box/tiny_front_cold_box.h \
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/tiny_front_config_box.h core/box/wrapper_env_box.h
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/hakmem.h:
core/hakmem_build_flags.h:
core/hakmem_config.h:
core/hakmem_features.h:
core/hakmem_internal.h:
core/hakmem_sys.h:
core/hakmem_whale.h:
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/box/ptr_type_box.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/hakmem_bigcache.h:
core/hakmem_pool.h:
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/hak_lane_classify.inc.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/hakmem_l25_pool.h:
core/hakmem_policy.h:
core/hakmem_learner.h:
core/hakmem_size_hist.h:
core/hakmem_ace.h:
core/hakmem_site_rules.h:
core/hakmem_tiny.h:
core/hakmem_trace.h:
core/hakmem_tiny_mini_mag.h:
core/hakmem_tiny_superslab.h:
core/superslab/superslab_types.h:
core/hakmem_tiny_superslab_constants.h:
core/superslab/superslab_inline.h:
core/superslab/superslab_types.h:
Remove dead code: UltraHot, RingCache, FrontC23, Class5 Hotpath (cherry-picked from 225b6fcc7, conflicts resolved)
2025-11-26 12:33:49 +09:00
core/superslab/../tiny_box_geometry.h:
core/superslab/../hakmem_tiny_superslab_constants.h:
core/superslab/../hakmem_tiny_config.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/tiny_debug_ring.h:
core/tiny_remote.h:
core/hakmem_tiny_superslab_constants.h:
core/tiny_fastcache.h:
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/hakmem_env_cache.h:
Phase12 debug: restore SUPERSLAB constants/APIs, implement Box2 drain boundary, fix tiny_fast_pop to return BASE, honor TLS SLL toggle in alloc/free fast paths, add fail-fast stubs, and quiet capacity sentinel. Update CURRENT_TASK with A/B results (SLL-off stable; SLL-on crash).
2025-11-14 01:02:00 +09:00
core/box/tiny_next_ptr_box.h:
core/hakmem_tiny_config.h:
core/tiny_nextptr.h:
Code Cleanup: Remove false positives, redundant validations, and reduce verbose logging Following the C7 stride upgrade fix (commit 23c0d9541), this commit performs comprehensive cleanup to improve code quality and reduce debug noise. ## Changes ### 1. Disable False Positive Checks (tiny_nextptr.h) - **Disabled**: NXT_MISALIGN validation block with `#if 0` - **Reason**: Produces false positives due to slab base offsets (2048, 65536) not being stride-aligned, causing all blocks to appear "misaligned" - **TODO**: Reimplement to check stride DISTANCE between consecutive blocks instead of absolute alignment to stride boundaries ### 2. Remove Redundant Geometry Validations **hakmem_tiny_refill_p0.inc.h (P0 batch refill)** - Removed 25-line CARVE_GEOMETRY_FIX validation block - Replaced with NOTE explaining redundancy - **Reason**: Stride table is now correct in tiny_block_stride_for_class(), defense-in-depth validation adds overhead without benefit **ss_legacy_backend_box.c (legacy backend)** - Removed 18-line LEGACY_FIX_GEOMETRY validation block - Replaced with NOTE explaining redundancy - **Reason**: Shared_pool validates geometry at acquisition time ### 3. Reduce Verbose Logging **hakmem_shared_pool.c (sp_fix_geometry_if_needed)** - Made SP_FIX_GEOMETRY logging conditional on `!HAKMEM_BUILD_RELEASE` - **Reason**: Geometry fixes are expected during stride upgrades, no need to log in release builds ### 4. Verification - Build: ✅ Successful (LTO warnings expected) - Test: ✅ 10K iterations (1.87M ops/s, no crashes) - NXT_MISALIGN false positives: ✅ Eliminated ## Files Modified - core/tiny_nextptr.h - Disabled false positive NXT_MISALIGN check - core/hakmem_tiny_refill_p0.inc.h - Removed redundant CARVE validation - core/box/ss_legacy_backend_box.c - Removed redundant LEGACY validation - core/hakmem_shared_pool.c - Made SP_FIX_GEOMETRY logging debug-only ## Impact - **Code clarity**: Removed 43 lines of redundant validation code - **Debug noise**: Reduced false positive diagnostics - **Performance**: Eliminated overhead from redundant geometry checks - **Maintainability**: Single source of truth for geometry validation 🧹 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:00:24 +09:00
core/tiny_region_id.h:
core/tiny_box_geometry.h:
core/ptr_track.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/hakmem_super_registry.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/ss_addr_map_box.h:
core/box/../hakmem_build_flags.h:
Cleanup: Add tiny_debug_api.h to eliminate guard/failfast implicit warnings Created central header for debug instrumentation API to fix implicit function declaration warnings across the codebase. Changes: 1. Created core/tiny_debug_api.h - Declares guard system API (3 functions) - Declares failfast debugging API (3 functions) - Uses forward declarations for SuperSlab/TinySlabMeta 2. Updated 3 files to include tiny_debug_api.h: - core/tiny_region_id.h (removed inline externs) - core/hakmem_tiny_tls_ops.h - core/tiny_superslab_alloc.inc.h Warnings eliminated (6 of 11 total): ✅ tiny_guard_is_enabled() ✅ tiny_guard_on_alloc() ✅ tiny_guard_on_invalid() ✅ tiny_failfast_log() ✅ tiny_failfast_abort_ptr() ✅ tiny_refill_failfast_level() Remaining warnings (deferred to P1): - ss_active_add (2 occurrences) - expand_superslab_head - hkm_ace_set_tls_capacity - smallmid_backend_free Impact: - Cleaner build output - Better type safety for debug functions - No behavior changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 06:47:13 +09:00
core/tiny_debug_api.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/hakmem_elo.h:
core/hakmem_ace_stats.h:
core/hakmem_batch.h:
core/hakmem_evo.h:
core/hakmem_debug.h:
core/hakmem_prof.h:
core/hakmem_syscall.h:
core/hakmem_ace_controller.h:
core/hakmem_ace_metrics.h:
core/hakmem_ace_ucb1.h:
Phase 21-1-B: Ring cache Alloc/Free 統合 - C2/C3 hot path integration **統合内容**: - Alloc path (tiny_alloc_fast.inc.h): Ring pop → HeapV2/UltraHot/SLL fallback - Free path (tiny_free_fast_v2.inc.h): Ring push → HeapV2/SLL fallback - Lazy init: 最初の alloc/free 時に自動初期化（thread-safe） **設計**: - Lazy init パターン（ENV control と同様） - ring_cache_pop/push 内で slots == NULL チェック → ring_cache_init() 呼び出し - Include 構造: ファイルトップレベルに #include 追加（関数内 include 禁止） **Makefile 修正**: - TINY_BENCH_OBJS_BASE に core/front/tiny_ring_cache.o 追加 - Link エラー修正: 4箇所の object list に追加 **動作確認**: - Ring OFF (default): 83K ops/s (1K iterations) ✅ - Ring ON (HAKMEM_TINY_HOT_RING_ENABLE=1): 78K ops/s ✅ - クラッシュなし、正常動作確認 **次のステップ**: Phase 21-1-C (Refill/Cascade 実装)
2025-11-16 07:51:37 +09:00
core/box/bench_fast_box.h:
Box TLS-SLL: fix splice head normalization and remove false misalignment guard; add header-aware linear link instrumentation; log splice details in debug.\n\n- Normalize head before publishing to TLS SLL (avoid user-ptr head)\n- Remove size-mod alignment guard (stride!=size); keep small-ptr fail-fast only\n- Drop heuristic base normalization to avoid corrupting base\n- Add [LINEAR_LINK]/[SPLICE_LINK]/[SPLICE_SET_HEAD] debug logs (debug-only)\n- Verified debug build on bench_fixed_size_hakmem with visible carve/splice traces
2025-11-11 00:02:24 +09:00
core/ptr_trace.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/box/hak_exit_debug.inc.h:
core/box/hak_kpi_util.inc.h:
core/box/hak_core_init.inc.h:
core/hakmem_phase7_config.h:
Phase 21-1-B: Ring cache Alloc/Free 統合 - C2/C3 hot path integration **統合内容**: - Alloc path (tiny_alloc_fast.inc.h): Ring pop → HeapV2/UltraHot/SLL fallback - Free path (tiny_free_fast_v2.inc.h): Ring push → HeapV2/SLL fallback - Lazy init: 最初の alloc/free 時に自動初期化（thread-safe） **設計**: - Lazy init パターン（ENV control と同様） - ring_cache_pop/push 内で slots == NULL チェック → ring_cache_init() 呼び出し - Include 構造: ファイルトップレベルに #include 追加（関数内 include 禁止） **Makefile 修正**: - TINY_BENCH_OBJS_BASE に core/front/tiny_ring_cache.o 追加 - Link エラー修正: 4箇所の object list に追加 **動作確認**: - Ring OFF (default): 83K ops/s (1K iterations) ✅ - Ring ON (HAKMEM_TINY_HOT_RING_ENABLE=1): 78K ops/s ✅ - クラッシュなし、正常動作確認 **次のステップ**: Phase 21-1-C (Refill/Cascade 実装)
2025-11-16 07:51:37 +09:00
core/box/ss_hot_prewarm_box.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/box/hak_alloc_api.inc.h:
Phase 17-1: Small-Mid Allocator - TLS Frontend Cache (結果: ±0.3%, 層分離成功) Summary: ======== Phase 17-1 implements Small-Mid allocator as TLS frontend cache with Tiny backend delegation. Result: Clean layer separation achieved with minimal overhead (±0.3%), but no performance gain. Conclusion: Frontend-only approach is dead end. Phase 17-2 (dedicated backend) required for 2-3x target. Implementation: =============== 1. Small-Mid TLS frontend (256B/512B/1KB - 3 classes) - TLS freelist (32/24/16 capacity) - Backend delegation to Tiny C5/C6/C7 - Header conversion (0xa0 → 0xb0) 2. Auto-adjust Tiny boundary - When Small-Mid ON: Tiny auto-limits to C0-C5 (0-255B) - When Small-Mid OFF: Tiny default C0-C7 (0-1023B) - Prevents routing conflict 3. Routing order fix - Small-Mid BEFORE Tiny (critical for proper execution) - Fall-through on TLS miss Files Modified: =============== - core/hakmem_smallmid.h/c: TLS freelist + backend delegation - core/hakmem_tiny.c: tiny_get_max_size() auto-adjust - core/box/hak_alloc_api.inc.h: Routing order (Small-Mid → Tiny) - CURRENT_TASK.md: Phase 17-1 results + Phase 17-2 plan A/B Benchmark Results: ====================== | Size | Config A (OFF) | Config B (ON) | Delta | % Change | |--------|----------------|---------------|----------|----------| | 256B | 5.87M ops/s | 6.06M ops/s | +191K | +3.3% | | 512B | 6.02M ops/s | 5.91M ops/s | -112K | -1.9% | | 1024B | 5.58M ops/s | 5.54M ops/s | -35K | -0.6% | | Overall| 5.82M ops/s | 5.84M ops/s | +20K | +0.3% | Analysis: ========= ✅ SUCCESS: Clean layer separation (Small-Mid ↔ Tiny coexist) ✅ SUCCESS: Minimal overhead (±0.3% = measurement noise) ❌ FAIL: No performance gain (target was 2-4x) Root Cause: ----------- - Delegation overhead = TLS savings (net gain ≈ 0 instructions) - Small-Mid TLS alloc: ~3-5 instructions - Tiny backend delegation: ~3-5 instructions - Header conversion: ~2 instructions - No batching: 1:1 delegation to Tiny (no refill amortization) Lessons Learned: ================ - Frontend-only approach ineffective (backend calls not reduced) - Dedicated backend essential for meaningful improvement - Clean separation achieved = solid foundation for Phase 17-2 Next Steps (Phase 17-2): ======================== - Dedicated Small-Mid SuperSlab backend (separate from Tiny) - TLS batch refill (8-16 blocks per refill) - Optimized 0xb0 header fast path (no delegation) - Target: 12-15M ops/s (2.0-2.6x improvement) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 02:37:24 +09:00
core/box/../hakmem_tiny.h:
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/../hakmem_pool.h:
Phase 17-1: Small-Mid Allocator - TLS Frontend Cache (結果: ±0.3%, 層分離成功) Summary: ======== Phase 17-1 implements Small-Mid allocator as TLS frontend cache with Tiny backend delegation. Result: Clean layer separation achieved with minimal overhead (±0.3%), but no performance gain. Conclusion: Frontend-only approach is dead end. Phase 17-2 (dedicated backend) required for 2-3x target. Implementation: =============== 1. Small-Mid TLS frontend (256B/512B/1KB - 3 classes) - TLS freelist (32/24/16 capacity) - Backend delegation to Tiny C5/C6/C7 - Header conversion (0xa0 → 0xb0) 2. Auto-adjust Tiny boundary - When Small-Mid ON: Tiny auto-limits to C0-C5 (0-255B) - When Small-Mid OFF: Tiny default C0-C7 (0-1023B) - Prevents routing conflict 3. Routing order fix - Small-Mid BEFORE Tiny (critical for proper execution) - Fall-through on TLS miss Files Modified: =============== - core/hakmem_smallmid.h/c: TLS freelist + backend delegation - core/hakmem_tiny.c: tiny_get_max_size() auto-adjust - core/box/hak_alloc_api.inc.h: Routing order (Small-Mid → Tiny) - CURRENT_TASK.md: Phase 17-1 results + Phase 17-2 plan A/B Benchmark Results: ====================== | Size | Config A (OFF) | Config B (ON) | Delta | % Change | |--------|----------------|---------------|----------|----------| | 256B | 5.87M ops/s | 6.06M ops/s | +191K | +3.3% | | 512B | 6.02M ops/s | 5.91M ops/s | -112K | -1.9% | | 1024B | 5.58M ops/s | 5.54M ops/s | -35K | -0.6% | | Overall| 5.82M ops/s | 5.84M ops/s | +20K | +0.3% | Analysis: ========= ✅ SUCCESS: Clean layer separation (Small-Mid ↔ Tiny coexist) ✅ SUCCESS: Minimal overhead (±0.3% = measurement noise) ❌ FAIL: No performance gain (target was 2-4x) Root Cause: ----------- - Delegation overhead = TLS savings (net gain ≈ 0 instructions) - Small-Mid TLS alloc: ~3-5 instructions - Tiny backend delegation: ~3-5 instructions - Header conversion: ~2 instructions - No batching: 1:1 delegation to Tiny (no refill amortization) Lessons Learned: ================ - Frontend-only approach ineffective (backend calls not reduced) - Dedicated backend essential for meaningful improvement - Clean separation achieved = solid foundation for Phase 17-2 Next Steps (Phase 17-2): ======================== - Dedicated Small-Mid SuperSlab backend (separate from Tiny) - TLS batch refill (8-16 blocks per refill) - Optimized 0xb0 header fast path (no delegation) - Target: 12-15M ops/s (2.0-2.6x improvement) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 02:37:24 +09:00
core/box/../hakmem_smallmid.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/mid_large_config_box.h:
core/box/../hakmem_config.h:
core/box/../hakmem_features.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/box/hak_free_api.inc.h:
core/hakmem_tiny_superslab.h:
core/box/../tiny_free_fast_v2.inc.h:
core/box/../tiny_region_id.h:
core/box/../hakmem_build_flags.h:
Tiny: Enable P0 batch refill by default + docs and task update Summary - Default P0 ON: Build-time HAKMEM_TINY_P0_BATCH_REFILL=1 remains; runtime gate now defaults to ON (HAKMEM_TINY_P0_ENABLE unset or not '0'). Kill switch preserved via HAKMEM_TINY_P0_DISABLE=1. - Fix critical bug: After freelist→SLL batch splice, increment TinySlabMeta::used by 'from_freelist' to mirror non-P0 behavior (prevents under-accounting and follow-on carve invariants from breaking). - Add low-overhead A/B toggles for triage: HAKMEM_TINY_P0_NO_DRAIN (skip remote drain), HAKMEM_TINY_P0_LOG (emit [P0_COUNTER_OK/MISMATCH] based on total_active_blocks delta). - Keep linear carve fail-fast guards across simple/general/TLS-bump paths. Perf (1T, 100k×256B) - P0 OFF: ~2.73M ops/s (stable) - P0 ON (no drain): ~2.45M ops/s - P0 ON (normal drain): ~2.76M ops/s (fastest) Known - Rare [P0_COUNTER_MISMATCH] warnings persist (non-fatal). Continue auditing active/used balance around batch freelist splice and remote drain splice. Docs - Add docs/TINY_P0_BATCH_REFILL.md (runtime switches, behavior, perf notes). - Update CURRENT_TASK.md with Tiny P0 status (default ON) and next steps.
2025-11-09 22:12:34 +09:00
core/box/../hakmem_tiny_config.h:
Box TLS-SLL + free boundary hardening: normalize C0–C6 to base (ptr-1) at free boundary; route all caches/freelists via base; replace remaining g_tls_sll_head direct writes with Box API (tls_sll_push/splice) in refill/magazine/ultra; keep C7 excluded. Fixes rbp=0xa0 free crash by preventing header overwrite and centralizing TLS-SLL invariants.
2025-11-10 16:48:20 +09:00
core/box/../box/tls_sll_box.h:
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/box/../box/../hakmem_internal.h:
Box TLS-SLL + free boundary hardening: normalize C0–C6 to base (ptr-1) at free boundary; route all caches/freelists via base; replace remaining g_tls_sll_head direct writes with Box API (tls_sll_push/splice) in refill/magazine/ultra; keep C7 excluded. Fixes rbp=0xa0 free crash by preventing header overwrite and centralizing TLS-SLL invariants.
2025-11-10 16:48:20 +09:00
core/box/../box/../hakmem_tiny_config.h:
Box TLS-SLL: fix splice head normalization and remove false misalignment guard; add header-aware linear link instrumentation; log splice details in debug.\n\n- Normalize head before publishing to TLS SLL (avoid user-ptr head)\n- Remove size-mod alignment guard (stride!=size); keep small-ptr fail-fast only\n- Drop heuristic base normalization to avoid corrupting base\n- Add [LINEAR_LINK]/[SPLICE_LINK]/[SPLICE_SET_HEAD] debug logs (debug-only)\n- Verified debug build on bench_fixed_size_hakmem with visible carve/splice traces
2025-11-11 00:02:24 +09:00
core/box/../box/../hakmem_build_flags.h:
Cleanup: Consolidate debug ENV vars to HAKMEM_DEBUG_LEVEL Integrated 4 new debug environment variables added during bug fixes into the existing unified HAKMEM_DEBUG_LEVEL system (expanded to 0-5 levels). Changes: 1. Expanded HAKMEM_DEBUG_LEVEL from 0-3 to 0-5 levels: - 0 = OFF (production) - 1 = ERROR (critical errors) - 2 = WARN (warnings) - 3 = INFO (allocation paths, header validation, stats) - 4 = DEBUG (guard instrumentation, failfast) - 5 = TRACE (verbose tracing) 2. Integrated 4 environment variables: - HAKMEM_ALLOC_PATH_TRACE → HAKMEM_DEBUG_LEVEL >= 3 (INFO) - HAKMEM_TINY_SLL_VALIDATE_HDR → HAKMEM_DEBUG_LEVEL >= 3 (INFO) - HAKMEM_TINY_REFILL_FAILFAST → HAKMEM_DEBUG_LEVEL >= 4 (DEBUG) - HAKMEM_TINY_GUARD → HAKMEM_DEBUG_LEVEL >= 4 (DEBUG) 3. Kept 2 special-purpose variables (fine-grained control): - HAKMEM_TINY_GUARD_CLASS (target class for guard) - HAKMEM_TINY_GUARD_MAX (max guard events) 4. Backward compatibility: - Legacy ENV vars still work via hak_debug_check_level() - New code uses unified system - No behavior changes for existing users Updated files: - core/hakmem_debug_master.h (level 0-5 expansion) - core/hakmem_tiny_superslab_internal.h (alloc path trace) - core/box/tls_sll_box.h (header validation) - core/tiny_failfast.c (failfast level) - core/tiny_refill_opt.h (failfast guard) - core/hakmem_tiny_ace_guard_box.inc (guard enable) - core/hakmem_tiny.c (include hakmem_debug_master.h) Impact: - Simpler debug control: HAKMEM_DEBUG_LEVEL=3 instead of 4 separate ENVs - Easier to discover/use - Consistent debug levels across codebase - Reduces ENV variable proliferation (43+ vars surveyed) Future work: - Consolidate remaining 39+ debug variables (documented in survey) - Gradual migration over 2-3 releases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 06:57:03 +09:00
core/box/../box/../hakmem_debug_master.h:
Phase E3-FINAL: Fix Box API offset bugs - ALL classes now use correct offsets ## Root Cause Analysis (GPT5) **Physical Layout Constraints**: - Class 0: 8B = [1B header][7B payload] → offset 1 = 9B needed = ❌ IMPOSSIBLE - Class 1-6: >=16B = [1B header][15B+ payload] → offset 1 = ✅ POSSIBLE - Class 7: 1KB → offset 0 (compatibility) **Correct Specification**: - HAKMEM_TINY_HEADER_CLASSIDX != 0: - Class 0, 7: next at offset 0 (overwrites header when on freelist) - Class 1-6: next at offset 1 (after header) - HAKMEM_TINY_HEADER_CLASSIDX == 0: - All classes: next at offset 0 **Previous Bug**: - Attempted "ALL classes offset 1" unification - Class 0 with offset 1 caused immediate SEGV (9B > 8B block size) - Mixed 2-arg/3-arg API caused confusion ## Fixes Applied ### 1. Restored 3-Argument Box API (core/box/tiny_next_ptr_box.h) ```c // Correct signatures void tiny_next_write(int class_idx, void* base, void* next_value) void* tiny_next_read(int class_idx, const void* base) // Correct offset calculation size_t offset = (class_idx == 0 || class_idx == 7) ? 0 : 1; ``` ### 2. Updated 123+ Call Sites Across 34 Files - hakmem_tiny_hot_pop_v4.inc.h (4 locations) - hakmem_tiny_fastcache.inc.h (3 locations) - hakmem_tiny_tls_list.h (12 locations) - superslab_inline.h (5 locations) - tiny_fastcache.h (3 locations) - ptr_trace.h (macro definitions) - tls_sll_box.h (2 locations) - + 27 additional files Pattern: `tiny_next_read(base)` → `tiny_next_read(class_idx, base)` Pattern: `tiny_next_write(base, next)` → `tiny_next_write(class_idx, base, next)` ### 3. Added Sentinel Detection Guards - tiny_fast_push(): Block nodes with sentinel in ptr or ptr->next - tls_list_push(): Block nodes with sentinel in ptr or ptr->next - Defense-in-depth against remote free sentinel leakage ## Verification (GPT5 Report) **Test Command**: `./out/release/bench_random_mixed_hakmem --iterations=70000` **Results**: - ✅ Main loop completed successfully - ✅ Drain phase completed successfully - ✅ NO SEGV (previous crash at iteration 66151 is FIXED) - ℹ️ Final log: "tiny_alloc(1024) failed" is normal fallback to Mid/ACE layers **Analysis**: - Class 0 immediate SEGV: ✅ RESOLVED (correct offset 0 now used) - 66K iteration crash: ✅ RESOLVED (offset consistency fixed) - Box API conflicts: ✅ RESOLVED (unified 3-arg API) ## Technical Details ### Offset Logic Justification ``` Class 0: 8B block → next pointer (8B) fits ONLY at offset 0 Class 1: 16B block → next pointer (8B) fits at offset 1 (after 1B header) Class 2: 32B block → next pointer (8B) fits at offset 1 ... Class 6: 512B block → next pointer (8B) fits at offset 1 Class 7: 1024B block → offset 0 for legacy compatibility ``` ### Files Modified (Summary) - Core API: `box/tiny_next_ptr_box.h` - Hot paths: `hakmem_tiny_hot_pop*.inc.h`, `tiny_fastcache.h` - TLS layers: `hakmem_tiny_tls_list.h`, `hakmem_tiny_tls_ops.h` - SuperSlab: `superslab_inline.h`, `tiny_superslab_*.inc.h` - Refill: `hakmem_tiny_refill.inc.h`, `tiny_refill_opt.h` - Free paths: `tiny_free_magazine.inc.h`, `tiny_superslab_free.inc.h` - Documentation: Multiple Phase E3 reports ## Remaining Work None for Box API offset bugs - all structural issues resolved. Future enhancements (non-critical): - Periodic `grep -R '*(void**)' core/` to detect direct pointer access violations - Enforce Box API usage via static analysis - Document offset rationale in architecture docs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 06:50:20 +09:00
core/box/../box/../tiny_remote.h:
tiny: fix TLS list next_off scope; default TLS_LIST=1; add sentinel guards; header-aware TLS ops; release quiet for benches
2025-11-11 10:00:36 +09:00
core/box/../box/../tiny_region_id.h:
Add Box I (Integrity), Box E (Expansion), and comprehensive P0 debugging infrastructure ## Major Additions ### 1. Box I: Integrity Verification System (NEW - 703 lines) - Files: core/box/integrity_box.h (267 lines), core/box/integrity_box.c (436 lines) - Purpose: Unified integrity checking across all HAKMEM subsystems - Features: * 4-level integrity checking (0-4, compile-time controlled) * Priority 1: TLS array bounds validation * Priority 2: Freelist pointer validation * Priority 3: TLS canary monitoring * Priority ALPHA: Slab metadata invariant checking (5 invariants) * Atomic statistics tracking (thread-safe) * Beautiful BOX_BOUNDARY design pattern ### 2. Box E: SuperSlab Expansion System (COMPLETE) - Files: core/box/superslab_expansion_box.h, core/box/superslab_expansion_box.c - Purpose: Safe SuperSlab expansion with TLS state guarantee - Features: * Immediate slab 0 binding after expansion * TLS state snapshot and restoration * Design by Contract (pre/post-conditions, invariants) * Thread-safe with mutex protection ### 3. Comprehensive Integrity Checking System - File: core/hakmem_tiny_integrity.h (NEW) - Unified validation functions for all allocator subsystems - Uninitialized memory pattern detection (0xa2, 0xcc, 0xdd, 0xfe) - Pointer range validation (null-page, kernel-space) ### 4. P0 Bug Investigation - Root Cause Identified **Bug**: SEGV at iteration 28440 (deterministic with seed 42) **Pattern**: 0xa2a2a2a2a2a2a2a2 (uninitialized/ASan poisoning) **Location**: TLS SLL (Single-Linked List) cache layer **Root Cause**: Race condition or use-after-free in TLS list management (class 0) **Detection**: Box I successfully caught invalid pointer at exact crash point ### 5. Defensive Improvements - Defensive memset in SuperSlab allocation (all metadata arrays) - Enhanced pointer validation with pattern detection - BOX_BOUNDARY markers throughout codebase (beautiful modular design) - 5 metadata invariant checks in allocation/free/refill paths ## Integration Points - Modified 13 files with Box I/E integration - Added 10+ BOX_BOUNDARY markers - 5 critical integrity check points in P0 refill path ## Test Results (100K iterations) - Baseline: 7.22M ops/s - Hotpath ON: 8.98M ops/s (+24% improvement ✓) - P0 Bug: Still crashes at 28440 iterations (TLS SLL race condition) - Root cause: Identified but not yet fixed (requires deeper investigation) ## Performance - Box I overhead: Zero in release builds (HAKMEM_INTEGRITY_LEVEL=0) - Debug builds: Full validation enabled (HAKMEM_INTEGRITY_LEVEL=4) - Beautiful modular design maintains clean separation of concerns ## Known Issues - P0 Bug at 28440 iterations: Race condition in TLS SLL cache (class 0) - Cause: Use-after-free or race in remote free draining - Next step: Valgrind investigation to pinpoint exact corruption location ## Code Quality - Total new code: ~1400 lines (Box I + Box E + integrity system) - Design: Beautiful Box Theory with clear boundaries - Modularity: Complete separation of concerns - Documentation: Comprehensive inline comments and BOX_BOUNDARY markers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-12 02:45:00 +09:00
core/box/../box/../hakmem_tiny_integrity.h:
core/box/../box/../hakmem_tiny.h:
Phase E3-FINAL: Fix Box API offset bugs - ALL classes now use correct offsets ## Root Cause Analysis (GPT5) **Physical Layout Constraints**: - Class 0: 8B = [1B header][7B payload] → offset 1 = 9B needed = ❌ IMPOSSIBLE - Class 1-6: >=16B = [1B header][15B+ payload] → offset 1 = ✅ POSSIBLE - Class 7: 1KB → offset 0 (compatibility) **Correct Specification**: - HAKMEM_TINY_HEADER_CLASSIDX != 0: - Class 0, 7: next at offset 0 (overwrites header when on freelist) - Class 1-6: next at offset 1 (after header) - HAKMEM_TINY_HEADER_CLASSIDX == 0: - All classes: next at offset 0 **Previous Bug**: - Attempted "ALL classes offset 1" unification - Class 0 with offset 1 caused immediate SEGV (9B > 8B block size) - Mixed 2-arg/3-arg API caused confusion ## Fixes Applied ### 1. Restored 3-Argument Box API (core/box/tiny_next_ptr_box.h) ```c // Correct signatures void tiny_next_write(int class_idx, void* base, void* next_value) void* tiny_next_read(int class_idx, const void* base) // Correct offset calculation size_t offset = (class_idx == 0 || class_idx == 7) ? 0 : 1; ``` ### 2. Updated 123+ Call Sites Across 34 Files - hakmem_tiny_hot_pop_v4.inc.h (4 locations) - hakmem_tiny_fastcache.inc.h (3 locations) - hakmem_tiny_tls_list.h (12 locations) - superslab_inline.h (5 locations) - tiny_fastcache.h (3 locations) - ptr_trace.h (macro definitions) - tls_sll_box.h (2 locations) - + 27 additional files Pattern: `tiny_next_read(base)` → `tiny_next_read(class_idx, base)` Pattern: `tiny_next_write(base, next)` → `tiny_next_write(class_idx, base, next)` ### 3. Added Sentinel Detection Guards - tiny_fast_push(): Block nodes with sentinel in ptr or ptr->next - tls_list_push(): Block nodes with sentinel in ptr or ptr->next - Defense-in-depth against remote free sentinel leakage ## Verification (GPT5 Report) **Test Command**: `./out/release/bench_random_mixed_hakmem --iterations=70000` **Results**: - ✅ Main loop completed successfully - ✅ Drain phase completed successfully - ✅ NO SEGV (previous crash at iteration 66151 is FIXED) - ℹ️ Final log: "tiny_alloc(1024) failed" is normal fallback to Mid/ACE layers **Analysis**: - Class 0 immediate SEGV: ✅ RESOLVED (correct offset 0 now used) - 66K iteration crash: ✅ RESOLVED (offset consistency fixed) - Box API conflicts: ✅ RESOLVED (unified 3-arg API) ## Technical Details ### Offset Logic Justification ``` Class 0: 8B block → next pointer (8B) fits ONLY at offset 0 Class 1: 16B block → next pointer (8B) fits at offset 1 (after 1B header) Class 2: 32B block → next pointer (8B) fits at offset 1 ... Class 6: 512B block → next pointer (8B) fits at offset 1 Class 7: 1024B block → offset 0 for legacy compatibility ``` ### Files Modified (Summary) - Core API: `box/tiny_next_ptr_box.h` - Hot paths: `hakmem_tiny_hot_pop*.inc.h`, `tiny_fastcache.h` - TLS layers: `hakmem_tiny_tls_list.h`, `hakmem_tiny_tls_ops.h` - SuperSlab: `superslab_inline.h`, `tiny_superslab_*.inc.h` - Refill: `hakmem_tiny_refill.inc.h`, `tiny_refill_opt.h` - Free paths: `tiny_free_magazine.inc.h`, `tiny_superslab_free.inc.h` - Documentation: Multiple Phase E3 reports ## Remaining Work None for Box API offset bugs - all structural issues resolved. Future enhancements (non-critical): - Periodic `grep -R '*(void**)' core/` to detect direct pointer access violations - Enforce Box API usage via static analysis - Document offset rationale in architecture docs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 06:50:20 +09:00
core/box/../box/../ptr_track.h:
Front-Direct implementation: SS→FC direct refill + SLL complete bypass ## Summary Implemented Front-Direct architecture with complete SLL bypass: - Direct SuperSlab → FastCache refill (1-hop, bypasses SLL) - SLL-free allocation/free paths when Front-Direct enabled - Legacy path sealing (SLL inline opt-in, SFC cascade ENV-only) ## New Modules - core/refill/ss_refill_fc.h (236 lines): Standard SS→FC refill entry point - Remote drain → Freelist → Carve priority - Header restoration for C1-C6 (NOT C0/C7) - ENV: HAKMEM_TINY_P0_DRAIN_THRESH, HAKMEM_TINY_P0_NO_DRAIN - core/front/fast_cache.h: FastCache (L1) type definition - core/front/quick_slot.h: QuickSlot (L0) type definition ## Allocation Path (core/tiny_alloc_fast.inc.h) - Added s_front_direct_alloc TLS flag (lazy ENV check) - SLL pop guarded by: g_tls_sll_enable && !s_front_direct_alloc - Refill dispatch: - Front-Direct: ss_refill_fc_fill() → fastcache_pop() (1-hop) - Legacy: sll_refill_batch_from_ss() → SLL → FC (2-hop, A/B only) - SLL inline pop sealed (requires HAKMEM_TINY_INLINE_SLL=1 opt-in) ## Free Path (core/hakmem_tiny_free.inc, core/hakmem_tiny_fastcache.inc.h) - FC priority: Try fastcache_push() first (same-thread free) - tiny_fast_push() bypass: Returns 0 when s_front_direct_free || !g_tls_sll_enable - Fallback: Magazine/slow path (safe, bypasses SLL) ## Legacy Sealing - SFC cascade: Default OFF (ENV-only via HAKMEM_TINY_SFC_CASCADE=1) - Deleted: core/hakmem_tiny_free.inc.bak, core/pool_refill_legacy.c.bak - Documentation: ss_refill_fc_fill() promoted as CANONICAL refill entry ## ENV Controls - HAKMEM_TINY_FRONT_DIRECT=1: Enable Front-Direct (SS→FC direct) - HAKMEM_TINY_P0_DIRECT_FC_ALL=1: Same as above (alt name) - HAKMEM_TINY_REFILL_BATCH=1: Enable batch refill (also enables Front-Direct) - HAKMEM_TINY_SFC_CASCADE=1: Enable SFC cascade (default OFF) - HAKMEM_TINY_INLINE_SLL=1: Enable inline SLL pop (default OFF, requires AGGRESSIVE_INLINE) ## Benchmarks (Front-Direct Enabled) ```bash ENV: HAKMEM_BENCH_FAST_FRONT=1 HAKMEM_TINY_FRONT_DIRECT=1 HAKMEM_TINY_REFILL_BATCH=1 HAKMEM_TINY_P0_DIRECT_FC_ALL=1 HAKMEM_TINY_REFILL_COUNT_HOT=256 HAKMEM_TINY_REFILL_COUNT_MID=96 HAKMEM_TINY_BUMP_CHUNK=256 bench_random_mixed (16-1040B random, 200K iter): 256 slots: 1.44M ops/s (STABLE, 0 SEGV) 128 slots: 1.44M ops/s (STABLE, 0 SEGV) bench_fixed_size (fixed size, 200K iter): 256B: 4.06M ops/s (has debug logs, expected >10M without logs) 128B: Similar (debug logs affect) ``` ## Verification - TRACE_RING test (10K iter): **0 SLL events** detected ✅ - Complete SLL bypass confirmed when Front-Direct=1 - Stable execution: 200K iterations × multiple sizes, 0 SEGV ## Next Steps - Disable debug logs in hak_alloc_api.inc.h (call_num 14250-14280 range) - Re-benchmark with clean Release build (target: 10-15M ops/s) - 128/256B shortcut path optimization (FC hit rate improvement) Co-Authored-By: ChatGPT <chatgpt@openai.com> Suggested-By: ultrathink
2025-11-14 05:41:49 +09:00
core/box/../box/../tiny_debug_ring.h:
Code Cleanup: Remove false positives, redundant validations, and reduce verbose logging Following the C7 stride upgrade fix (commit 23c0d9541), this commit performs comprehensive cleanup to improve code quality and reduce debug noise. ## Changes ### 1. Disable False Positive Checks (tiny_nextptr.h) - **Disabled**: NXT_MISALIGN validation block with `#if 0` - **Reason**: Produces false positives due to slab base offsets (2048, 65536) not being stride-aligned, causing all blocks to appear "misaligned" - **TODO**: Reimplement to check stride DISTANCE between consecutive blocks instead of absolute alignment to stride boundaries ### 2. Remove Redundant Geometry Validations **hakmem_tiny_refill_p0.inc.h (P0 batch refill)** - Removed 25-line CARVE_GEOMETRY_FIX validation block - Replaced with NOTE explaining redundancy - **Reason**: Stride table is now correct in tiny_block_stride_for_class(), defense-in-depth validation adds overhead without benefit **ss_legacy_backend_box.c (legacy backend)** - Removed 18-line LEGACY_FIX_GEOMETRY validation block - Replaced with NOTE explaining redundancy - **Reason**: Shared_pool validates geometry at acquisition time ### 3. Reduce Verbose Logging **hakmem_shared_pool.c (sp_fix_geometry_if_needed)** - Made SP_FIX_GEOMETRY logging conditional on `!HAKMEM_BUILD_RELEASE` - **Reason**: Geometry fixes are expected during stride upgrades, no need to log in release builds ### 4. Verification - Build: ✅ Successful (LTO warnings expected) - Test: ✅ 10K iterations (1.87M ops/s, no crashes) - NXT_MISALIGN false positives: ✅ Eliminated ## Files Modified - core/tiny_nextptr.h - Disabled false positive NXT_MISALIGN check - core/hakmem_tiny_refill_p0.inc.h - Removed redundant CARVE validation - core/box/ss_legacy_backend_box.c - Removed redundant LEGACY validation - core/hakmem_shared_pool.c - Made SP_FIX_GEOMETRY logging debug-only ## Impact - **Code clarity**: Removed 43 lines of redundant validation code - **Debug noise**: Reduced false positive diagnostics - **Performance**: Eliminated overhead from redundant geometry checks - **Maintainability**: Single source of truth for geometry validation 🧹 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 23:00:24 +09:00
core/box/../box/../superslab/superslab_inline.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../box/tiny_header_box.h:
core/box/../box/../tiny_nextptr.h:
Fix drain box compilation: Use pthread_self() directly Issue: - tiny_self_u32() is static inline, cannot be linked from drain box - Link error: undefined reference to 'tiny_self_u32' Fix: - Use pthread_self() directly like hakmem_tiny_superslab.c:917 - Added <pthread.h> include - Changed extern declaration from size_t to const size_t 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 07:10:46 +09:00
core/box/../box/tls_sll_drain_box.h:
core/box/../box/tls_sll_box.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../box/slab_recycling_box.h:
core/box/../box/../hakmem_tiny_superslab.h:
core/box/../box/../hakmem_shared_pool.h:
core/box/../box/../superslab/superslab_types.h:
core/box/../box/ss_hot_cold_box.h:
core/box/../box/../superslab/superslab_types.h:
Fix drain box compilation: Use pthread_self() directly Issue: - tiny_self_u32() is static inline, cannot be linked from drain box - Link error: undefined reference to 'tiny_self_u32' Fix: - Use pthread_self() directly like hakmem_tiny_superslab.c:917 - Added <pthread.h> include - Changed extern declaration from size_t to const size_t 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-14 07:10:46 +09:00
core/box/../box/free_local_box.h:
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/box/../box/ptr_type_box.h:
core/box/../box/free_publish_box.h:
core/hakmem_tiny.h:
core/tiny_region_id.h:
Add Box I (Integrity), Box E (Expansion), and comprehensive P0 debugging infrastructure ## Major Additions ### 1. Box I: Integrity Verification System (NEW - 703 lines) - Files: core/box/integrity_box.h (267 lines), core/box/integrity_box.c (436 lines) - Purpose: Unified integrity checking across all HAKMEM subsystems - Features: * 4-level integrity checking (0-4, compile-time controlled) * Priority 1: TLS array bounds validation * Priority 2: Freelist pointer validation * Priority 3: TLS canary monitoring * Priority ALPHA: Slab metadata invariant checking (5 invariants) * Atomic statistics tracking (thread-safe) * Beautiful BOX_BOUNDARY design pattern ### 2. Box E: SuperSlab Expansion System (COMPLETE) - Files: core/box/superslab_expansion_box.h, core/box/superslab_expansion_box.c - Purpose: Safe SuperSlab expansion with TLS state guarantee - Features: * Immediate slab 0 binding after expansion * TLS state snapshot and restoration * Design by Contract (pre/post-conditions, invariants) * Thread-safe with mutex protection ### 3. Comprehensive Integrity Checking System - File: core/hakmem_tiny_integrity.h (NEW) - Unified validation functions for all allocator subsystems - Uninitialized memory pattern detection (0xa2, 0xcc, 0xdd, 0xfe) - Pointer range validation (null-page, kernel-space) ### 4. P0 Bug Investigation - Root Cause Identified **Bug**: SEGV at iteration 28440 (deterministic with seed 42) **Pattern**: 0xa2a2a2a2a2a2a2a2 (uninitialized/ASan poisoning) **Location**: TLS SLL (Single-Linked List) cache layer **Root Cause**: Race condition or use-after-free in TLS list management (class 0) **Detection**: Box I successfully caught invalid pointer at exact crash point ### 5. Defensive Improvements - Defensive memset in SuperSlab allocation (all metadata arrays) - Enhanced pointer validation with pattern detection - BOX_BOUNDARY markers throughout codebase (beautiful modular design) - 5 metadata invariant checks in allocation/free/refill paths ## Integration Points - Modified 13 files with Box I/E integration - Added 10+ BOX_BOUNDARY markers - 5 critical integrity check points in P0 refill path ## Test Results (100K iterations) - Baseline: 7.22M ops/s - Hotpath ON: 8.98M ops/s (+24% improvement ✓) - P0 Bug: Still crashes at 28440 iterations (TLS SLL race condition) - Root cause: Identified but not yet fixed (requires deeper investigation) ## Performance - Box I overhead: Zero in release builds (HAKMEM_INTEGRITY_LEVEL=0) - Debug builds: Full validation enabled (HAKMEM_INTEGRITY_LEVEL=4) - Beautiful modular design maintains clean separation of concerns ## Known Issues - P0 Bug at 28440 iterations: Race condition in TLS SLL cache (class 0) - Cause: Use-after-free or race in remote free draining - Next step: Valgrind investigation to pinpoint exact corruption location ## Code Quality - Total new code: ~1400 lines (Box I + Box E + integrity system) - Design: Beautiful Box Theory with clear boundaries - Modularity: Complete separation of concerns - Documentation: Comprehensive inline comments and BOX_BOUNDARY markers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-12 02:45:00 +09:00
core/box/../hakmem_tiny_integrity.h:
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/../hakmem_env_cache.h:
Phase 23 Unified Cache + PageFaultTelemetry generalization: Mid/VM page-fault bottleneck identified Summary: - Phase 23 Unified Cache: +30% improvement (Random Mixed 256B: 18.18M → 23.68M ops/s) - PageFaultTelemetry: Extended to generic buckets (C0-C7, MID, L25, SSM) - Measurement-driven decision: Mid/VM page-faults (80-100K) >> Tiny (6K) → prioritize Mid/VM optimization Phase 23 Changes: 1. Unified Cache implementation (core/front/tiny_unified_cache.{c,h}) - Direct SuperSlab carve (TLS SLL bypass) - Self-contained pop-or-refill pattern - ENV: HAKMEM_TINY_UNIFIED_CACHE=1, HAKMEM_TINY_UNIFIED_C{0-7}=128 2. Fast path pruning (tiny_alloc_fast.inc.h, tiny_free_fast_v2.inc.h) - Unified ON → direct cache access (skip all intermediate layers) - Alloc: unified_cache_pop_or_refill() → immediate fail to slow - Free: unified_cache_push() → fallback to SLL only if full PageFaultTelemetry Changes: 3. Generic bucket architecture (core/box/pagefault_telemetry_box.{c,h}) - PF_BUCKET_{C0-C7, MID, L25, SSM} for domain-specific measurement - Integration: hak_pool_try_alloc(), l25_alloc_new_run(), shared_pool_allocate_superslab_unlocked() 4. Measurement results (Random Mixed 500K / 256B): - Tiny C2-C7: 2-33 pages, high reuse (64-3.8 touches/page) - SSM: 512 pages (initialization footprint) - MID/L25: 0 (unused in this workload) - Mid/Large VM benchmarks: 80-100K page-faults (13-16x higher than Tiny) Ring Cache Enhancements: 5. Hot Ring Cache (core/front/tiny_ring_cache.{c,h}) - ENV: HAKMEM_TINY_HOT_RING_ENABLE=1, HAKMEM_TINY_HOT_RING_C{0-7}=size - Conditional compilation cleanup Documentation: 6. Analysis reports - RANDOM_MIXED_BOTTLENECK_ANALYSIS.md: Page-fault breakdown - RANDOM_MIXED_SUMMARY.md: Phase 23 summary - RING_CACHE_ACTIVATION_GUIDE.md: Ring cache usage - CURRENT_TASK.md: Updated with Phase 23 results and Phase 24 plan Next Steps (Phase 24): - Target: Mid/VM PageArena/HotSpanBox (page-fault reduction 80-100K → 30-40K) - Tiny SSM optimization deferred (low ROI, ~6K page-faults already optimal) - Expected improvement: +30-50% for Mid/Large workloads Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 02:47:58 +09:00
core/box/../superslab/superslab_inline.h:
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../box/ss_slab_meta_box.h:
Remove dead code: UltraHot, RingCache, FrontC23, Class5 Hotpath (cherry-picked from 225b6fcc7, conflicts resolved)
2025-11-26 12:33:49 +09:00
core/box/../box/slab_freelist_atomic.h:
Phase 23 Unified Cache + PageFaultTelemetry generalization: Mid/VM page-fault bottleneck identified Summary: - Phase 23 Unified Cache: +30% improvement (Random Mixed 256B: 18.18M → 23.68M ops/s) - PageFaultTelemetry: Extended to generic buckets (C0-C7, MID, L25, SSM) - Measurement-driven decision: Mid/VM page-faults (80-100K) >> Tiny (6K) → prioritize Mid/VM optimization Phase 23 Changes: 1. Unified Cache implementation (core/front/tiny_unified_cache.{c,h}) - Direct SuperSlab carve (TLS SLL bypass) - Self-contained pop-or-refill pattern - ENV: HAKMEM_TINY_UNIFIED_CACHE=1, HAKMEM_TINY_UNIFIED_C{0-7}=128 2. Fast path pruning (tiny_alloc_fast.inc.h, tiny_free_fast_v2.inc.h) - Unified ON → direct cache access (skip all intermediate layers) - Alloc: unified_cache_pop_or_refill() → immediate fail to slow - Free: unified_cache_push() → fallback to SLL only if full PageFaultTelemetry Changes: 3. Generic bucket architecture (core/box/pagefault_telemetry_box.{c,h}) - PF_BUCKET_{C0-C7, MID, L25, SSM} for domain-specific measurement - Integration: hak_pool_try_alloc(), l25_alloc_new_run(), shared_pool_allocate_superslab_unlocked() 4. Measurement results (Random Mixed 500K / 256B): - Tiny C2-C7: 2-33 pages, high reuse (64-3.8 touches/page) - SSM: 512 pages (initialization footprint) - MID/L25: 0 (unused in this workload) - Mid/Large VM benchmarks: 80-100K page-faults (13-16x higher than Tiny) Ring Cache Enhancements: 5. Hot Ring Cache (core/front/tiny_ring_cache.{c,h}) - ENV: HAKMEM_TINY_HOT_RING_ENABLE=1, HAKMEM_TINY_HOT_RING_C{0-7}=size - Conditional compilation cleanup Documentation: 6. Analysis reports - RANDOM_MIXED_BOTTLENECK_ANALYSIS.md: Page-fault breakdown - RANDOM_MIXED_SUMMARY.md: Phase 23 summary - RING_CACHE_ACTIVATION_GUIDE.md: Ring cache usage - CURRENT_TASK.md: Updated with Phase 23 results and Phase 24 plan Next Steps (Phase 24): - Target: Mid/VM PageArena/HotSpanBox (page-fault reduction 80-100K → 30-40K) - Tiny SSM optimization deferred (low ROI, ~6K page-faults already optimal) - Expected improvement: +30-50% for Mid/Large workloads Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 02:47:58 +09:00
core/box/../box/free_remote_box.h:
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/hakmem_tiny_integrity.h:
Phase 15: Box BenchMeta separation + ExternalGuard debug + investigation report - Implement Box BenchMeta pattern in bench_random_mixed.c (BENCH_META_CALLOC/FREE) - Add enhanced debug logging to external_guard_box.h (caller tracking, FG classification) - Document investigation in PHASE15_BUG_ANALYSIS.md Issue: Page-aligned MIDCAND pointer not in SuperSlab registry → ExternalGuard → crash Hypothesis: May be pre-existing SuperSlab bug (not Phase 15-specific) Next: Test in Phase 14-C to verify
2025-11-15 23:00:21 +09:00
core/box/front_gate_v2.h:
core/box/external_guard_box.h:
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/ss_slab_meta_box.h:
feat: Add ACE allocation failure tracing and debug hooks This commit introduces a comprehensive tracing mechanism for allocation failures within the Adaptive Cache Engine (ACE) component. This feature allows for precise identification of the root cause for Out-Of-Memory (OOM) issues related to ACE allocations. Key changes include: - **ACE Tracing Implementation**: - Added environment variable to enable/disable detailed logging of allocation failures. - Instrumented , , and to distinguish between "Threshold" (size class mismatch), "Exhaustion" (pool depletion), and "MapFail" (OS memory allocation failure). - **Build System Fixes**: - Corrected to ensure is properly linked into , resolving an error. - **LD_PRELOAD Wrapper Adjustments**: - Investigated and understood the wrapper's behavior under , particularly its interaction with and checks. - Enabled debugging flags for environment to prevent unintended fallbacks to 's for non-tiny allocations, allowing comprehensive testing of the allocator. - **Debugging & Verification**: - Introduced temporary verbose logging to pinpoint execution flow issues within interception and routing. These temporary logs have been removed. - Created to facilitate testing of the tracing features. This feature will significantly aid in diagnosing and resolving allocation-related OOM issues in by providing clear insights into the failure pathways.
2025-12-01 16:37:59 +09:00
core/box/fg_tiny_gate_box.h:
Tiny: fix header/stride mismatch and harden refill paths - Root cause: header-based class indexing (HEADER_CLASSIDX=1) wrote a 1-byte header during allocation, but linear carve/refill and initial slab capacity still used bare class block sizes. This mismatch could overrun slab usable space and corrupt freelists, causing reproducible SEGV at ~100k iters. Changes - Superslab: compute capacity with effective stride (block_size + header for classes 0..6; class7 remains headerless) in superslab_init_slab(). Add a debug-only bound check in superslab_alloc_from_slab() to fail fast if carve would exceed usable bytes. - Refill (non-P0 and P0): use header-aware stride for all linear carving and TLS window bump operations. Ensure alignment/validation in tiny_refill_opt.h also uses stride, not raw class size. - Drain: keep existing defense-in-depth for remote sentinel and sanitize nodes before splicing into freelist (already present). Notes - This unifies the memory layout across alloc/linear-carve/refill with a single stride definition and keeps class7 (1024B) headerless as designed. - Debug builds add fail-fast checks; release builds remain lean. Next - Re-run Tiny benches (256/1024B) in debug to confirm stability, then in release. If any remaining crash persists, bisect with HAKMEM_TINY_P0_BATCH_REFILL=0 to isolate P0 batch carve, and continue reducing branch-miss as planned.
2025-11-09 18:55:50 +09:00
core/box/hak_wrappers.inc.h:
Phase 15: Box BenchMeta separation + ExternalGuard debug + investigation report - Implement Box BenchMeta pattern in bench_random_mixed.c (BENCH_META_CALLOC/FREE) - Add enhanced debug logging to external_guard_box.h (caller tracking, FG classification) - Document investigation in PHASE15_BUG_ANALYSIS.md Issue: Page-aligned MIDCAND pointer not in SuperSlab registry → ExternalGuard → crash Hypothesis: May be pre-existing SuperSlab bug (not Phase 15-specific) Next: Test in Phase 14-C to verify
2025-11-15 23:00:21 +09:00
core/box/front_gate_classifier.h:
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../front/malloc_tiny_fast.h:
core/box/../front/../hakmem_build_flags.h:
core/box/../front/../hakmem_tiny_config.h:
ENV Cleanup Step 8: Gate HAKMEM_SUPER_LOOKUP_DEBUG in header Gate HAKMEM_SUPER_LOOKUP_DEBUG environment variable behind #if !HAKMEM_BUILD_RELEASE in hakmem_super_registry.h inline function. Changes: - Wrap s_dbg initialization in conditional compilation - Release builds use constant s_dbg = 0 for complete elimination - Debug logging in hak_super_lookup() now fully compiled out in release Performance: 30.3M ops/s Larson (stable, no regression) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 01:45:45 +09:00
core/box/../front/../superslab/superslab_inline.h:
core/box/../front/../box/ss_slab_meta_box.h:
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../front/tiny_unified_cache.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../front/../box/tiny_front_config_box.h:
core/box/../front/../box/../hakmem_build_flags.h:
Phase 3d-B: TLS Cache Merge - Unified g_tls_sll[] structure (+12-18% expected) Merge separate g_tls_sll_head[] and g_tls_sll_count[] arrays into unified TinyTLSSLL struct to improve L1D cache locality. Expected performance gain: +12-18% from reducing cache line splits (2 loads → 1 load per operation). Changes: - core/hakmem_tiny.h: Add TinyTLSSLL type (16B aligned, head+count+pad) - core/hakmem_tiny.c: Replace separate arrays with g_tls_sll[8] - core/box/tls_sll_box.h: Update Box API (13 sites) for unified access - Updated 32+ files: All g_tls_sll_head[i] → g_tls_sll[i].head - Updated 32+ files: All g_tls_sll_count[i] → g_tls_sll[i].count - core/hakmem_tiny_integrity.h: Unified canary guards - core/box/integrity_box.c: Simplified canary validation - Makefile: Added core/box/tiny_sizeclass_hist_box.o to link Build: ✅ PASS (10K ops sanity test) Warnings: Only pre-existing LTO type mismatches (unrelated) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-20 07:32:30 +09:00
core/box/../front/../tiny_region_id.h:
core/box/../front/../hakmem_tiny.h:
Refactor: Clean up SuperSlab shared pool code - Removed unused/disabled L0 cache implementation from core/hakmem_shared_pool.c. - Deleted stale backup file core/hakmem_tiny_superslab.c.bak. - Removed untracked and obsolete shared_pool source files.
2025-11-30 15:27:53 +09:00
core/box/../front/../box/tiny_front_hot_box.h:
core/box/../front/../box/../hakmem_tiny_config.h:
core/box/../front/../box/../tiny_region_id.h:
core/box/../front/../box/../front/tiny_unified_cache.h:
core/box/../front/../box/tiny_front_cold_box.h:
core/box/tiny_front_config_box.h:
Save current state before investigating TLS_SLL_HDR_RESET
2025-12-03 10:34:39 +09:00
core/box/wrapper_env_box.h:
Reference in New Issue Copy Permalink