hakmem/CURRENT_TASK.md

## HAKMEM Bug Investigation: OOM Spam (ACE 33KB) - December 1, 2025

### Objective
Investigate and provide a mechanism to diagnose "OOM spam caused by continuous NULL returns for ACE 33KB allocations." The goal is to distinguish between:
1. Threshold issues (size class rounding)
2. Cache exhaustion (pool empty)
3. Mapping failures (OS mmap failure)

### Work Performed & Resolution

1.  **Implemented ACE Tracing**:
    *   Added a runtime-controlled tracing mechanism via the `HAKMEM_ACE_TRACE=1` environment variable.
    *   Instrumentation was added to `core/hakmem_ace.c`, `core/hakmem_pool.c`, and `core/hakmem_l25_pool.c` to log specific failure reasons to `stderr`.
    *   Log messages distinguish between `[ACE-FAIL] Threshold`, `[ACE-FAIL] Exhaustion`, and `[ACE-FAIL] MapFail`.

2.  **Resolved Build & Linkage Issues**:
    *   **Undefined Symbol `classify_ptr`**: Identified that `core/box/front_gate_classifier.c` was not correctly linked into `libhakmem.so`. The `Makefile` was updated to include `core/box/front_gate_classifier_shared.o` in the `SHARED_OBJS` list.
    *   **Removed Temporary Debug Logs**: All interim `write(2, ...)` and `fprintf(stderr, ...)` debug statements introduced during the investigation have been removed to restore a clean code state.

3.  **Clarified `malloc` Wrapper Behavior**:
    *   Discovered that `libhakmem.so`'s `malloc` wrapper had logic to force fallback to `libc`'s `malloc` for larger allocations (`> TINY_MAX_SIZE`) and when `jemalloc` was detected, especially under `LD_PRELOAD`.
    *   This was preventing 33KB allocations from reaching the `hakmem` ACE layer.
    *   **Solution**: Identified the necessary environment variables to disable these bypasses for testing purposes: `HAKMEM_LD_SAFE=0` and `HAKMEM_LD_BLOCK_JEMALLOC=0`.

4.  **Verified Trace Functionality**:
    *   A test program (`test_ace_trace.c`) was used to allocate 33KB.
    *   By setting `HAKMEM_WMAX_MID=1.01` and `HAKMEM_WMAX_LARGE=1.01` (to force threshold failures), the `[ACE-FAIL] Threshold` logs were successfully generated, confirming the tracing mechanism works as intended.

### How to Use the Trace Feature (for Users)

To diagnose the 33KB OOM spam issue in your application:

1.  **Ensure Correct `libhakmem.so` Build**:
    Make sure `libhakmem.so` is built without `POOL_TLS_PHASE1` enabled (e.g., `make shared POOL_TLS_PHASE1=0`). The current `libhakmem.so` reflects this.

2.  **Run Your Application with Specific Environment Variables**:
    ```bash
    export HAKMEM_FRONT_GATE_UNIFIED=0
    export HAKMEM_SMALLMID_ENABLE=0
    export HAKMEM_FORCE_LIBC_ALLOC=0
    export HAKMEM_LD_BLOCK_JEMALLOC=0
    export HAKMEM_ACE_TRACE=1          # Crucial for seeing the logs
    export HAKMEM_WMAX_MID=1.60        # Use default or adjust as needed for W_MAX analysis
    export HAKMEM_WMAX_LARGE=1.30      # Use default or adjust as needed for W_MAX analysis
    export LD_PRELOAD=/path/to/hakmem/libhakmem.so

    ./your_application 2> stderr.log   # Redirect stderr to a file for analysis
    ```

3.  **Analyze `stderr.log`**:
    Look for `[ACE-FAIL]` messages to determine if the issue is a `Threshold` (e.g., `size=33000 wmax=...`), `Exhaustion` (pool empty), or `MapFail` (OS allocation error). This will provide the necessary data to pinpoint the root cause of the OOM spam.

This setup will allow for precise diagnosis of 33KB allocation failures within the hakmem ACE component.