tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
71b1354d326c6d0b4013b3f65431f0a7aa492550
hakmem/core/bench_profile.h
Moe Charm (CI) 71b1354d32 Phase 10: FREE-TINY-FAST MONO LEGACY DIRECT (GO +1.89%) 
Results:
- A/B test: +1.89% on Mixed (10-run, clean env)
- Baseline: 51.96M ops/s
- Optimized: 52.94M ops/s
- Improvement: +984K ops/s (+1.89%)
- C6-heavy verification: +7.86% (nonlegacy_mask works correctly, no misfires)

Strategy:
- Extend Phase 9 (C0-C3 DUALHOT) to C4-C7 LEGACY DIRECT
- Fail-Fast principle: Never misclassify MID/ULTRA/V7 as LEGACY
- nonlegacy_mask: Cached at init, hot path uses single bit operation

Success factors:
1. Performance improvement: +1.89% (1.9x GO threshold)
2. Safety verified: nonlegacy_mask prevents MID v3 misfire in C6-heavy
3. Phase 9 coexistence: C0-C3 (Phase 9) + C4-C7 (Phase 10) = full LEGACY coverage
4. Minimal overhead: Single bit operation in hot path (mask & (1u<<class))

Implementation:
- Patch 1: ENV gate box (free_tiny_fast_mono_legacy_direct_env_box.h)
  - ENV: HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT=0/1 (default 0)
  - nonlegacy_mask cached (reuses free_policy_fast_v2_nonlegacy_mask())
  - Probe window: 64 (avoid bench_profile putenv race)
- Patch 2: Early-exit in free_tiny_fast() (malloc_tiny_fast.h)
  - Conditions: !nonlegacy_mask, route==LEGACY, !LARSON_FIX, done==1
  - Direct call: tiny_legacy_fallback_free_base()
- Patch 3: Visibility (free_path_stats_box.h)
  - mono_legacy_direct_hit counter (compile-out in release)
- Patch 4: cleanenv extension (run_mixed_10_cleanenv.sh)
  - ENV leak protection

Safety verification (C6-heavy):
- OFF: 19.75M ops/s
- ON: 21.30M ops/s (+7.86%)
- nonlegacy_mask correctly excludes C6 (MID v3 active)
- Improvement from C0-C5, C7 direct path acceleration

Files modified:
- core/bench_profile.h: add to MIXED_TINYV3_C7_SAFE preset
- core/front/malloc_tiny_fast.h: early-exit insertion
- core/box/free_path_stats_box.h: counter
- core/box/free_tiny_fast_mono_legacy_direct_env_box.h: NEW (ENV gate + nonlegacy_mask)
- scripts/run_mixed_10_cleanenv.sh: ENV leak protection

Health check: PASSED (all profiles)

Promotion: Added to MIXED_TINYV3_C7_SAFE preset (default ON, opt-out)

Rollback: HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT=0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-14 20:09:40 +09:00

189 lines
10 KiB
C
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

#pragma once
#include <dlfcn.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#ifdef USE_HAKMEM
#include "box/wrapper_env_box.h" // wrapper_env_refresh_from_env (Phase 2 B4)
#include "box/tiny_static_route_box.h" // tiny_static_route_refresh_from_env (Phase 3 C3)
#include "box/hakmem_env_snapshot_box.h" // hakmem_env_snapshot_refresh_from_env (Phase 4 E1)
#include "box/tiny_free_route_cache_env_box.h" // tiny_free_static_route_refresh_from_env (Phase 8)
#endif
// env が未設定のときだけ既定値を入れる
static inline void bench_setenv_default(const char* key, const char* val) {
if (getenv(key) != NULL) return;
static void* (*real_malloc)(size_t) = NULL;
static int (*real_putenv)(char*) = NULL;
if (!real_malloc) {
real_malloc = (void* (*)(size_t))dlsym(RTLD_NEXT, "malloc");
if (!real_malloc) real_malloc = malloc;
}
if (!real_putenv) {
real_putenv = (int (*)(char*))dlsym(RTLD_NEXT, "putenv");
if (!real_putenv) real_putenv = putenv;
}
size_t klen = strlen(key);
size_t vlen = strlen(val);
char* buf = (char*)real_malloc(klen + vlen + 2);
if (!buf) return;
memcpy(buf, key, klen);
buf[klen] = '=';
memcpy(buf + klen + 1, val, vlen);
buf[klen + 1 + vlen] = '\0';
{
char msg[256];
int n = snprintf(msg, sizeof(msg), "[bench_profile] set %s=%s\n", key, val);
if (n > 0) {
if (n > (int)sizeof(msg)) n = (int)sizeof(msg);
ssize_t w = write(2, msg, (size_t)n);
(void)w;
}
}
real_putenv(buf); // takes ownership; do not free
}
// ベンチ専用: HAKMEM_PROFILE に応じて ENV をプリセットする
static inline void bench_apply_profile(void) {
const char* p = getenv("HAKMEM_PROFILE");
if (!p || !*p) return;
if (strcmp(p, "MIXED_TINYV3_C7_SAFE") == 0) {
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C7_HOT", "1");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x80");
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_ENABLED", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_CLASSES", "0x0");
bench_setenv_default("HAKMEM_TINY_PTR_FAST_CLASSIFY_V4_ENABLED", "0");
bench_setenv_default("HAKMEM_SMALL_SEGMENT_V4_ENABLED", "0");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
bench_setenv_default("HAKMEM_TINY_FRONT_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_TINY_FRONT_V3_LUT_ENABLED", "1");
bench_setenv_default("HAKMEM_TINY_PTR_FAST_CLASSIFY_ENABLED", "1");
// Phase FREE-TINY-FAST-DUALHOT-1: C0-C3 direct fast free (skip policy snapshot)
bench_setenv_default("HAKMEM_FREE_TINY_FAST_HOTCOLD", "1");
// Phase 2 B4: Wrapper hot/cold split (malloc/free wrapper shape)
bench_setenv_default("HAKMEM_WRAP_SHAPE", "1");
// Phase 4 E1: ENV Snapshot Consolidation (+3.92% proven on Mixed)
bench_setenv_default("HAKMEM_ENV_SNAPSHOT", "1");
// Phase 5 E4-1: Free wrapper ENV snapshot (+3.51% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FREE_WRAPPER_ENV_SNAPSHOT", "1");
// Phase 5 E4-2: Malloc wrapper ENV snapshot (+21.83% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_MALLOC_WRAPPER_ENV_SNAPSHOT", "1");
// Phase 5 E5-1: Free Tiny Direct Path (+3.35% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FREE_TINY_DIRECT", "1");
// Phase 6-1: Front FastLane (Layer Collapse) (+11.13% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FRONT_FASTLANE", "1");
// Phase 6-2: Front FastLane Free DeDup (+5.18% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FRONT_FASTLANE_FREE_DEDUP", "1");
// Phase 9: FREE-TINY-FAST MONO DUALHOT (+2.72% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FREE_TINY_FAST_MONO_DUALHOT", "1");
// Phase 10: FREE-TINY-FAST MONO LEGACY DIRECT (+1.89% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FREE_TINY_FAST_MONO_LEGACY_DIRECT", "1");
// Phase 4-4: C6 ULTRA free+alloc 統合を有効化 (default OFF, manual opt-in)
bench_setenv_default("HAKMEM_TINY_C6_ULTRA_FREE_ENABLED", "0");
// Phase MID-V3: Mid/Pool HotBox v3
// Mixed (16–1024B) では MID_V3(C6) が大きく遅くなるため、デフォルト OFF に固定。
// C6-heavy プロファイル側でのみ ON を推奨する（C6-heavy のみ最適化対象）。
bench_setenv_default("HAKMEM_MID_V3_ENABLED", "0");
bench_setenv_default("HAKMEM_MID_V3_CLASSES", "0x0");
// Phase 2 B3: Routing branch shape optimization (LIKELY on LEGACY, cold helper for rare routes)
bench_setenv_default("HAKMEM_TINY_ALLOC_ROUTE_SHAPE", "1");
// Phase 3 C3: Static routing (policy_snapshot bypass, +2.2% proven)
bench_setenv_default("HAKMEM_TINY_STATIC_ROUTE", "1");
// Phase 3 D1: Free route cache (TLS cache for free path routing, +2.19% proven)
bench_setenv_default("HAKMEM_FREE_STATIC_ROUTE", "1");
} else if (strcmp(p, "C6_HEAVY_LEGACY_POOLV1") == 0) {
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C6_HOT", "0");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x80");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
bench_setenv_default("HAKMEM_POOL_V1_FLATTEN_ENABLED", "0");
bench_setenv_default("HAKMEM_MID_DESC_CACHE_ENABLED", "1");
// Phase 4-4: C6 ULTRA free+alloc 統合を有効化 (default OFF, manual opt-in)
bench_setenv_default("HAKMEM_TINY_C6_ULTRA_FREE_ENABLED", "0");
// Phase MID-V3: Mid/Pool HotBox v3 (257-768B, C6 only)
bench_setenv_default("HAKMEM_MID_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_MID_V3_CLASSES", "0x40");
// Phase 6-1: Front FastLane (Layer Collapse) (+11.13% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FRONT_FASTLANE", "1");
// Phase 6-2: Front FastLane Free DeDup (+5.18% proven on Mixed, 10-run)
bench_setenv_default("HAKMEM_FRONT_FASTLANE_FREE_DEDUP", "1");
// Phase 2 B3: Routing branch shape optimization (LIKELY on LEGACY, cold helper for rare routes)
bench_setenv_default("HAKMEM_TINY_ALLOC_ROUTE_SHAPE", "1");
} else if (strcmp(p, "C6_V7_STUB") == 0) {
// Phase v7-1: C6-only v7 stub 実験用（MID v3 fallback）
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C6_HOT", "0");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x80");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
bench_setenv_default("HAKMEM_MID_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_MID_V3_CLASSES", "0x40");
// v7 stub ON (C6-only)
bench_setenv_default("HAKMEM_SMALL_HEAP_V7_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V7_CLASSES", "0x40");
} else if (strcmp(p, "C6_HEAVY_LEGACY_POOLV1_FLATTEN") == 0) {
// LEGACY mid/smallmid ベンチ専用（C7_SAFE では使用しない）
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "LEGACY");
bench_setenv_default("HAKMEM_TINY_C6_HOT", "0");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x80");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
bench_setenv_default("HAKMEM_POOL_V1_FLATTEN_ENABLED", "1");
bench_setenv_default("HAKMEM_POOL_V1_FLATTEN_STATS", "1");
bench_setenv_default("HAKMEM_POOL_ZERO_MODE", "header");
} else if (strcmp(p, "DEBUG_TINY_FRONT_PERF") == 0) {
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C7_HOT", "1");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x80");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
bench_setenv_default("HAKMEM_TINY_FRONT_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_TINY_FRONT_V3_LUT_ENABLED", "1");
bench_setenv_default("HAKMEM_TINY_PTR_FAST_CLASSIFY_ENABLED", "1");
} else if (strcmp(p, "C6_SMALL_HEAP_V3_EXPERIMENT") == 0) {
// C6 を SmallObject v3 に載せる研究用（標準では使用しない）
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C6_HOT", "1");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x40"); // C6 only
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_ENABLED", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_CLASSES", "0x0");
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
} else if (strcmp(p, "C6_SMALL_HEAP_V4_EXPERIMENT") == 0) {
// C6 を SmallObject v4 に載せる研究用（標準では使用しない）
bench_setenv_default("HAKMEM_TINY_HEAP_PROFILE", "C7_SAFE");
bench_setenv_default("HAKMEM_TINY_C6_HOT", "1");
bench_setenv_default("HAKMEM_TINY_HOTHEAP_V2", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_ENABLED", "0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V3_CLASSES", "0x0");
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_ENABLED", "1");
bench_setenv_default("HAKMEM_SMALL_HEAP_V4_CLASSES", "0x40"); // C6 only
bench_setenv_default("HAKMEM_POOL_V2_ENABLED", "0");
}
#ifdef USE_HAKMEM
// Phase 3 C3 Step 0: Ensure policy snapshot reflects final ENV after putenv defaults.
small_policy_v7_bump_version();
// Phase 2 B4: Sync wrapper ENV cache after bench_profile putenv defaults.
wrapper_env_refresh_from_env();
// Phase 3 C3: Sync static route cache after bench_profile putenv defaults.
tiny_static_route_refresh_from_env();
// Phase 4 E1: Sync ENV snapshot cache after bench_profile putenv defaults.
hakmem_env_snapshot_refresh_from_env();
// Phase 8: Sync free static route ENV cache after bench_profile putenv defaults.
tiny_free_static_route_refresh_from_env();
#endif
}
Reference in New Issue View Git Blame Copy Permalink