tomoaki/hakmem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 17-1 Revision: Small-Mid Front Box Only (ChatGPT Strategy)

Browse Source 
STRATEGY CHANGE (ChatGPT reviewed):
- Phase 17-1: Build FRONT BOX ONLY (no dedicated SuperSlab backend)
- Backend: Reuse existing Tiny SuperSlab/SharedPool APIs
- Goal: Measure performance impact before building dedicated infrastructure
- A/B test: Does thin front layer improve 256-1KB performance?

RATIONALE (ChatGPT analysis):
1. Tiny/Middle/Large need different properties - same SuperSlab causes conflict
2. Metadata shapes collide - struct bloat → L1 miss increase
3. Learning signals get muddied - size-specific control becomes difficult

IMPLEMENTATION:
- Reduced size classes: 5 → 3 (256B/512B/1KB only)
- Removed dedicated SuperSlab backend stub
- Backend: Direct delegation to hak_tiny_alloc/free
- TLS freelist: Thin front cache (32/24/16 capacity)
- Fast path: TLS hit (pop/push with header 0xb0)
- Slow path: Backend alloc via Tiny (no TLS refill)
- Free path: TLS push if space, else delegate to Tiny

ARCHITECTURE:
  Tiny:      0-255B    (C0-C5, unchanged)
  Small-Mid: 256-1KB   (SM0-SM2, Front Box, backend=Tiny)
  Mid:       8KB-32KB  (existing)

FILES CHANGED:
- hakmem_smallmid.h: Reduced to 3 classes, updated docs
- hakmem_smallmid.c: Removed SuperSlab stub, added backend delegation

NEXT STEPS:
- Integrate into hak_alloc_api.inc.h routing
- A/B benchmark: Small-Mid ON/OFF comparison
- If successful (2x improvement), consider Phase 17-2 dedicated backend

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Moe Charm (CI)
2025-11-16 01:51:43 +09:00
parent 993c5419b7
commit cdaf117581
2 changed files with 96 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files

139
core/hakmem_smallmid.c
View File

@ -1,19 +1,27 @@
/** /**
* hakmem_smallmid.c - Small-Mid Allocator Box Implementation * hakmem_smallmid.c - Small-Mid Allocator Front Box Implementation
* *
* Phase 17: Dedicated allocator layer for 256B-4KB range * Phase 17-1: Front Box Only (No Dedicated SuperSlab Backend)
*
* Strategy (ChatGPT reviewed):
* - Thin front layer with TLS freelist (256B/512B/1KB)
* - Backend: Use existing Tiny SuperSlab/SharedPool APIs
* - Goal: Measure performance impact before building dedicated backend
* - A/B test: Does Small-Mid front improve 256-1KB performance?
* *
* Architecture: * Architecture:
* - Dedicated SuperSlab pool (separated from Tiny) * - 3 size classes: 256B/512B/1KB (reduced from 5)
* - TLS freelist for fast alloc/free * - TLS freelist for fast alloc/free (static inline)
* - Header-based class identification (Phase 7) * - Backend: Call Tiny allocator APIs (reuse existing infrastructure)
* - ENV controlled (HAKMEM_SMALLMID_ENABLE=1) * - ENV controlled (HAKMEM_SMALLMID_ENABLE=1)
* *
* Created: 2025-11-16 * Created: 2025-11-16
* Updated: 2025-11-16 (Phase 17-1 revision - Front Box only)
*/ */
#include "hakmem_smallmid.h" #include "hakmem_smallmid.h"
#include "hakmem_build_flags.h" #include "hakmem_build_flags.h"
#include "hakmem_tiny.h" // For backend: hak_tiny_alloc / hak_tiny_free
#include "tiny_region_id.h" // For header writing #include "tiny_region_id.h" // For header writing
#include <string.h> #include <string.h>
#include <pthread.h> #include <pthread.h>
@ -26,15 +34,13 @@ __thread void* g_smallmid_tls_head[SMALLMID_NUM_CLASSES] = {NULL};
__thread uint32_t g_smallmid_tls_count[SMALLMID_NUM_CLASSES] = {0}; __thread uint32_t g_smallmid_tls_count[SMALLMID_NUM_CLASSES] = {0};
// ============================================================================ // ============================================================================
// Size Class Table // Size Class Table (Phase 17-1: 3 classes)
// ============================================================================ // ============================================================================
const size_t g_smallmid_class_sizes[SMALLMID_NUM_CLASSES] = { const size_t g_smallmid_class_sizes[SMALLMID_NUM_CLASSES] = {
256, // SM0: 256B 256, // SM0: 256B
512, // SM1: 512B 512, // SM1: 512B
1024, // SM2: 1KB 1024 // SM2: 1KB
2048, // SM3: 2KB
4096 // SM4: 4KB
}; };
// ============================================================================ // ============================================================================
@ -95,7 +101,7 @@ void smallmid_init(void) {
pthread_mutex_lock(&g_smallmid_init_lock); pthread_mutex_lock(&g_smallmid_init_lock);
if (!g_smallmid_initialized) { if (!g_smallmid_initialized) {
SMALLMID_LOG("Initializing Small-Mid allocator..."); SMALLMID_LOG("Initializing Small-Mid Front Box...");
// Check ENV // Check ENV
if (!smallmid_is_enabled()) { if (!smallmid_is_enabled()) {
@ -105,12 +111,11 @@ void smallmid_init(void) {
return; return;
} }
// TODO Phase 17-2: Initialize dedicated SuperSlab pool // Phase 17-1: No dedicated backend - use existing Tiny infrastructure
// - Allocate SuperSlab pool (separated from Tiny) // No additional initialization needed (TLS state is static)
// - Set up refill logic
g_smallmid_initialized = 1; g_smallmid_initialized = 1;
SMALLMID_LOG("Small-Mid allocator initialized (5 classes: 256B-4KB)"); SMALLMID_LOG("Small-Mid Front Box initialized (3 classes: 256B/512B/1KB, backend=Tiny)");
} }
pthread_mutex_unlock(&g_smallmid_init_lock); pthread_mutex_unlock(&g_smallmid_init_lock);
@ -165,32 +170,41 @@ static inline bool smallmid_tls_push(int class_idx, void* ptr) {
} }
// ============================================================================ // ============================================================================
// SuperSlab Backend (Stub for Phase 17-2) // Backend: Use Tiny Allocator APIs (Phase 17-1)
// ============================================================================ // ============================================================================
/** /**
* smallmid_superslab_refill - Refill TLS from dedicated SuperSlab pool * smallmid_backend_alloc - Allocate from Tiny backend
* *
* @param class_idx Size class index * @param size Allocation size
* @param count Number of blocks to refill * @return Allocated pointer (user pointer, no Small-Mid header)
* @return true on success, false on OOM
* *
* TODO Phase 17-2: Implement dedicated SuperSlab backend * Phase 17-1: Delegate to existing Tiny allocator infrastructure
* This reuses Tiny's SuperSlab/SharedPool without building dedicated backend
*/ */
static bool smallmid_superslab_refill(int class_idx, uint32_t count) { static inline void* smallmid_backend_alloc(size_t size) {
(void)class_idx;
(void)count;
#ifdef HAKMEM_SMALLMID_STATS #ifdef HAKMEM_SMALLMID_STATS
__atomic_fetch_add(&g_smallmid_stats.tls_misses, 1, __ATOMIC_RELAXED); __atomic_fetch_add(&g_smallmid_stats.tls_misses, 1, __ATOMIC_RELAXED);
__atomic_fetch_add(&g_smallmid_stats.superslab_refills, 1, __ATOMIC_RELAXED);
#endif #endif
// TODO: Allocate from dedicated SuperSlab pool // Call Tiny allocator (reuses existing SuperSlab/SharedPool)
// For now, return false (not implemented) void* ptr = hak_tiny_alloc(size);
SMALLMID_LOG("smallmid_superslab_refill: not yet implemented (class=%d, count=%u)", SMALLMID_LOG("smallmid_backend_alloc(%zu) = %p (via Tiny)", size, ptr);
class_idx, count); return ptr;
return false; }
/**
* smallmid_backend_free - Free to Tiny backend
*
* @param ptr User pointer (no Small-Mid header)
* @param size Allocation size
*
* Phase 17-1: Delegate to existing Tiny allocator infrastructure
*/
static inline void smallmid_backend_free(void* ptr, size_t size) {
(void)size; // Unused: Tiny free reads header, doesn't need size
SMALLMID_LOG("smallmid_backend_free(%p) (via Tiny)", ptr);
hak_tiny_free(ptr);
} }
// ============================================================================ // ============================================================================
@ -233,22 +247,16 @@ void* smallmid_alloc(size_t size) {
return (uint8_t*)ptr + 1; // Return user pointer (skip header) return (uint8_t*)ptr + 1; // Return user pointer (skip header)
} }
// Slow path: Refill from SuperSlab // Slow path: Allocate from Tiny backend (no refill, direct delegation)
uint32_t refill_count = smallmid_tls_capacity(class_idx) / 2; // Refill 50% // Phase 17-1: Simplified - no TLS refill, just pass through to Tiny
if (!smallmid_superslab_refill(class_idx, refill_count)) { void* backend_ptr = smallmid_backend_alloc(size);
SMALLMID_LOG("smallmid_alloc(%zu) = NULL (SuperSlab refill failed)", size); if (!backend_ptr) {
return NULL; // OOM or not implemented yet SMALLMID_LOG("smallmid_alloc(%zu) = NULL (backend alloc failed)", size);
}
// Try again after refill
ptr = smallmid_tls_pop(class_idx);
if (ptr) {
SMALLMID_LOG("smallmid_alloc(%zu) = %p (after refill, class=%d)", size, ptr, class_idx);
return (uint8_t*)ptr + 1; // Return user pointer (skip header)
}
SMALLMID_LOG("smallmid_alloc(%zu) = NULL (refill succeeded but TLS still empty?)", size);
return NULL; return NULL;
}
SMALLMID_LOG("smallmid_alloc(%zu) = %p (backend alloc, class=%d)", size, backend_ptr, class_idx);
return backend_ptr; // Backend returns user pointer directly
} }
// ============================================================================ // ============================================================================
@ -267,31 +275,32 @@ void smallmid_free(void* ptr) {
__atomic_fetch_add(&g_smallmid_stats.total_frees, 1, __ATOMIC_RELAXED); __atomic_fetch_add(&g_smallmid_stats.total_frees, 1, __ATOMIC_RELAXED);
#endif #endif
// Header-based fast free (Phase 7 technology) // Phase 17-1: Read header to identify if this is a Small-Mid TLS allocation
// Read 1-byte header at [ptr - 1] to get class_idx // or a backend (Tiny) allocation
uint8_t* base = (uint8_t*)ptr - 1; uint8_t* base = (uint8_t*)ptr - 1;
uint8_t header = *base; uint8_t header = *base;
// Header format: 0xa0 | class_idx (same as Tiny) // Small-Mid TLS allocations have magic 0xb0
// For Small-Mid, we use a different magic to distinguish from Tiny // Tiny allocations have magic 0xa0
// Small-Mid magic: 0xb0 | class_idx uint8_t magic = header & 0xf0;
int class_idx = header & 0x0f; int class_idx = header & 0x0f;
if (class_idx < 0 || class_idx >= SMALLMID_NUM_CLASSES) { if (magic == 0xb0 && class_idx >= 0 && class_idx < SMALLMID_NUM_CLASSES) {
SMALLMID_LOG("smallmid_free(%p): invalid class_idx=%d from header=0x%02x", // This is a Small-Mid TLS allocation, push to TLS freelist
ptr, class_idx, header);
return; // Invalid header, skip
}
// Push to TLS freelist
if (smallmid_tls_push(class_idx, base)) { if (smallmid_tls_push(class_idx, base)) {
SMALLMID_LOG("smallmid_free(%p): pushed to TLS (class=%d)", ptr, class_idx); SMALLMID_LOG("smallmid_free(%p): pushed to TLS (class=%d)", ptr, class_idx);
return; return;
} }
// TLS full: Drain to remote or SuperSlab // TLS full: Delegate to Tiny backend
// TODO Phase 17-3: Implement remote drain SMALLMID_LOG("smallmid_free(%p): TLS full, delegating to backend", ptr);
SMALLMID_LOG("smallmid_free(%p): TLS full, remote drain not yet implemented", ptr); // Fall through to backend free
}
// This is a backend (Tiny) allocation, or TLS full - delegate to Tiny
// Tiny will handle the free based on its own header (0xa0)
size_t size = 0; // Tiny free doesn't need size, it reads header
smallmid_backend_free(ptr, size);
} }
// ============================================================================ // ============================================================================
@ -303,9 +312,15 @@ void smallmid_thread_exit(void) {
SMALLMID_LOG("smallmid_thread_exit: cleaning up TLS state"); SMALLMID_LOG("smallmid_thread_exit: cleaning up TLS state");
// TODO Phase 17-3: Return TLS blocks to SuperSlab // Phase 17-1: Return TLS blocks to Tiny backend
// For now, just reset counts (memory leak for testing)
for (int i = 0; i < SMALLMID_NUM_CLASSES; i++) { for (int i = 0; i < SMALLMID_NUM_CLASSES; i++) {
void* head = g_smallmid_tls_head[i];
while (head) {
void* next = *(void**)((uint8_t*)head + 1);
void* user_ptr = (uint8_t*)head + 1;
smallmid_backend_free(user_ptr, 0);
head = next;
}
g_smallmid_tls_head[i] = NULL; g_smallmid_tls_head[i] = NULL;
g_smallmid_tls_count[i] = 0; g_smallmid_tls_count[i] = 0;
} }

39
core/hakmem_smallmid.h
View File

@ -37,21 +37,19 @@ extern "C" {
#endif #endif
// ============================================================================ // ============================================================================
// Size Classes // Size Classes (Phase 17-1: Front Box Only, 3 classes)
// ============================================================================ // ============================================================================
#define SMALLMID_NUM_CLASSES 5 #define SMALLMID_NUM_CLASSES 3
// Size class indices // Size class indices
#define SMALLMID_CLASS_256B 0 // 256B blocks #define SMALLMID_CLASS_256B 0 // 256B blocks
#define SMALLMID_CLASS_512B 1 // 512B blocks #define SMALLMID_CLASS_512B 1 // 512B blocks
#define SMALLMID_CLASS_1KB 2 // 1KB blocks #define SMALLMID_CLASS_1KB 2 // 1KB blocks
#define SMALLMID_CLASS_2KB 3 // 2KB blocks
#define SMALLMID_CLASS_4KB 4 // 4KB blocks
// Size boundaries // Size boundaries
#define SMALLMID_MIN_SIZE (256) // 256B (must be > Tiny max when enabled) #define SMALLMID_MIN_SIZE (256) // 256B (must be > Tiny max when enabled)
#define SMALLMID_MAX_SIZE (4096) // 4KB #define SMALLMID_MAX_SIZE (1024) // 1KB (reduced for Phase 17-1)
// ============================================================================ // ============================================================================
// TLS Freelist State // TLS Freelist State
@ -66,11 +64,10 @@ extern __thread void* g_smallmid_tls_head[SMALLMID_NUM_CLASSES];
extern __thread uint32_t g_smallmid_tls_count[SMALLMID_NUM_CLASSES]; extern __thread uint32_t g_smallmid_tls_count[SMALLMID_NUM_CLASSES];
// Capacity limits (per-class TLS cache) // Capacity limits (per-class TLS cache)
#define SMALLMID_TLS_CAPACITY_256B 64 // Phase 17-1: Conservative limits for Front Box
#define SMALLMID_TLS_CAPACITY_512B 48 #define SMALLMID_TLS_CAPACITY_256B 32
#define SMALLMID_TLS_CAPACITY_1KB 32 #define SMALLMID_TLS_CAPACITY_512B 24
#define SMALLMID_TLS_CAPACITY_2KB 24 #define SMALLMID_TLS_CAPACITY_1KB 16
#define SMALLMID_TLS_CAPACITY_4KB 16
// ============================================================================ // ============================================================================
// Size Class Mapping // Size Class Mapping
@ -78,34 +75,32 @@ extern __thread uint32_t g_smallmid_tls_count[SMALLMID_NUM_CLASSES];
/** /**
* g_smallmid_class_sizes - Size class stride table * g_smallmid_class_sizes - Size class stride table
* [SM0]=256, [SM1]=512, [SM2]=1024, [SM3]=2048, [SM4]=4096 * Phase 17-1: [SM0]=256, [SM1]=512, [SM2]=1024
*/ */
extern const size_t g_smallmid_class_sizes[SMALLMID_NUM_CLASSES]; extern const size_t g_smallmid_class_sizes[SMALLMID_NUM_CLASSES];
/** /**
* smallmid_size_to_class - Convert size to size class index * smallmid_size_to_class - Convert size to size class index
* *
* @param size Allocation size (256-4096) * @param size Allocation size (256-1024)
* @return Size class index (0-4), or -1 if out of range * @return Size class index (0-2), or -1 if out of range
*/ */
static inline int smallmid_size_to_class(size_t size) { static inline int smallmid_size_to_class(size_t size) {
if (size <= 256) return SMALLMID_CLASS_256B; if (size <= 256) return SMALLMID_CLASS_256B;
if (size <= 512) return SMALLMID_CLASS_512B; if (size <= 512) return SMALLMID_CLASS_512B;
if (size <= 1024) return SMALLMID_CLASS_1KB; if (size <= 1024) return SMALLMID_CLASS_1KB;
if (size <= 2048) return SMALLMID_CLASS_2KB;
if (size <= 4096) return SMALLMID_CLASS_4KB;
return -1; // Out of range return -1; // Out of range
} }
/** /**
* smallmid_class_to_size - Convert size class to block size * smallmid_class_to_size - Convert size class to block size
* *
* @param class_idx Size class index (0-4) * @param class_idx Size class index (0-2)
* @return Block size in bytes (256/512/1024/2048/4096) * @return Block size in bytes (256/512/1024)
*/ */
static inline size_t smallmid_class_to_size(int class_idx) { static inline size_t smallmid_class_to_size(int class_idx) {
static const size_t sizes[SMALLMID_NUM_CLASSES] = { static const size_t sizes[SMALLMID_NUM_CLASSES] = {
256, 512, 1024, 2048, 4096 256, 512, 1024
}; };
return (class_idx >= 0 && class_idx < SMALLMID_NUM_CLASSES) ? sizes[class_idx] : 0; return (class_idx >= 0 && class_idx < SMALLMID_NUM_CLASSES) ? sizes[class_idx] : 0;
} }
@ -114,7 +109,7 @@ static inline size_t smallmid_class_to_size(int class_idx) {
* smallmid_is_in_range - Check if size is in Small-Mid range * smallmid_is_in_range - Check if size is in Small-Mid range
* *
* @param size Allocation size * @param size Allocation size
* @return true if 256B ≤ size ≤ 4KB * @return true if 256B ≤ size ≤ 1KB
*/ */
static inline bool smallmid_is_in_range(size_t size) { static inline bool smallmid_is_in_range(size_t size) {
return (size >= SMALLMID_MIN_SIZE && size <= SMALLMID_MAX_SIZE); return (size >= SMALLMID_MIN_SIZE && size <= SMALLMID_MAX_SIZE);
@ -123,16 +118,14 @@ static inline bool smallmid_is_in_range(size_t size) {
/** /**
* smallmid_tls_capacity - Get TLS cache capacity for given class * smallmid_tls_capacity - Get TLS cache capacity for given class
* *
* @param class_idx Size class index (0-4) * @param class_idx Size class index (0-2)
* @return TLS cache capacity * @return TLS cache capacity
*/ */
static inline uint32_t smallmid_tls_capacity(int class_idx) { static inline uint32_t smallmid_tls_capacity(int class_idx) {
static const uint32_t capacities[SMALLMID_NUM_CLASSES] = { static const uint32_t capacities[SMALLMID_NUM_CLASSES] = {
SMALLMID_TLS_CAPACITY_256B, SMALLMID_TLS_CAPACITY_256B,
SMALLMID_TLS_CAPACITY_512B, SMALLMID_TLS_CAPACITY_512B,
SMALLMID_TLS_CAPACITY_1KB, SMALLMID_TLS_CAPACITY_1KB
SMALLMID_TLS_CAPACITY_2KB,
SMALLMID_TLS_CAPACITY_4KB
}; };
return (class_idx >= 0 && class_idx < SMALLMID_NUM_CLASSES) ? capacities[class_idx] : 0; return (class_idx >= 0 && class_idx < SMALLMID_NUM_CLASSES) ? capacities[class_idx] : 0;
} }