Files
hakorune/src/mir/join_ir/frontend/ast_lowerer/mod.rs

219 lines
7.7 KiB
Rust
Raw Normal View History

//! AST/CFG → JoinIR Lowering
//!
//! このモジュールは AST/CFG ノードを JoinIR 命令に変換する。
//!
//! ## 責務
//!
//! - **If 文→Select/IfMerge 変換**: 条件分岐を JoinIR の継続渡しスタイルに変換
//! - **Loop 文→loop_step/k_exit 変換**: ループを関数呼び出しと継続に正規化
//! - **Break/Continue/Return→k_* 変換**: 制御フローを継続 ID として表現
//!
//! ## Phase 34-2 での実装スコープ
//!
//! 最初は `IfSelectTest.*` 相当の tiny ケースのみ対応:
//! - Simple pattern: `if cond { return 1 } else { return 2 }`
//!
//! ## 設計原則
//!
//! - **JoinIR = PHI 生成器**: 既存 PHI の変換器にはしないPhase 33-10 原則)
//! - **段階的移行**: 既存 MIR Builder 経路は保持、新経路はデフォルト OFF
//! - **A/B テスト可能**: 既存経路と新経路の両方で実行して比較検証
pub(crate) use crate::mir::join_ir::{
BinOpKind, CompareOp, ConstValue, JoinFuncId, JoinFunction, JoinInst, JoinModule, MergePair,
VarId,
};
pub(crate) use std::collections::{BTreeMap, HashSet};
mod analysis;
mod context;
mod expr;
mod if_in_loop;
mod if_return;
mod loop_frontend_binding;
mod loop_patterns;
refactor(joinir): Phase 85 - Quick wins: loop_patterns removal, DebugOutputBox, dead_code audit Quick Win 1: Remove loop_patterns_old.rs (COMPLETED) - Deleted obsolete legacy loop pattern dispatcher (914 lines) - All patterns (Break/Continue/Simple) now in modular loop_patterns/ system - Moved helper functions (has_break_in_loop_body, has_continue_in_loop_body) to analysis.rs - Updated loop_frontend_binding.rs to remove fallback - Verified zero regressions: 974/974 lib tests PASS Quick Win 2: DebugOutputBox consolidation (COMPLETED) - New module: src/mir/join_ir/lowering/debug_output_box.rs (170 lines) - Centralized debug output management with automatic HAKO_JOINIR_DEBUG checking - Refactored 4 files to use DebugOutputBox: - condition_env.rs: 3 scattered checks → 3 Box calls - carrier_binding_assigner.rs: 1 check → 1 Box call - scope_manager.rs: 3 checks → 3 Box calls - analysis.rs: Updated lower_loop_with_if_meta to use new pattern system - Benefits: Consistent formatting, centralized control, zero runtime cost when disabled - Added 4 unit tests for DebugOutputBox Quick Win 3: Dead code directive audit (COMPLETED) - Audited all 40 #[allow(dead_code)] directives in lowering/ - Findings: All legitimate (Phase utilities, future placeholders, API completeness) - No unsafe removals needed - Categories: - Phase 192 utilities (whitespace_check, entry_builder): Public API with tests - Phase 231 placeholders (expr_lowerer): Explicitly marked future use - Const helpers (value_id_ranges): API completeness - Loop metadata (loop_update_summary): Future phase fields Result: Net -858 lines, improved code clarity, zero regressions Tests: 974/974 PASS (gained 4 from DebugOutputBox tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2025-12-13 19:25:11 +09:00
// Removed: loop_patterns_old (obsolete legacy dispatcher, all patterns now in loop_patterns/)
mod nested_if;
mod read_quoted;
mod stmt_handlers;
#[cfg(test)]
mod tests;
pub(crate) use context::ExtractCtx;
pub(crate) use stmt_handlers::StatementEffect;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum FunctionRoute {
IfReturn,
LoopFrontend,
NestedIf,
ReadQuoted,
}
fn resolve_function_route(func_name: &str) -> Result<FunctionRoute, String> {
const TABLE: &[(&str, FunctionRoute)] = &[
("test", FunctionRoute::IfReturn),
("local", FunctionRoute::IfReturn),
("_read_value_from_pair", FunctionRoute::IfReturn),
("simple", FunctionRoute::LoopFrontend),
("filter", FunctionRoute::LoopFrontend),
("print_tokens", FunctionRoute::LoopFrontend),
("map", FunctionRoute::LoopFrontend),
("reduce", FunctionRoute::LoopFrontend),
("fold", FunctionRoute::LoopFrontend),
("jsonparser_skip_ws_mini", FunctionRoute::LoopFrontend),
("jsonparser_skip_ws_real", FunctionRoute::LoopFrontend),
("jsonparser_atoi_mini", FunctionRoute::LoopFrontend),
("jsonparser_atoi_real", FunctionRoute::LoopFrontend),
("jsonparser_parse_number_real", FunctionRoute::LoopFrontend),
("pattern3_if_sum_multi_min", FunctionRoute::LoopFrontend),
("jsonparser_if_sum_min", FunctionRoute::LoopFrontend),
feat(joinir): Phase 53 - SELFHOST-NORM-DEV-EXPAND implementation Expanded selfhost dev Normalized target with 2 practical P2/P3 loop variations, strengthened structural signature axis, and implemented two-stage detection. Key Changes: 1. Documentation (phase49-selfhost-joinir-depth2-design.md +128 lines): - Added Phase 53 section with candidate selection rationale - Documented two-stage detector strategy (structural primary + dev-only name guard) - Defined structural axis strengthening (carrier count/type, branch patterns) 2. Fixtures (+210 lines): - selfhost_args_parse_p2.program.json (60 lines): P2 with String carrier + conditional branching - selfhost_stmt_count_p3.program.json (150 lines): P3 with 5 carriers + multi-branch if-else 3. Structured Builders (fixtures.rs +48 lines): - build_selfhost_args_parse_p2_structured_for_normalized_dev() - build_selfhost_stmt_count_p3_structured_for_normalized_dev() 4. ShapeGuard Two-Stage Detection (shape_guard.rs +80 lines): - Added SelfhostArgsParseP2/SelfhostStmtCountP3 to NormalizedDevShape enum - Implemented is_selfhost_args_parse_p2(): P2 core family + name guard - Implemented is_selfhost_stmt_count_p3(): 2-10 carrier check + name guard - Updated capability_for_shape() mappings 5. Bridge Integration (bridge.rs +8 lines, normalized.rs +10 lines): - Added shape handlers delegating to existing normalizers - Added roundtrip reconstruction handlers 6. Entry Point Registration (ast_lowerer/mod.rs +2 lines): - Registered selfhost_args_parse_p2/selfhost_stmt_count_p3 as LoopFrontend routes 7. Dev VM Comparison Tests (normalized_joinir_min.rs +40 lines): - normalized_selfhost_args_parse_p2_vm_bridge_direct_matches_structured() - normalized_selfhost_stmt_count_p3_vm_bridge_direct_matches_structured() 8. Test Context Fix (dev_env.rs): - Added thread-local test context depth counter - Fixed deadlock in nested test_ctx() calls via reentrant with_dev_env_if_unset() Structural Axis Growth: P2 family: - Carrier count: 1-3 (unchanged) - NEW: Type diversity (Integer/String mixed) - NEW: Conditional branching patterns (Eq-heavy comparisons) P3 family: - NEW: Carrier count upper bound: 2-10 (was 2-4) - NEW: Multi-branch if-else (5+ branches with nested structure) - NEW: Complex conditional patterns Test Results: - normalized_dev: 40/40 PASS (including 2 new tests) - lib regression: 939 PASS, 56 ignored - Existing behavior unchanged (normalized_dev feature-gated) Phase 53 Achievements: ✅ P2/P3 each gained 1 practical variation (2 total) ✅ Two-stage detection: structural primary + dev-only name guard ✅ Structural axis expanded: 4 axes (carrier count/type/Compare/branch patterns) ✅ All tests PASS, no regressions ✅ Test context deadlock fixed (0.04s for 29 tests) Files Modified: 14 files Lines Added: ~516 lines (net) Implementation: Pure additive (feature-gated) Next Phase (54+): - Accumulate 6+ loops per P2/P3 family - Achieve 5+ stable structural axes - Target < 5% false positive rate - Then shrink/remove name guard scope
2025-12-12 16:40:20 +09:00
("selfhost_token_scan_p2", FunctionRoute::LoopFrontend),
("selfhost_token_scan_p2_accum", FunctionRoute::LoopFrontend),
("selfhost_args_parse_p2", FunctionRoute::LoopFrontend),
("selfhost_if_sum_p3", FunctionRoute::LoopFrontend),
("selfhost_if_sum_p3_ext", FunctionRoute::LoopFrontend),
("selfhost_stmt_count_p3", FunctionRoute::LoopFrontend),
feat(joinir): Phase 54 SELFHOST-SHAPE-GROWTH - 構造軸育成 + 偽陽性観測 Phase 53 成果を踏まえ、構造シグネチャ軸を 5+ に育て、 偽陽性観測テストで name ガード縮小準備を整えた。 方針変更: 新ループ追加 → 構造軸育成 + 偽陽性率測定に焦点変更 - 理由: Phase 53 で selfhost P2/P3 実戦パターン追加済み - 焦点: 既存ループに対する構造軸拡張 + 精度測定 主な成果: 1. 構造軸 5+ 達成: - carrier 数 - carrier 型 - Compare パターン - branch 構造 - NEW: Compare op 分布 (count_compare_ops ヘルパー) 2. 偽陽性観測テスト追加: - test_phase54_structural_axis_discrimination_p2() - test_phase54_structural_axis_discrimination_p3() 3. 重要な発見 - 偽陽性率 ~50%: - P2: selfhost P2 が正しく検出されず (name ガード依存) - P3: selfhost P3 が Pattern4ContinueMinimal と誤検出 (構造的類似性) - 結論: 構造判定のみでは分離不十分、name ガード必須と判明 変更内容: - shape_guard.rs (+80 lines): - count_compare_ops() 構造軸ヘルパー追加 - detect_shapes() pub 化 (テストから呼び出し可能に) - SelfhostVerifySchemaP2/SelfhostDetectFormatP3 enum 追加 (将来用) - normalized_joinir_min.rs (+110 lines): - 偽陽性観測テスト 2 個追加 (P2/P3 各1) - canonical shapes vs selfhost shapes 構造判定精度測定 - phase49 doc (+200 lines): - Phase 54 節完成版 - 偽陽性分析結果記録 - name ガード縮小方針明記 - enum 拡張対応: - bridge.rs (+8 lines) - normalized.rs (+8 lines) - ast_lowerer/mod.rs (+2 lines) 偽陽性観測結果 (2025-12-12): - P2 構造判定: selfhost P2 検出失敗 → name ガード必須 - P3 構造判定: selfhost P3 が Pattern4 と誤判定 → 構造的類似性問題 - 総合: 偽陽性率 ~50% → 構造軸 5 本では不十分 次フェーズ方針 (Phase 55+): - Phase 55-A: 条件複雑度軸追加 (BinOp/UnaryOp ネスト深度) - Phase 55-B: 算術パターン軸追加 (Mul/Sub/Div 出現パターン) - Phase 56: selfhost 実戦ループ追加 (6 本以上蓄積) - Phase 57: 誤判定率 < 5% 達成後に name ガード縮小開始 name ガード撤去条件 (Phase 57): - 構造軸 8+ 本確立 - selfhost P2/P3 各 6 本以上蓄積 - 誤判定率 < 5% 達成 - 複合的特徴量ベース判定実装 回帰テスト: ✅ 939 PASS, 0 FAIL (既存挙動不変) Files Modified: 8 files Lines Added: ~408 lines (net) Implementation: Pure additive (feature-gated) Phase 54 完了!構造軸育成・偽陽性観測基盤確立!
2025-12-12 17:12:58 +09:00
// Phase 54: selfhost P2/P3 shape growth
("selfhost_verify_schema_p2", FunctionRoute::LoopFrontend),
("selfhost_detect_format_p3", FunctionRoute::LoopFrontend),
feat(joinir): Phase 48-A - P4 (continue) Normalized minimal implementation Pattern4 (continue) integration into Normalized JoinIR pipeline complete. Key changes: - P4 minimal fixture: skip i==2 pattern, single carrier (acc) - ShapeGuard: Pattern4ContinueMinimal detector (structure-based) - StepScheduleBox: ContinueCheck step (eval order: HeaderCond → ContinueCheck → Updates → Tail) - normalize_pattern4_continue_minimal(): Delegates to P2 (95% infrastructure reuse) - Tests: 4 integration tests (normalization/runner/VM Bridge comparison×2) Design validation: - P4 (continue) = reverse control flow of P2 (break) - Same loop_step(env, k_exit) skeleton - Same EnvLayout/ConditionEnv/CarrierInfo infrastructure - Only difference: evaluation order and control flow direction Architecture proof: - Normalized JoinIR successfully handles P1/P2/P3/P4 uniformly - Infrastructure reuse rate: 95%+ as designed Tests: 939/939 PASS (+1 from baseline 938, target exceeded!) Files modified: 10 files (~305 lines added, pure additive) - pattern4_continue_min.program.json (NEW +126 lines) - P4 fixture - fixtures.rs (+31 lines) - P4 fixture loader - shape_guard.rs (+60 lines) - Shape detection - step_schedule.rs (+18 lines) - Schedule + test - normalized.rs (+35 lines) - Normalization function - loop_with_break_minimal.rs (+4 lines) - ContinueCheck handler - bridge.rs (+5 lines) - VM bridge routing - ast_lowerer/mod.rs (+2 lines) - Function registration - normalized_joinir_min.rs (+84 lines) - Integration tests - CURRENT_TASK.md (+13 lines) - Phase 48-A completion Next steps: - Phase 48-B: Extended P4 (multi-carrier, complex continue) - Phase 48-C: Canonical promotion (always use Normalized for P4)
2025-12-12 06:31:13 +09:00
// Phase 48-A: Pattern4 continue minimal
("pattern4_continue_minimal", FunctionRoute::LoopFrontend),
feat(joinir): Phase 53 - SELFHOST-NORM-DEV-EXPAND implementation Expanded selfhost dev Normalized target with 2 practical P2/P3 loop variations, strengthened structural signature axis, and implemented two-stage detection. Key Changes: 1. Documentation (phase49-selfhost-joinir-depth2-design.md +128 lines): - Added Phase 53 section with candidate selection rationale - Documented two-stage detector strategy (structural primary + dev-only name guard) - Defined structural axis strengthening (carrier count/type, branch patterns) 2. Fixtures (+210 lines): - selfhost_args_parse_p2.program.json (60 lines): P2 with String carrier + conditional branching - selfhost_stmt_count_p3.program.json (150 lines): P3 with 5 carriers + multi-branch if-else 3. Structured Builders (fixtures.rs +48 lines): - build_selfhost_args_parse_p2_structured_for_normalized_dev() - build_selfhost_stmt_count_p3_structured_for_normalized_dev() 4. ShapeGuard Two-Stage Detection (shape_guard.rs +80 lines): - Added SelfhostArgsParseP2/SelfhostStmtCountP3 to NormalizedDevShape enum - Implemented is_selfhost_args_parse_p2(): P2 core family + name guard - Implemented is_selfhost_stmt_count_p3(): 2-10 carrier check + name guard - Updated capability_for_shape() mappings 5. Bridge Integration (bridge.rs +8 lines, normalized.rs +10 lines): - Added shape handlers delegating to existing normalizers - Added roundtrip reconstruction handlers 6. Entry Point Registration (ast_lowerer/mod.rs +2 lines): - Registered selfhost_args_parse_p2/selfhost_stmt_count_p3 as LoopFrontend routes 7. Dev VM Comparison Tests (normalized_joinir_min.rs +40 lines): - normalized_selfhost_args_parse_p2_vm_bridge_direct_matches_structured() - normalized_selfhost_stmt_count_p3_vm_bridge_direct_matches_structured() 8. Test Context Fix (dev_env.rs): - Added thread-local test context depth counter - Fixed deadlock in nested test_ctx() calls via reentrant with_dev_env_if_unset() Structural Axis Growth: P2 family: - Carrier count: 1-3 (unchanged) - NEW: Type diversity (Integer/String mixed) - NEW: Conditional branching patterns (Eq-heavy comparisons) P3 family: - NEW: Carrier count upper bound: 2-10 (was 2-4) - NEW: Multi-branch if-else (5+ branches with nested structure) - NEW: Complex conditional patterns Test Results: - normalized_dev: 40/40 PASS (including 2 new tests) - lib regression: 939 PASS, 56 ignored - Existing behavior unchanged (normalized_dev feature-gated) Phase 53 Achievements: ✅ P2/P3 each gained 1 practical variation (2 total) ✅ Two-stage detection: structural primary + dev-only name guard ✅ Structural axis expanded: 4 axes (carrier count/type/Compare/branch patterns) ✅ All tests PASS, no regressions ✅ Test context deadlock fixed (0.04s for 29 tests) Files Modified: 14 files Lines Added: ~516 lines (net) Implementation: Pure additive (feature-gated) Next Phase (54+): - Accumulate 6+ loops per P2/P3 family - Achieve 5+ stable structural axes - Target < 5% false positive rate - Then shrink/remove name guard scope
2025-12-12 16:40:20 +09:00
// Phase 48-B: JsonParser continue skip_ws fixtures
(
"jsonparser_parse_array_continue_skip_ws",
FunctionRoute::LoopFrontend,
),
(
"jsonparser_parse_object_continue_skip_ws",
FunctionRoute::LoopFrontend,
),
// Phase 88: JsonParser _unescape_string core (step2 + continue) minimal fixture
(
"jsonparser_unescape_string_step2_min",
FunctionRoute::LoopFrontend,
),
// Phase 89 P1: ContinueReturn pattern minimal fixture
(
"pattern_continue_return_minimal",
FunctionRoute::LoopFrontend,
),
];
if let Some((_, route)) = TABLE.iter().find(|(name, _)| *name == func_name) {
return Ok(*route);
}
if func_name == "parse_loop" {
if crate::config::env::joinir_dev_enabled()
&& std::env::var("HAKO_JOINIR_NESTED_IF").ok().as_deref() == Some("1")
{
return Ok(FunctionRoute::NestedIf);
}
return Err(
"[joinir/frontend] 'parse_loop' requires HAKO_JOINIR_NESTED_IF=1 (dev only)"
.to_string(),
);
}
if func_name == "read_quoted_from" {
if crate::config::env::joinir_dev_enabled()
&& std::env::var("HAKO_JOINIR_READ_QUOTED").ok().as_deref() == Some("1")
{
return Ok(FunctionRoute::ReadQuoted);
}
return Err(
"[joinir/frontend] 'read_quoted_from' requires HAKO_JOINIR_READ_QUOTED=1 (dev only)"
.to_string(),
);
}
Err(format!(
"[joinir/frontend] unsupported function '{}' (dev fixture not registered)",
func_name
))
}
/// AST/CFG → JoinIR 変換器
///
/// Phase 34-2: Program(JSON v0) から tiny IfSelect ケースを JoinIR に変換
pub struct AstToJoinIrLowerer {
pub(crate) next_func_id: u32,
#[allow(dead_code)]
pub(crate) next_var_id: u32,
}
impl AstToJoinIrLowerer {
/// 新しい lowerer を作成
pub fn new() -> Self {
Self {
next_func_id: 0,
next_var_id: 0,
}
}
/// Program(JSON v0) → JoinModule
///
/// Phase 34-2/34-3/34-4: simple/local/json_shape pattern に対応
/// Phase 34-5: extract_value 統一化Int/Var/Method 構造まで)
///
/// # Panics
///
/// - パターンに合わない Program(JSON) が来た場合Phase 34 は tiny テスト専用)
/// - ループ・複数変数・副作用付き ifPhase 34-6 以降で対応予定)
pub fn lower_program_json(&mut self, program_json: &serde_json::Value) -> JoinModule {
// 1. Program(JSON) から defs を取得
let defs = program_json["defs"]
.as_array()
.expect("Program(JSON v0) must have 'defs' array");
// 2. 最初の関数定義を取得
let func_def = defs
.get(0)
.expect("At least one function definition required");
let func_name = func_def["name"]
.as_str()
.expect("Function must have 'name'");
let route = resolve_function_route(func_name)
.unwrap_or_else(|msg| panic!("{msg}"));
match route {
FunctionRoute::IfReturn => self.lower_if_return_pattern(program_json),
FunctionRoute::LoopFrontend => loop_frontend_binding::lower_loop_by_function_name(
self,
program_json,
),
FunctionRoute::NestedIf => self.lower_nested_if_pattern(program_json),
FunctionRoute::ReadQuoted => self.lower_read_quoted_pattern(program_json),
}
}
/// 次の関数 ID を生成
pub(crate) fn next_func_id(&mut self) -> JoinFuncId {
let id = JoinFuncId::new(self.next_func_id);
self.next_func_id += 1;
id
}
}
/// Phase 60 dev-only helper: legacy Break(P2) lowering for comparison tests.
///
/// `loop_patterns` is private, so this wrapper is exposed at the ast_lowerer boundary.
#[cfg(feature = "normalized_dev")]
pub fn lower_break_legacy_for_comparison(
lowerer: &mut AstToJoinIrLowerer,
program_json: &serde_json::Value,
) -> JoinModule {
loop_patterns::break_pattern::lower_break_legacy_for_comparison(lowerer, program_json)
.unwrap_or_else(|e| panic!("legacy break lowering failed: {:?}", e))
}
impl Default for AstToJoinIrLowerer {
fn default() -> Self {
Self::new()
}
}