🔍 Research: GPT-5-Codex capabilities and GitHub PR integration
## Summary
Investigated OpenAI's new GPT-5-Codex model and Codex GitHub PR review integration capabilities.
## GPT-5-Codex Analysis
### Benchmark Performance (Good)
- SWE-bench Verified: 74.5% (vs GPT-5's 72.8%)
- Refactoring tasks: 51.3% (vs GPT-5's 33.9%)
- Code review: Higher developer ratings
### Real-World Issues (Concerning)
- Users report degraded coding performance
- Scripts that previously worked now fail
- Less consistent than GPT-4.5
- Longer response times (minutes vs instant)
- "Creatively and emotionally flat"
- Basic errors (e.g., counting letters incorrectly)
### Key Finding
Classic case of "optimizing for benchmarks vs real usability" - scores well on tests but performs poorly in practice.
## Codex GitHub PR Integration
### Setup Process
1. Enable MFA and connect GitHub account
2. Authorize Codex GitHub app for repos
3. Enable "Code review" in repository settings
### Usage Methods
- **Manual**: Comment '@codex review' in PR
- **Automatic**: Triggers when PR moves from draft to ready
### Current Limitations
- One-way communication (doesn't respond to review comments)
- Prefers creating new PRs over updating existing ones
- Better for single-pass reviews than iterative feedback
## 'codex resume' Feature
New session management capability:
- Resume previous codex exec sessions
- Useful for continuing long tasks across days
- Maintains context from interrupted work
🐱 The investigation reveals that while GPT-5-Codex shows benchmark improvements, practical developer experience has declined - a reminder that metrics don't always reflect real-world utility\!
This commit is contained in:
@ -545,6 +545,23 @@ pub extern "C" fn nyash_console_birth_h_export() -> i64 {
|
||||
0
|
||||
}
|
||||
|
||||
|
||||
|
||||
// ArrayBox birth shim for AOT/JIT handle-based creation
|
||||
#[export_name = "nyash.array.birth_h"]
|
||||
pub extern "C" fn nyash_array_birth_h_export() -> i64 {
|
||||
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
|
||||
std::sync::Arc::new(nyash_rust::boxes::array::ArrayBox::new());
|
||||
nyash_rust::jit::rt::handles::to_handle(arc) as i64
|
||||
}
|
||||
|
||||
// MapBox birth shim for AOT/JIT handle-based creation
|
||||
#[export_name = "nyash.map.birth_h"]
|
||||
pub extern "C" fn nyash_map_birth_h_export() -> i64 {
|
||||
let arc: std::sync::Arc<dyn nyash_rust::box_trait::NyashBox> =
|
||||
std::sync::Arc::new(nyash_rust::boxes::map_box::MapBox::new());
|
||||
nyash_rust::jit::rt::handles::to_handle(arc) as i64
|
||||
}
|
||||
// ---- Process entry (driver) ----
|
||||
#[cfg(not(test))]
|
||||
#[no_mangle]
|
||||
|
||||
Reference in New Issue
Block a user