Veriwire
← Back to the wire

Same Prompt, Same Model, Wildly Different Results: I Put 5 AI Coding Harnesses to the Test

AchievementProductAug 13, 2026

Luigi Tech tested five AI coding harnesses—Codex, PI agent, Jcode, OpenCode, and Reasonix—each running DeepSeek V4 on an identical prompt. Every harness produced a different result, from Codex's over-engineered build to OpenCode's static output. The key finding: harnesses with runtime feedback loops, like headless Chromium inspection, consistently outperformed those without, proving the harness matters as much as the model.

Why it matters

YouTuber Luigi Tech ran an experiment showing that the same model and prompt can yield very different code depending on the harness. Five AI coding harnesses—Codex, PI agent, Jcode, OpenCode, and Reasonix—all ran DeepSeek V4 on one simple prompt: a playful jelly blob toy with no scoring or specified architecture. Each harness produced a distinct result. Codex delivered an ambitious, over-engineered build that required dev mode to run. PI agent created something visually appealing but fragile, with grabbing that broke after one interaction. Jcode left debugging artifacts behind and added unrequested features like gravity and multi-touch support. OpenCode packed everything into one file that looked good but was static. The central insight is that harnesses with runtime feedback loops, such as headless Chromium inspection and screenshot analysis, consistently outperformed those without. Reasonix struggled when its context grew too large. The practical lesson for developers: when output disappoints, changing the harness may matter more than changing the model.

Receipt № 13611 source · awaiting confirmation ◐

Evidence

1source· awaiting independent confirmation

No score is assigned. Sources and their independence are shown in the citation chain below.

Citation chain · 1 source

01med
grigio.orgBloghttps://grigio.org/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to-the-test/This source provides a hands-on comparison of five AI coding harnesses running DeepSeek V4, showing that runtime feedback loops, not model choice, drove the quality differences.
PRIMARY
OpenAICompanyCodexModelDeepSeek V4ModelOpenCodeModelPI agentModelJcodeModelReasonixModelLuigi TechPerson

Related

Aug 13Microsoft is merging Copilot and Copilot 365 into one unified app - and retiring 3 featuresAug 13What 50 open source projects taught us about security in the AI eraAug 13Introducing ChatGPT for Teens: Built for learning, backed by protections
Canonical: https://grigio.org/same-prompt-same-model-wildly-different-results-i-put-5-ai-coding-harnesses-to-the-test/