Catching Bugs Before They Land: JiTTesting’s Human-on-the-Loop Approach at Full Scale

Why it matters

Meta reports that its Just-in-Time Tests system, which uses large language models to generate executable tests that catch bugs before code changes land, has moved from research prototype into production infrastructure. A paper, 'Just-in-Time Catching Test Generation at Meta,' details the deployment. The system combines two test-generation workflows with an LLM-as-Judge ensemble and a rule-based assessor to filter false positives, while maintaining reduced human review load. The story matters because it illustrates a shift in how large software organizations apply language models to quality assurance. Rather than having models review code the way a human reviewer would, Meta generates tests that demonstrate concrete behavioral differences introduced by a change, an approach the company says catches classes of bugs that review-style tools might miss. The deployment also shows how automated systems can be layered with deterministic checks and human oversight to manage false positives at scale, a persistent problem for AI-assisted testing. For readers tracking enterprise adoption of AI in software development, this is a detailed account of a production deployment rather than a lab experiment.