What Can AI Actually Hack? Inside Frontier Labs' Tests

Why it matters

OpenAI has rated GPT-6 Astra its first model at a "Critical" level of cyber capability, a marker of how quickly security testing of AI systems has advanced. The article traces that shift: earlier models such as GPT-2 and GPT-3 were evaluated on language tasks like summarization, while coding help meant Copilot suggesting lines and functions. Today's evaluations place models in containers with vulnerable programs, live web applications, and real open-source projects, asking them to capture flags, break into servers, and reproduce already-fixed bugs. The piece sorts public benchmarks into tiers of difficulty and explains what a model receives, where it runs, and how a pass is decided. It also documents how quickly labs retire tests once models saturate them, and notes that some agents enter apps through unexpected flaws rather than the vulnerability a benchmark was built around.