Google wants video AI to understand what people need as they work

Why it matters

A research team led by Google, with collaborators from Stanford, Oxford, and other U.S. universities, has introduced StreamVibe, a video agent designed to decide on its own when to offer help during everyday tasks such as cooking or assembling furniture. Existing video assistants, the researchers argue, handle only explicit commands and struggle with vague requests, changing instructions, and tracking what has already been done, leading to repeated or mistimed advice. To measure the problem, the team built a benchmark of examples drawn from many hours of everyday recordings, with Gemini 2.5 Flash suggesting candidate moments to intervene and human evaluators approving each case. The trained model decides each second whether to stay silent or speak, with reinforcement learning encouraging restraint and a memory system tracking pending requests. The results are telling: while the new model outperforms other open models, every tested system degrades sharply in continuous operation, especially over longer videos, showing how far proactive video assistance remains from reliable real-world use.