Google AI Research Releases Agile Deliberation Framework for Gray-Area Image Classifiers

Why it matters

Google Research has introduced Agile Deliberation, a human-in-the-loop framework for building image classifiers around subjective concepts such as healthy food or clickbait, where users may not know the concept boundary in advance. The system prompts Gemini 2.5 Pro and Gemini 2.5 Flash through an API without fine-tuning, turning a vague idea into a written definition that serves both as a human-readable policy and as a prompt for a vision-language classifier. Rather than relying on uncalibrated confidence scores, it retrieves borderline images by meaning using CLIP embeddings and a FAISS index, then clusters them and selects groups via a multi-armed bandit. An evaluation with participants lacking moderation experience, each comparing the tool against manual deliberation, found the framework outperformed zero-shot Gemini 2.5 Flash on average F1, with users reporting less effort and stress. The design drew on interviews with content moderation experts, and the code is available on GitHub with a demo notebook.