Google Research has released Agile Deliberation, a new framework designed to build image classifiers for subjective categories. The system guides users through an interactive deliberation workflow using an LLM-driven optimization loop. Researchers open-sourced the underlying code and algorithms on GitHub.
The framework splits the task into concept scoping and concept iteration phases. It uses vision-language models to surface semantically borderline images to challenge user definitions. 18 live user sessions of 1.5 hours each were conducted to evaluate the framework. Participants reported significantly reduced cognitive fatigue and anxiety compared to manual prompt engineering.
The framework yielded classifiers that substantially outperformed standard baselines on challenging subjective images. This approach offers a structured way to handle ambiguous data that traditional models struggle to categorize. Users building custom models for nuanced tasks should find this tool useful for improving accuracy and reducing the mental load of prompt engineering.
