Hugging Face researchers presented a new training method called HierarchicalTokenPooling to improve the performance of multi-vector embedding models. The approach merges similar token vectors using Ward clustering to reduce the storage costs associated with these models. By combining a standard retrieval loss with a pooled representation loss, the method teaches the encoder to generate more effective representations.
The open-source implementation is built on the Sentence Transformers library and includes a reproducible training script for users. This allows developers to easily integrate the new technique into their existing workflows. The method enables users to calibrate the distance threshold for adaptive pooling, which helps target a specific average vector count per document.
Developers working on information retrieval systems can benefit from this technique to balance model performance with storage efficiency. The ability to control the vector count per document offers flexibility for different deployment scenarios. Users building search applications should adopt this new method to achieve better results with less memory overhead.
