Skip to content
Main Site News Console

Token Pooling-Aware Training for Multi-Vector Embedding Models

· Hugging Face Translated
教程模型卡

Published on November 11, 2026

Image 8: Tom Aarsen's avatar

Tom Aarsen tomaarsen

Multi-vector embedding models give each query token an independent chance to find a match within a document. While this makes them exceptionally expressive for search tasks, the number of document vectors quickly balloons. A corpus of millions of documents can easily scale to hundreds of millions of vectors!

Token pooling shrinks these representations by merging similar token vectors. However, the more aggressively you compress, the easier it is to lose useful information. Pooling-aware training teaches the encoder to preserve critical details when vectors are merged, helping you achieve better retrieval performance within the same storage budget.

In this post, I will explain how pooling works, train a model for compressed retrieval using Sentence Transformers, and explore the resulting quality-versus-storage trade-offs. At pool_factor=4, my conventionally trained model retains 94.6% of its unpooled NanoBEIR nDCG@10 performance while reducing document vectors by 4x. Under the exact same storage budget, token pooling-aware training boosts this retention rate to 97.8%! Let’s dive into how it works.

This article builds on the pooling-aware training explored in Stefan Josef’s Learn to Pool and LightOn’s pooling regularization research (the latter also explores targeted training and adaptive budgets). Here, I will focus on understanding the method and putting it into practice through a concise, reproducible Sentence Transformers example.

Image 9: Mean NanoBEIR nDCG@10 versus stored document vectors for pool factors 2, 3, 4, and 6 and unpooled retrieval, comparing ordinary and pooling-aware training.

Pooling-aware training improves retrieval quality under the same vector budget. Details below.

Table of Contents

Why Multi-Vector Retrieval Needs So Many Vectors

Single-vector embedding models compress an entire document into a single vector. In contrast, multi-vector models like ColBERT retain a set of contextualized token vectors, allowing different parts of a query to match against different parts of a document.

For instance, a query for a “laptop with long battery life” can match evidence about laptops in one section of a document and evidence about battery life in another. Through the MaxSim (maximum similarity) scoring mechanism, each query vector selects its best match