Published on November 11, 2026
Multi-vector embedding models give each query token an independent chance to find a match within a document. While this makes them exceptionally expressive for search tasks, the number of document vectors quickly balloons. A corpus of millions of documents can easily scale to hundreds of millions of vectors!
Token pooling shrinks these representations by merging similar token vectors. However, the more aggressively you compress, the easier it is to lose useful information. Pooling-aware training teaches the encoder to preserve critical details when vectors are merged, helping you achieve better retrieval performance within the same storage budget.
In this post, I will explain how pooling works, train a model for compressed retrieval using Sentence Transformers, and explore the resulting quality-versus-storage trade-offs. At pool_factor=4, my conventionally trained model retains 94.6% of its unpooled NanoBEIR nDCG@10 performance while reducing document vectors by 4x. Under the exact same storage budget, token pooling-aware training boosts this retention rate to 97.8%! Let’s dive into how it works.
This article builds on the pooling-aware training explored in Stefan Josef’s Learn to Pool and LightOn’s pooling regularization research (the latter also explores targeted training and adaptive budgets). Here, I will focus on understanding the method and putting it into practice through a concise, reproducible Sentence Transformers example.
Pooling-aware training improves retrieval quality under the same vector budget. Details below.
Table of Contents
Why Multi-Vector Retrieval Needs So Many Vectors
Single-vector embedding models compress an entire document into a single vector. In contrast, multi-vector models like ColBERT retain a set of contextualized token vectors, allowing different parts of a query to match against different parts of a document.
For instance, a query for a “laptop with long battery life” can match evidence about laptops in one section of a document and evidence about battery life in another. Through the MaxSim (maximum similarity) scoring mechanism, each query vector selects its best match

