Visual document retrieval systems typically store around a thousand vectors per page, a cost that late-interaction methods accept in exchange for state-of-the-art accuracy. A new research paper, Generative Late-Interaction Embeddings For Visual Document Retrieval, proposes a way to shrink that storage burden without retraining the underlying encoder.
The authors studied the geometry of these embeddings across three encoders and found two consistent properties: the vectors lie exactly on the unit sphere, and they concentrate near a manifold with an intrinsic dimension of only five to six. This led to two practical insights. First, standard k-means centroids fall inside the sphere, which systematically underestimates MaxSim scores; normalizing them to the surface is a free correction worth up to +0.093 nDCG@5 over raw centroids. Second, because the page manifold has few degrees of freedom, the full set of vectors can be regenerated from just a few.
Based on this, the team introduces Generative Late-Interaction Embeddings (GLIE): k vectors per page, where k is much smaller than N, learned from the normalized centroids to serve as both a lightweight index and a basis for regenerating the page's full embedding set. At query time, search runs exclusively on these k vectors, and a decoder expands only the top candidates back to all N vectors for exact rescoring.
On ViDoRe v1, GLIE retains nearly 80% of the uncompressed system's nDCG@5 at four vectors per page, compared with 70% for the best prior post-hoc method. The results use a 415K-parameter network fitted in under three GPU-minutes on just a thousand training pages. At a matched training budget, fine-tuning the encoder does not reach even the training-free stage of GLIE, and the full system beats it at every budget. The patterns hold across a second encoder and ViDoRe v2.
By reconstructing evidence on demand rather than sampling it, GLIE opens a new axis for storage-efficient retrieval, with the decoder as its main design surface. In the reported evaluation, only the top 20 candidates are expanded and reranked with MaxSim.