Hugging Face has released a new research paper introducing UEmbed (Unified Embedding), a decoder-only multimodal embedding model that produces both sparse lexical and dense representations in a single causal forward pass. This innovation addresses key limitations in existing learned sparse retrieval (LSR) methods, which have traditionally relied on encoder-style architectures and auxiliary cross-modal modules for multimodal support.
The model works by appending N learnable special tokens to the input and partitioning the vocabulary into N disjoint subsets. Each token's causal hidden state predicts sparse weights over its assigned subset, and the subsets are concatenated to form the full sparse vector. This design allows UEmbed to unify dense and sparse embeddings in one model, extending sparse retrieval to handle both text and multimodal inputs seamlessly.
UEmbed is trained on public data and released in three scales: 2B, 4B, and 9B parameters. The largest variant, UEmbed-9B, achieves scores of 71.8 (dense) and 71.0 (sparse) on the MMEB-v2 benchmark, outperforming other multimodal embedding models trained on publicly available data, such as RzenEmbed. On the BEIR benchmark, UEmbed remains competitive with strong dense and sparse baselines.
The paper also demonstrates UEmbed's practical utility across three dimensions: effectiveness, efficiency, and agentic applications. By unifying sparse and dense embeddings in a single model, UEmbed offers a new paradigm for retrieval systems, potentially simplifying architectures and improving performance in applications like web search and retrieval-augmented generation.