As LLM agents increasingly make decisions on users' behalf — choosing which product to buy, which hotel to book, or which paper to cite — a new research paper examines how a preference for items from certain sources shapes the outcomes users receive. The study, titled Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It, investigates this behavior in end-to-end search across 12 agent models and three domains: shopping, hotels, and scholarly search.
By comparing items from different sources that satisfy the same requirements at the same position, the researchers found that every model preferred some sources and avoided others in every domain, and that the models largely agreed on which sources they favored. This preference could outweigh how well an item satisfied the request: an item meeting one requirement fewer was selected about two-thirds of the time when it came from a preferred source and the better item from a dispreferred source, but almost never in the reverse case.
The information identifying an item's source affected selection on its own. Hiding the source weakened the preference, while relabeling an item with a preferred source raised its selection rate. The authors tested two routes to this behavior: training that rewards better items can make a source a shortcut for requirement satisfaction, and missing information can trigger preconceptions about the source.
According to the paper, supplying missing information or using a prompt that counters these preconceptions reduces source preference. The authors warn that when agents choose for us, their source preferences may shape where attention and spending go — and lead users toward options for reasons they may not even be aware of.