A recent research paper from Hugging Face investigates how automatic speaker verification (ASV) systems can be enhanced by leveraging multiple utterances and multimodal information, posing a threat to speaker anonymization techniques.
The study, titled "Multimodal Speaker Verification as a Threat to Speaker Anonymization," notes that most ASV systems operate on single utterances, but real-world interactions involve multiple utterances. As speech accumulates, richer speaker information becomes available through acoustic, prosodic, and linguistic cues, which may challenge anonymization methods that primarily target vocal characteristics.
The researchers examined ASV in a multi-utterance, multimodal setting to see if aggregating information across anonymized speech impacts privacy. They first studied audio-only aggregation across multiple anonymized utterances and observed consistent performance improvements as more speech became available. Then, incorporating prosodic and linguistic information, they found that multimodal systems outperform unimodal approaches.
Comparing aggregation strategies, frame-level aggregation yielded the lowest equal error rates (EERs). Even with only five anonymized utterances, combining audio and text reduced EER by over 15% relative to audio-only aggregation, demonstrating that substantial speaker-discriminative information remains accessible despite anonymization.