Hugging Face researchers have released MameLoshnLM, the first open-source 8-billion-parameter language model built specifically for Yiddish, a language with a rich literary tradition but scarce digital resources. The project also introduces two new resources: Oytser, a high-quality Yiddish pretraining corpus, and Kashes, a multi-task benchmark covering translation, linguistic analysis, information extraction, and language understanding.
The team found that existing multilingual corpora and benchmarks are poor proxies for Yiddish, often containing noisy, machine-translated, or misclassified text. By continuing pretraining on Llama 3.1 8B with authentic Yiddish sources, MameLoshnLM outperforms open baselines of similar scale across the benchmark tasks.
Qualitative analyses show that MameLoshnLM better captures lexical and morphological patterns that define the language, compared to general-purpose multilingual models. This points to a broader failure mode of noisy web-scale multilingual data for low-resource languages.
The work provides a foundation for Yiddish NLP and a practical template for developing language models for historically rich but digitally underrepresented languages.