Large language models (LLMs) often exhibit inconsistent performance across languages, a phenomenon known as language-specific competency (LSC). This means the same query can yield different results depending on the language used, due to misaligned semantic representations internally. Existing solutions either route all queries through English, which limits expressivity, or train on language-balanced data, which can reduce overall performance.
To address this, researchers from Hugging Face have introduced HOTFIXR (Hardness Optimized Training data For Improving X-Lingual Reasoning), a data-centric framework that probes a student model to identify its multilingual weaknesses and generates targeted synthetic training data to mitigate them. This approach aims to improve multilingual performance without the trade-offs of previous methods.
In evaluations across three in-distribution tasks, three out-of-distribution tasks, and four out-of-distribution languages, HOTFIXR demonstrated significant gains: an average improvement of 6.2% on in-distribution tasks, a 3.7% reduction in catastrophic forgetting on out-of-distribution tasks, and a 7.1% improvement on out-of-distribution languages.
The framework's ability to enhance cross-lingual reasoning while maintaining overall capability is crucial for real-world applications that require multilingual proficiency. The researchers plan to release the code upon acceptance, which could facilitate further adoption and research in this area.