Researchers have introduced LexReward, a taxonomy-driven framework for reward modeling in legal language models, according to a paper posted on Hugging Face. The work argues that legal responses demand reward signals that go beyond simple answer correctness to capture multidimensional quality.
LexReward evaluates legal response quality along three complementary dimensions: Style, which covers lexical and syntactic quality; Element, which assesses legal subjects, facts, statutes, and decisions; and Chain, which examines the order, completeness, correctness, and non-redundancy of legal reasoning. For each dimension, the authors develop rubrics specifying evaluation criteria and quality levels.
The resulting rewards are used to build pairwise preference data for Direct Preference Optimization (DPO) and for training a family of reward models called LexRM. The paper reports that rubric-based rewards reliably distinguish legal responses of different quality, and that DPO training on the preference data improves performance across all three dimensions.
The learned reward models also support downstream optimization: each dimension-specific reward model improves policy performance in its corresponding dimension through reinforcement learning, without requiring reference answers at reward time. Dimension-wise analyses further support the effectiveness of the proposed taxonomy and reward construction.