Fine-tuning instruct models can boost performance on a target task while causing the model to drift behaviorally from its reference version, often degrading capabilities it already had. A new paper, Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models, flips the usual framing: instead of treating drift as an uncontrolled side effect, the authors specify a behavioral drift budget before optimization and ask how to maximize target-task performance within that budget.
The work argues that, locally, behavioral drift induces a shared geometry anchored at the reference model, with the drift budget defining a boundary in that space. Because drift fixes the distance from the reference, the remaining degree of freedom is the update direction. This reformulates fine-tuning as a direction-selection problem and predicts that changing which directions are accessible can qualitatively alter the outcome.
The authors test this in a stringent QA-only setting, where strong instruct models are fine-tuned only on final answers but must still generate multi-step reasoning at inference. Despite this mismatch, a coarse layer-selective probe reverses the failure of QA-only fine-tuning and reveals effective directions, with multiple neighboring configurations improving target performance while preserving reasoning and general capabilities.
Across Qwen3-8B and Qwen3-14B, these directions substantially improve scientific reasoning and multilingual translation. Over more than 100 languages, the resulting models match or outperform dedicated translation systems and provide a stronger initialization for subsequent reinforcement learning, according to the paper.
The authors conclude that fine-tuning is not just about how much a model changes, but how that change is spent. Code and model collections are available via the project's GitHub and Hugging Face collection links.