Generic parameter-efficient fine-tuning (PEFT) methods, originally designed for language models, often fail silently when applied to real-time object detectors. These detectors contain heterogeneous operators and detection-specific components that impose placement constraints not found in standard Transformer stacks. To address this, researchers from Tencent introduced YOLO-PEFT, a structure-aware framework that formulates adapter placement as an auditable constraint-planning problem.
The framework takes a detector graph, a PEFT request, and a resource budget as input. It assigns operator and semantic roles, evaluates explicit predicates for operator validity, detector semantics, graph interfaces, and deployment requirements, and records a reason code for each excluded module. The system then either emits a budgeted target-module plan or returns Refuse before training begins, ensuring that only viable configurations proceed.
Under the official VOC07+12 trainval-to-VOC07 test protocol, planner-selected RS-LoRA achieved 0.7138 mAP50-95 on YOLO11s and 0.7307 on YOLO12s, compared with 0.6428 and 0.6662 for full fine-tuning (Full-SFT). On RT-DETR-L, all seven evaluated LoRA-family configurations crossed a predefined catastrophic threshold, supporting a calibrated Refuse-to-Full-SFT decision within the evaluated coverage.
A controlled audit on YOLO11 showed that LoRA reduces peak training memory by 43.9%, though training takes 1.72 times longer. The framework replaces manual trial-and-error with explicit, inspectable planning while preserving verified train-save-merge-export paths. However, refusal on unseen detector architectures remains an open validation problem.