A new paper posted on Hugging Face examines whether the "lossless" guarantee claimed for Orthrus, a hybrid autoregressive-diffusion architecture for accelerating language-model inference, holds up under different numerical precisions.
Orthrus generates multiple tokens in parallel while relying on a frozen autoregressive backbone, and its central claim is that an intra-model consensus mechanism enables speculative decoding that produces the same output sequence as the reference model. The authors of the new study independently reproduced Orthrus and tested that claim.
Under BF16 inference, exact trajectory matching occurred in only 45% of cases for the original checkpoint and 43% for an independently trained model, across 1,190 prompts spanning 12 domains. The probability of exact matching was also strongly associated with the response-conditional perplexity of the reference model.
Despite this trajectory divergence, the paper reports no systematic degradation on downstream lm-eval-harness benchmarks. When the trajectory evaluation was repeated in FP32, exact matching was achieved on all evaluated prompts. The authors conclude that Orthrus's practical losslessness depends on numerical precision, and that exact trajectory equivalence should be assessed separately from downstream task performance.