#DPO

Text Degeneration in LLMs: The Hidden Production Cost Inflating Inference by 42%

A structural failure mode in autoregressive language models causes fewer than 3% of requests to consume nearly half of total inference time. New research from DharmaOCR shows the problem is built into training objectives—and proposes a fix grounded in the training distribution itself.