AI Research
TorchSpec Cracks Open Speculative Decoding's Scaling Nightmare
Hidden states from a 1T-param monster like Kimi K2.5 are burying your training pipeline under gigabytes of data. TorchSpec says it can stream them without the usual disk or memory meltdowns— but does it actually pay off?