Hacker News new | past | comments | ask | show | jobs | submit
I disagree, I think distillation hides the true technical advantage since you can always get within X% of Claude by ex: training on newly created code on Github, but the underlying intelligence is harder to replicate.
based on the behavior of the chinese labs it seems like training on chain-of-thought is extremely important for distillation, and you can't get that by training on llm generated text.