Hacker News new | past | comments | ask | show | jobs | submit
Bonsai 2 has a bit different architecture as far I understand and speculative decoding is not so great yet. Before that point, original model is faster if you have the memory. Would be nice to see the speed with proper speculative decoder.