Hacker News new | past | comments | ask | show | jobs | submit
aunty_helen | on: GPU-unlock: FP/BF16 instruction issue rate on Nvidia 3000's cards
Thanks so much. 27b is a favourite model and last night my dev ran some benchmarks to achieve 133tks about 3.2x which changes the economics.
xenonite | parent
great to hear, which quant + KV quant + KV size are you on? is it llama.cpp?
loading story #50228135