Hacker News new | past | comments | ask | show | jobs | submit
WithinReason | on: Do transformers need three projections? Systematic study of QKV variants
Not every one can afford millions to publish a paper
spindump8930 | parent
That's why you do several small and medium scale tests, fit a curve, and ideally show that the trend persists at several scales. Not a single large or medium run - see the other comments down thread for example sizes.