It's surprising that Exponential Time Hypothesis would be false. Quoting a paper from 2022[1]:
> We prove that the time complexity of self-attention is necessarily quadratic in the input length, unless the Strong Exponential Time Hypothesis (SETH) is false[...]
If OpenAI's proof is correct, does that mean self-attention can be implemented in sub-quadratic time? I.e. we could eventually have all the most advanced LLMs running on cheap hardware? To put it more playfully, is ETH being false actually bearish for Nvidia?
loading story #50256574
No, SETH implies quadratic time does not mean !SETH implies !quadratic time.
loading story #50252779
SETH is a much stronger claim than ETH
loading story #50253778
loading story #50253918
loading story #50253641
It's playful!
But it's also not even wrong, in the Pauli sense.
That makes me wonder if "I see your CS proof, how does this affect LLMs / nvidia stonk price???" being immediately self-described as "playful" ensures criticism looks cranky.
loading story #50255031