Hacker News new | past | comments | ask | show | jobs | submit
gmerc | on: I'm not convinced the AI bubble exists
That's because there's a temporary supply side crunch.
pierreb-aiva | parent | next
That only explains why the price of those GPUs at resale is high, but it doesn't invalidate the fact that those GPUs are still very useful way past a 2 to 3 years depreciation schedule advocated by Michael Burry et al.

Older GPUs can still be used to run model inference, albeit more slowly than newer GPUs. We already see frontier labs turning this insight into an opportunity to charge a premium on token delivery speed, when models run on newer hardware. As AI becomes more widely spread throughout the economy, not all workloads will require fast inference, and there will be many use cases for previous generation hardware. The only exception is if the latest batch of specialised chips become ubiquitous very quickly, but then price of compute AND intelligence per watt will both go down dramatically, which would reduce the need for sustained high capex and opex

adityaathalye | parent
Yes it is a supply-side crunch (chip manufacturing), but IMHO, it is not temporary. Only two things will remove the supply constraint.

- Demand drops like a stone: raw TFLOPS / per model. This will happen if someone figures out insanely fast and power-efficient math (specifically, multiplication) for existing chip architectures, that makes it dirt-cheap to keep re-computing ever-better SOTA LLMs, as well as to on-line compute inference with ever-bigger models (scaling laws rule). These are different workloads, AFAIK.

- Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention) and which can be manufactured at similar bin sizes, at similar scale (i.e. equivalent per-die yield), price-competitively (no more 10x the equivalent in SOTA Silicon compute).

I'd say that a math breakthrough might be more accessible than the transistor-manufacturing breakthrough.

vikramkr | root | parent | next
IDK if Moore's law restarting is possible. But I wouldn't rule out some sort of non-silicon compute breakthrough that functionally jumps us a couple orders of magnitude. The entire world economy being aggressively focused on compute seems like a good environment to drive investment into figuring out photonic computers or whatever. We do know at least one other non silicon highly energy efficient compute technology (biological brain) exists and works very well
IAmBroom | root | parent
> Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention)

That assumes that there is no hard, physical limit on Moore's Law, which is obviously false. There is a limit, and as long as we depend on mass flow of charge through conductors, it's nowhere near quantum sizes.

Where that limit is has not yet been determined, but at this point the support structure to write transistors is the size of a house trailer, using a staggering amount of toxic chemicals (just to run the exotic lasers). So, advancements today may scale very nonlinearly in cost.

The very phrase "Moore's law restarts" acknowledges that we are no longer on-track.

adityaathalye | root | parent
Yep, I mean to imply practically impossible, in context of bubble-timelines. Theoretically anything is possible given enough decades. Practically, AGI has thus far been 20 years out, as has "clean" fusion power, 1 gee EM drives etc.