Hacker News new | past | comments | ask | show | jobs | submit
mchusma | on: Memory has grown to nearly two-thirds of AI chip component costs
Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic.

I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to bankrupt themselves, but it feels like there's got to be some way to push that risk off onto model providers and other people in the ecosystem to allow us to grow ram capacity more like 50% per year.

regularfry | parent | next
The openai deal would be absorbed by two years of that. And it would be inefficient for the RAM makers in a competitive market to leave buyers unsold-to.

I don't actually know what the rate of growth before October was, I'm sure someone round here will though.

minraws | parent | next
I mean the biggest risk is Chinese CXML benefits and capturing markets that others are leaving hanging and then being able to compete and push out the others when costs start to normalize.

As for 20-25% growth not being enough, I think it's not that far off, if we assume data center build out plans hit a wall and slow down significantly, and the AI heat starts to cool off.

I don't think 20-25% may be enough in the short term but if the AI build out stops within this year, we have a massive oversupply instead of a under supply.

loading story #48259982
loading story #48259863
loading story #48259571
foota | parent | next
In theory the new futures markets for chip components would help here, since it would allow DRAM suppliers to insulate themselves from that risk.
DoctorOetker | parent
According to the recent article HBM memory is 3x less efficient wafer area wise than LPDDR; but the bandwidth is more than triple.

What if its in everyone's interest to buy computers at say 1/3rd the rate and switch everything over to HBM?

the discrepancy between compute and memory has been growing for ages, perhaps a painful switch to HBM is exactly what we need?

Would you rather have 3 intermediate computers with low memory bandwidth, or wait a little longer statistically so that we can all enjoy a new computer at 1/3rd the rate but much higher bandwidth than the area ratio?

loading story #48259919
loading story #48259952
loading story #48260064