An interesting implication of this is that AI inference and training has a path to a ~3x hardware cost reduction (and maybe ~2x total cost reduction) without any technical innovation whatsoever, we just need to wait for dram supply to meet demand (either by manufacturing scaling or just waiting for the current rate of manufacturing to fill the demand spike).

I bought 96GB of RAM a couple of years ago for ~$250. That same RAM now costs $1200!

Everything I read seems to suggest that RAM capacity is going to grow at 20-25% a year, which just doesn't seem good enough. Even in consumer use cases, phones and laptops would benefit greatly by double the amount of RAM. And then obviously, the AI need is gigantic.

I don't see it going away. I mean, it may not grow as fast as now, but I don't see it growing away either. I get why the memory makers do not want to bankrupt themselves, but it feels like there's got to be some way to push that risk off onto model providers and other people in the ecosystem to allow us to grow ram capacity more like 50% per year.

I'm not moving past my DDR4 build (and the 32 GB of DDR4 2133 MHz backup chips I still have around from way back, before I got the current 3200 MHz ones) until the prices go back to being at least partially sane. This also means that CPU manufacturers are not getting my money (since the 5800X is fine for now) and I have no reason to get a new GPU either (though admittedly the B580 isn't perfect).

I find it deeply ironic, that iran has blocked helium supply- while it relies on AI created slopaganda to subvert its advesary. Its one of those afterwits of history.

Awful time for gamers and PC hobbyists not fully into AI.

I wonder why the hyperscalers aren't vertically integrating more and building their own fabs. Sure, a fab costs a billion dollars, but they're currently spending hundreds of billions of dollars purchasing chips from NVidia and others.

Bought a second hand Dell server a week ago. The entire rig with a 12-core CPU and 32GB DDR4 ecc RAM cost as much as I'd pay to buy 64 GB of DDR RAM alone. I hope there's an end to this absurdity soon enough otherwise the pain will affect other markets too. I read the other day that PC case sales have collapsed by more than 40%.

Memory manufactures sit on a war chest of IP. So even if someone has excess fab capacity and wants to get into memory manufacturing, they will have to fight an uphill battle of about a zillion patents.

Most memory companies have backroom deals to exchange tit-for-tat patent violations against each other.

Not sure how a new memory manufacture comes into being without getting sunk from licensing costs?

I really don’t want to give anyone ideas, but doesn’t this make the Nvidia 5090 an unbelievably good deal right now?

The VRAM in the 5090 is only made by one country in the world.

The 50xx series is special, because its ram is so dependent on a single commodity. It’s not like a 4090 or a 3090; their VRAM chips have been around for years.

If there’s a shortage or interruption in DDR7 VRAM, it seems like every GPU that requires it would explode in value.

I hope I don’t regret posting this because I’d really like to buy one myself…

I recently built a system at insane ddr4 prices ($2000 for 256gb). But that’s only after seeing how ddr5 prices were 3-4x that!

Good time to focus on more memory efficient means of training and inference.

SeedLM from Apple is an interesting approach for inference memory efficiency. I'd like to see someone try and build that into training so that it's not a post training compression step.

I assume that memory manufacturers don’t really care where the money is coming from, as long as the "numbers go up" game is working.

NVIDIA in their recent quarterly report stopped categorizing "Geforce" as a single category, and merged it into "Edge-Computing".

If you are a PC Gamer or PC Enthusiast as I am, then we have some dark times ahead.

It's still unclear to me: the shortage is semiconductor boules / wafers? or the shortage is semiconductor fab process step availability?

As long as the discussion seems focused on memory, I'd suspect the latter, but if its really the semiconductor boules/wafers, then I'd expect the boule growers to profit, not the memory makers, who just pass on the cost.

So which is it?

It seems to me the max memory you can buy in a laptop stagnated for the past 3 years or so.

Time to let ASML sell to the Chinese memory producers … or not.

Memory makes computation universal.

Since memory is becoming an expensive commodity, I guess the old ways of being precious on the efficient memory usage of your program (like it running on the constrained 1mb memory back then) are making a comeback.

I only feel sorrow for the electron devs, they will have a hard time.

I wish I had figured that out a year ago. MU up ~10x, SNDK up ~37x. My crystal ball is woefully under performing.

As models gain efficiency, will the need for ram cool?

A commodity rapidly increasing in price. What could go wrong?

Since January, I've been lucky and picking up various used DDR4 memory sticks for cheap-ish. I got a total of 64 GB for $180. I feel like I hit the jackpot!

Why did this happen so suddenly?

Why were tech savy investors unable to figure this out when the datacenter craze had already started?

How to explain this lag between quickly rising demand for all datacenter components besides memory?

Here’s the thing, what if memory manufacturers take this opportunity to collude and basically never reduce the price of memory below the current levels since it’s too hard for a new competitor to just rise up and undercut them? Everything I hear about is how hard and risky it is to spin up a new fab.

And by doing this, they ensure local LLMs never become feasible for the vast majority of people and AI companies solidify subscriptions forever.

I think the companies that drive up the prices here, need to pay an extra-tax to all of us. I fail to see why I now have to pay more due to the AI monster companies ruining the economy.

Anyone invested in Micron stock?

The algorithm advances are going to crash this so hard.

if we survive the bubble bursting and there isn't a "too big to fail" bailout with public money manipulation by bought politicians

we are going to have amazing cheap used hardware for a decade

[flagged]

[flagged]

[flagged]

[dead]

C