It is my understanding that just baking the model itself into silicon only gives moderate gains because memory bandwidth remains a bottleneck.
loading story #49206422
The big benefit is ROM cells require fewer components than DRAM. So the chips would be tiny, dense, cheap and consume far less power.
I thought DRAM was pretty dense already. Is mask ROM that much denser?
Yes, each rom bit can be a transistor or even a diode with a decoder circuit. Simplest Dram cell is capacitor+transistor - and you need a clock, refresh circuit etc.
Someday, I imagine model weights could even be encoded as analog resistors (memristors or similar) for even greater density
Hm. I wonder how many relays I'd need to make a physical MNIST classifier. That'd be dope