half a gigabyte not terabyte lol
Yeah, this thing is unusual. It resembles a consumer GPU but using normal DRAM with more channels, so then you get more RAM but less memory bandwidth than GDDR/HBM, which is the same thing Apple does. And the TDP is pretty attractive for what it is.
But what is that amount of SRAM for? It simultaneously seems like a lot, because SRAM is so expensive, but still not very much, because what can you fit in 0.5GB? Is there a specific workload that needs something that looks like this?
0.5 GB of SRAM is exactly what the Cuckatoo32 [1] Proof-of-Work scheme needs for optimal solving efficiency. Solutions are 42-cycles in random bipartite graphs of 2^32 edges on 2^32 + 2^32 nodes, which can be found by repeatedly removing edges that have only one adjacent edge. Tracking which nodes have incident edges takes 2^32 (completely randomly accessed) bits which is exactly 0.5GB. Another 0.5 GB is needed to track alive edges but those can be accessed serially for which DRAM suffices.
The Intel D270, Nvidia DGX Spark, and (somewhat) AMD Strix Halo also use the same approach, so it seems to be getting pretty common.
SRAM wise it doesn't seem too insane. About the same ratio of memory:SRAM as the 5090 (IIRC). Still more than I would have expected. I assume they found it helps process the current layer faster or something.