I feel like NAND process tech could become useful at solving some of these problems. A GPU where you can update the weights a few thousand times may be sufficient.
The basis of Taalas is "compute in memory" electronics - past Von Neumann's separation of processor and memory.
You need to be able to add|mul where the data (the weights) are stored.
NAND hasn't been scaling great lately. It seems like PCM or MRAM would both be better fits.
FPGA model storage?