Neural networks in general are here to stay. Whether that is the current day frontier-model style LLM or not doesn't really matter. We've already had multiple different Deep Learning/Machine Learning architectures that have been incredibly applicable to multiple different domains.

Especially right now in the RAM-pocalpyse where it is difficult to get any machine, yes, these older GPUs are still very very useful. A guy at my company was having trouble getting compute allocated, budgeted, and approved - so eventually he went and dug up a bunch of old Volta-based GPUs, setup a local compute farm, and is getting work done. They are absolutely slower than current state-of-the-art, but they are still better than no compute at all.

I think one of the silver-linings of the current hardware high prices and shortages, is that people are going to (A) focus on software efficiency (B) not be so obsessed with having the latest and greatest hardware. Even on the consumer side, how many games ACTUALLY can utilize a RTX 5090?