I think it's true that engineers who want to understand every layer of everything in depth, or who want to have platform ownership, are not necessarily the same group as the more "product itself" focused sort who want to write something and just push it, I don't actually think I'm sold at all that any of these groups, in a vacuum, have substantial demand for GPU compute unless that's someone's area of interest for a pet project.
This. Personally, I’d want a GPU to self host whatever model, because I think that’s fun, plain and simple. Probably many people do too. But the business is not making money from people who are just thinking about fun.