I don't have a great answer except learn as much about AI as possible - the easiest starting point is Simon Prince's book - and it's free online. Maybe start submitting changes to pytorch? Get a name for yourself? I don't know.

Most companies aren't doing a lot of heavy GPU optimization. That's why deepseek was able to come out of nowhere. Most (not all) AI research basically takes the given hardware (and most of the software) stack as a given and is about architecture, loss functions, data mix, activation functions blah blah blah.

Speculation - a good amount of work will go towards optimizations in future (and at the big shops like openAI, a good amount already is).