Maybe. I don't know power for the MI350, but the RTX PRO 6000 is normally a 200-300W card, so you pay 2x the power for 3/4th the RAM and 3x the bandwidth. The performance frontier looks very different in terms of power to hold a mid-size model and throughout per user or across users. Price and inference stack would be big factors in what I would choose.
MI350P, H200 NVL and Nvidia RTX PRO 6000 are all 600W but compute wise all of them should be 3 to 4 times more performant than A100 Ultra according to their specifications.
https://www.servethehome.com/the-amd-instinct-mi350p-is-a-hb...
600w is way off the vf curve though... Tiny bit of tuning and you'll get ~80-90% of the performance at ~150w nominal. At least that's what I got out of some pro 6000s when I spent 15 mins to underclock it.