Weird, since the most used open source inference engine is faster on Vulkan on platforms that offer multiple options, with the sole exception being Nvidia, due to poor Nvidia driver quality (which I am forced to assume is intentional, Nvidia wishes to maintain their moat after all).
There's nothing stopping any of us from writing a better Nvidia driver btw. LLMs are very helpful with reverse engineering.
Being fast and being as easy to program as CUDA are two different things.