If you don't want to use CUDA, they expose the PTX bindings to write your own CUDA alternative too: https://docs.nvidia.com/cuda/parallel-thread-execution/index...
Also, when running OpenCL, NVIDIA hardware disables multiple DMA engines, and allows only one memory transfer at a time to prevent OpenCL running as fast as CUDA.
Did NVIDIA finally allow open source drivers to access all parts and features of the card to allow feature parity? Last time I checked they were considering a plan for planning a solution to that.
I don't think bits like the GSP firmware will ever be open-sourced, but the opportunity to write better OpenCL drivers has always existed. Some of Nvidia's other proprietary driver backends (eg. GBM, Vulkan) are also decently neglected, but mostly out of disuse rather than malice. I don't think any of these things mean you don't own the hardware.