User space is not the same as kernel space.

User space applications can't access hardware or physical memory. They can't bypass permissions enforced by the OS. None of that applies to hardware or kernel drivers.