GPU "cores" are basically what a CPU would call SIMD lanes. So a GPU with 1024 'CUDA cores' might be structured as 16 relatively independent pieces that a CPU might call a core, each with a 64 wide SIMD unit.

GPUs work on vectors and matrices very often, that's what they are good at, so it makes a lot of sense that they can operate with SIMD I think!

There is something very SIMD-coded in GPU programming which is coalesced stores/loads, if a warp (32 threads) handles contiguous memory, it will create ~4 transactions instead of 32.

Welcome to the lucky 10,000! SIMD is actually a pretty integral part of how GPUs are able to work efficiently, it's part of why there's such a strong focus on branchless programming in the field.