* Men aren't directly affected by it (~50% population)
* Woman over 40 aren't generally affected by it
So woman between 18-40 who can vote are the group most affected by abortion policy. And as you point out, even they aren't directly affected until they actually need one. So the skin-in-the-game for most people is very low. Most people vote and are opinionated on it as a sort of proxy for woman's rights.
However, some issues like house affordability, crime, employment, etc are very high for skin-in-the-game. People are currently affected or know people currently affected by these issues.
I would absolutely be affected by my friend dying from something which should be preventable but has been made pretty much illegal.
I am not an 18-40 woman and I am affected by the abortion policies in my state.
Most Americans don't like abortion laws the don't take into account the health of the mother. So that type of law becomes a wedge among Republicans.
Conversely, if a state passes a six week ban (Florida), that's going to draw out these distinctions among Democrats.
I'm not making a moral claim. I'm commenting on the politics of campaigning on it. I think politicians are advantaged at avoiding wedge issues and focusing on material concerns that affect the most people.