It can absolutely matter, and in fact it is all the more important in a single-parent household.

You’re right that single vs. two parent household is the largest contributing factor. You’re wrong that it means that no other factors matter at all.