That's because people were totally focused on physics, and math was just a useful tool sometimes. Doing physics was the true goal and observation the final arbiter of truth.
Nowadays, that distinction is blurred but for the opposite reason; people think that anything conceived by sound math must be true, and observation has taken a back seat.
It's actually a 19th century idea. The discovery or acceptance of non-euclidean geometry in the 19th century untethered math from physics or physics from math.
> and it seems to be disappearing in the twenty-first.
It can't disappear because math is no longer tied to the physical world. Math is simply theorem generation regardless of whether the axioms and theorems apply to the physical world.
The math used in physics is only a tiny subset of possible math.
— V.I. Arnold: "On teaching mathematics" (1997)