Physicists get a failing grade for naming things.
It's not just a bad idea because of that BS, but even within the field it's just asking for trouble. We may all wish we were perfect Vulcans who have perfect mental separation between all concepts and emotions, but we aren't. It's going to have a small, but extremely persistent and long-term effect on the field if you seriously name a major part of it "magic". The emotional connotations simply can not help but smear into the putatively mathematical term. It's a high price to pay for what isn't really all that funny of a joke even the first time.
And of course the BS will crank up even higher. People get hurt by that, but I don't know how much to lay at the foot of people who are all but taunting them by naming something "magic", because most of the hurt was going to come anyhow and what particular guise it is wearing is of minimal importance. Still, why even sign up to be in the line of fire of responsibility for that sort of thing?