I don't think that's true at all. I think language carries history, and that we all come from different upbringings and life experiences that give language context.

There are words we use because they are familiar that are nevertheless hard to hear for some groups.

When that happens, one absolutely can share the impact of the words AND acknowledge no malicious intent. When I was a kid, "gay" meant bad. Tell you what, didn't feel good to hear it. "Hey, I'm gay" was met with, "Oh I didn't mean it like that". Like, sure, but also can we find another word for bad?