By that reasoning Nazi Germany and Fascist Italy weren't far right, because a very significant portion of their population actually voted for that. Or France now, our "Rassemblement National" used to be far right, but now enough people (about a third) vote for them that they no longer are.

Sorry if that feels like a strawman, but I find the idea of using popularity to determining what counts as "far" stupid and dangerous.