It's already happening.

https://www.cambridge.org/core/journals/international-review...

Back then the right wing were the industrialists who wanted immigration.

Hitler blamed all sorts of people, including the socialists (who were against immigration). Using Hitler to shut down any discussion about immigration is not very productive. Obviously there are limits. Less so in the U.S. because there is more space, but in Europe everyone is already living in tiny overpriced apartments on top of each other.