I would assume it's grating for anybody to hear their mother tongue butchered. More so when both sides know they could just switch English and have an adult conversation instead of struggling to buy a loaf of broad and a bottle of water. I always feel the urge to switch and have to remind myself that the other person is making a big effort on their side and that should be appreciated and respected.

P.S. My mother tongue is Spanish and it's many accents are anything but standardized.