I had something similar happen in Germany, on the way from the (international) airport to a hotel the driver started some kind of spiel that suddenly his banking machine had broken and he couldn't take card payments. My friend/colleague Jaap who was with me said we'd pay cash and I said no way, and after fiddling for a bit with his phone (mine wasn't a smartphone) gave the driver a different address. When we ended up in front of the police station the driver became a lot more friendly, drove us to the hotel instead and suddenly found that his banking machine had miraculously started working again...
I find that by giving in to such fraud I'm helping to perpetrate it so I've vowed not to let it happen, at the same time there is always a chance that such an interaction would turn violent. After all, they've already decided they want to steal from you. My weighing of this is that they have more to lose than me because I'm a transient and they are not.