> it can't be too much of an effect.
That is the problem with any argument for some new physics - it might exist, but it can't have much effect or we would detect it. Generally I only see people arguing for new physics because they really want faster than light travel (typically also without all the weird time effects, but a small minority would accept it with time effects)
Also many people want to find libertarian free will somewhere in new physics.