It's applied to the programming language too - the given source can be compiled to unlimited number of possible machine code representations (it can different just by NOPs).
Yes, definitely. And the same program can run on completely different architectures. But it has a well-defined semantics that must be preserved by the compiler or runtime.
I'm not saying it's all mathematical purity. Programming languages themselves have specs and sometimes they are ambiguous or allow undefined behavior. Compilers have bugs. But the gap between the source code and program behavior is typically extremely thin and language implementations work towards making it airtight.