The gains wouldn't be "free lunch", it's the result of time and effort researching optimal design and architecture.

Even if the idea of "no free lunch" was taken liberally discounting the cost of research, it would only be limiting to systems built from a foundation of optimization, but that's not the case. The foundation so far has been one of brute force scaling. Usually meaning there is lots of room for optimization.