Existence proof: everything Claude Code does, including talking to LLMs on GPUs via the Anthropic API, and shelling out to other tools installed on your system, could in principle be compiled into one big self-contained binary (or Turing Machine if you prefer) that is able to take natural language in and produce compiled binaries out.
If you wanted to go a bit harder you could train Claude to output machine code directly, but that's not necessary. Whether the machine code comes straight out of an LLM or out of a compiler that is fed source code that comes from an LLM is a mere implementation detail, there's not any fundamental difference.
I am not sure we actually want natural language to be compiled directly to machine code anyway. Those rich type systems that Haskell people get excited about are actually automatic proof checkers, similar to Lean, and having an agent check its output against such type checkers greatly improves the code generated by agents. If anything, we will probably see more widespread use of the lambda cube type systems (with extensions e.g. double negative elimination) as agents become a standard programming tool; human beings tend to find such type systems very tedious and difficult to work with, but agents are good at dealing with tedious tasks.
Yet the point about DSL is interesting, in particular internal/embedded DSLs [2]. I lied a little earlier, my third programming language was actually Forth. His creator said in 2001, "When you devised an application you wrote a hundred words [i.e. functions] or so that discussed the application and you used those hundred words to write a one line definition to solve the application" [3].
I've been trying to do that exactly that in other languages, make it so that the code expresses "naturally" what it does as much as possible. When you succeed, people call your code "beautiful" or "elegant" (nobody ever told me that :-) which, I think, subsumes "naturality".
This is of course harder to do in low-level languages such as the ones I mentioned (except Basic maybe) because you get caught in details such as memory management. But high-level languages are not exempt of these problems (and contrary to Forth or Lisp, their syntax is often written in stone). Then there's essential complexity [4], which can be "unnatural" no matter what, like thinking about things in a concurrent programming context, with languages that don't natively support it like they support automatic memory management (i.e. garbage collection).
The topic is dangerously close to "readability", which is known to be 80% subjective.
[1] https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD831...
[2] https://en.wikipedia.org/wiki/Domain-specific_language#Domai...