The tooling around LLMs is still nascent and, in my opinion, quite poor. You may be able to shotgun easily-falsifiable mathematical proofs with raw compute, but get into the messy world of real data, uncertainty quantification, and a host of other things that go into building real world models, and I think that an LLM plus tooling dedicated to the task plus a human in the loop will open up capabilities not available to an LLM in a chat or development centric software environment.

I don't believe this is true with current frontier coding agents. Or at least, it's becoming increasingly less true, such that I think it will effectively be nullified in ~6 months.

LLMs are the perfect tool for "finding the right words". They are literally translation machines.

Just ask the LLM for these words, obviously !

So what? The average joe shouldn't be trying to diagnose people with cancer.