Hacker News new | past | comments | ask | show | jobs | submit
In general I think there's going to be diminishing returns on any given aspect of LLMs because not everything can be infinitely improved upon. If something can write a medium sized program near flawlessly, and I'm writing medium sized programs - then nothing else it can, or cannot, do matters.

Maybe bigger models will always just get progressively better at writing out bigger and bigger programs to the point that we're one-shotting fully functioning operating systems with wide driver support in the future, but again these features just don't matter if that's not what you want to do.

I think we're already limited by our own desires when it comes to coding.

The reason you can't one-shot something complicated is that you haven't generally considered all the important choices. You have to be asked for every choice, and complicated things have a lot of choices. But I think you can do something waterfall-esque, which used to be a big no-no in programming, if you just list out all the decisions meticulously.

I've had some success with doing that, just talking it over and thinking a lot about design decisions, making a document, and then letting it loose on the implementation. Inevitably, you find that there were contradictions or missing choices in the design, and you have to answer some questions.

So, not one-shotting, but it's my fault, not the AI's fault.

You can of course tell it to just fill in the gaps with whatever it thinks is reasonable, and then you'll have something potentially quite different from what you imagined.

The big improvement still coming down the pipe is that the models have better "taste" about filling in the things you didn't meticulously specify. A model from a year ago could make an amazing game, but you had to specify every detail precisely; now you can get similar quality results with a much more vague prompt.
That was the idea behind waterfall. You can look up how good that worked.
I think LLMs can be improved much more but most likely significantly diminishing economic returns on the investment for that improvement. Ultimately, it becomes too expensive for a few % more improvement at increasing cost. I believe many of the LLM providers are approaching that problem. So it's time for them to commercialize viable long term business, maybe keeping a team still trying to dig up improvements without blowing the bank.