Hacker News new | past | comments | ask | show | jobs | submit
strken | on: LLMs work best when the user defines their acceptance criteria first
Human behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged.

I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.

seanmcdirmid | parent | next
Isn’t a modern LLM with thinking tokens fairly goal directed? But yes, we hallucinate in our sleep while LLMs will hallucinate details if the prompt isn’t grounded enough.
zarzavat | root | parent | next
The thing about dream logic is that it can be a completely rational series of steps, but there's usually a giant plot hole which you only realise the second you wake up.

This definitely matches my experience of talking to AI agents and chatbots. They can be extremely knowledgeable on arcane matters yet need to have obvious (to humans) assumptions pointed out to them, since they only have book smarts and not street smarts.

tovej | root | parent
Assuming this is not a rhetorical question: no, it is not. The only "goal" is to maximize plausibility.
seanmcdirmid | root | parent
Again, how is that different from humans? I’m not going around trying to prove my code correct when I write it manually.
loading story #47291928
satvikpendem | parent | next
A prompt for an LLM is also a goal direction and it'll produce code towards that goal. In the end, it's the human directing it, and the AI is a tool whose code needs review, same as it always has been.
basch | root | parent
Id argue humans have some sort of parallelness going on that machines dont yet. Thoughts happening at multiple abstraction levels simultaneously. As I am doing something, I am also running the continuous improvement cycle in my head, at all four steps concurrently. Is this working, is this the right direction, does this validate?

You could build layers and layers of LLMs watching the output of each others thoughts and offering different commentary as they go, folding all the thoughts back together at the end. Currently, a group of agents acts more like a discussion than something somewhat omnipotent or omnitemporal.

whoamii | parent | next
Some of my best code comes from my dreams though.
tsunamifury | parent | next
It’s amazing how much you get wrong here. As LLM attention layers are stacked goal functions.

What they lack is multi turn long walk goal functions — which is being solved to some degree by agents.

strken | root | parent
I don't argue that thinking and attention are missing. I argue that they are trying to do the job of human executive function but aren't as good at it.
nemo44x | parent | next
LLMs are literally goal machines. It’s all they do. So it’s important that you input specific goals for them to work towards. It’s also why logically you want to break the problem into many small problems with concrete goals.
andai | root | parent
Do you only mean instruct-tuned LLMs? Or the base (pretrained) model too?
nemo44x | root | parent
The entire system and the agent loop allows for more complex goal resolution. The LLM models language (obviously) and language is goal oriented so it models goal oriented language. It’s an emergent feature of the system.
spiderfarmer | parent
And yet LLM’s are incredibly useful as they are right now.
strken | root | parent
And yet they're going to be better in a decade, which will require understanding why they aren't perfect today.