If it’s not easily verifiable, LLMs aren’t good at it.
I think that’s mostly because they get so much more of that reinforcement learning - since it is so economical. I dont know if there is any evidence of a fundamental reason they can’t be just as good at other tasks, but it might be economically infeasible for awhile yet.
No one is curating vast amounts of data for them in other domains. Programmers send programs with fixes
loading story #48259384
RLVR doesn’t work for unverifiable tasks, so they won’t be able to effectively use tools to boost reliability for those tasks.
but what does it mean to be good at something that cant be verified. how do you know that they are not good at it, you are obviously using some measure.
sounds like an oxymoron of a claim.
It means having taste. People say Picasso was a great painter, but that cannot be verified (at least, not in the sense of a verified reward).