So, based on bitter, bitter experience around user feedback, this is generally not useful for training.
More generally, in order to make RL (or any other method) work, then we need tons of labelled data, which is very very expensive to collect.
Like, take a product manager, where can you get labelled data enough to train a model to talk to customers, get some insights, and figure out whether or not this makes sense and then determine how to build & market it.
None of this data exists basically anywhere, and it seems really hard to collect.
But even if you do, how does one train a model in an RL environment to talk to customers. Presumably they talk to agents, right? Does this generalise to actual people? (Maybe, maybe not).
And then consider the whole figure out what to do from the customer meetings, this is basically meta-cognition combined with awareness of human body language and emotions. How do we train for that? I'm not even convinced this is possible (but I've been wrong before).
I think the weakest form of my argument is that current LLM/RL pipelines make this borderline impossible to do. It's possible that new methods work incredibly well, but you'll still be blocked on the data (this is why I'm extremely sceptical of white collar work getting automated).
If you see a way around this, please help me understand what I'm missing.
Google informed me ages ago that if i don't disable gemini activity history, that humans will be reviewing my chats.
Meta and co are using people in the maledives (there were reports about this last year and earlier too).
Google also has partnerships with the biggest hospital provider in the US America market with a market share above 50%. There they get medical data and co.
The RL is a mix of real data and enhancing the data with big models.
You start with supervised finetuning to have the base for RL.
We already have some companies were AI is taking your call though, if the call is successful and no indication that the human was unhappy (basic sentence analysis) you can take it as good data or ignore it and you use some signal to filter out the bad ones and give them to humans which create data for RL and the RL makes sure the model adjusts.
And the other big elephant in the room: what/when is LLM generic enough from a ksillset point of view and knwoldegeable enought to learn adhoc through context only and figuring things out themselve. A new tool? It might run a sandbox and distill knowledge into context.