As I understand it, there's post-training step that does exactly that, they get real developers to select good and bad code and help the AI figure that out. AFAIK it's the reason why Claude was able to leap-frog OpenAI in coding last year, they did this refinement step a lot better.

And now they've taught the older models this, they can use existing models to bootstrap the newer models without humans involved, and bring humans in to just assess the trickier stuff.