...but is specifically reinforced into generating text that satisfies the humans training it, and therefore is innately predisposed towards approval-seeking generations.