This conversation may be about training but the defendant very well may be the end user of the language model versus the other parties that performed the training/inference. It then becomes even more interesting if you consider forum shopping for suits.
I am not a lawyer but do have to think about these problems. To be honest, I don't know where the law will land, but I do know that the tools are incredibly useful. I can't agree with the premise that all output should be open source.