I still think this question is a future minefield, as well as the question of what happens if an LLM outputs a patented algorithm
This is not really a question. Accidental patent infringement is still patent infringement. Just because you used new trendy technology to do it does not mean that it isn't infringement.