Question: could you train a model like this from before a known but important scientific breakthrough happened and see if it is able to work it out.
At least then you know the answer yourself and know it’s something that can be reasonably worked out.
This is an active area of research. Demis Hassabis proposed training a model with a strict knowledge cutoff before 1915, and seeing whether it can independently arrive at general relativity.
This is a really fascinating idea… Just another one for the list of side-projects I’d like to get around to but never will!