Without the prompts, which seem like they might be needed to reproduce the results, I am not sure how we can ever know.
And then there is this cryptic bit on the company web page article: "Weeks after one of the agents reached a solution, a physicist, who had independently used Claude to tackle the same problem, shared his result, and a colleague realized that Hongbin’s agent had reached the same answer earlier. The two solutions used different methods, so they were a useful check on each other."
I have questions: Who was the other mystery physicist?
Does that mean there were at least 2 humans involved in the research?
Or were they not involved enough to even understand the abstract, in which case it seems like academic fraud to include any human as an author?