Without the prompts, which seem like they might be needed to reproduce the results, I am not sure how we can ever know.
And then there is this cryptic bit on the company web page article: "Weeks after one of the agents reached a solution, a physicist, who had independently used Claude to tackle the same problem, shared his result, and a colleague realized that Hongbin’s agent had reached the same answer earlier. The two solutions used different methods, so they were a useful check on each other."
I have questions: Who was the other mystery physicist?
Does that mean there were at least 2 humans involved in the research?
Or were they not involved enough to even understand the abstract, in which case it seems like academic fraud to include any human as an author?
"Claude did it" doesn't cut it. What's Claude anyway? Some client app talking to Anthropic API's? That's just shuttling prompts and output back & forth.
Researchers x, y or z doesn't cover it either. In this case they'll have relied on Claude for some heavy lifting.
Suggestion: name all the parts involved. "This research was done by Alice, Bob & Carol, using Claude with compute provided by Anthropic" - or something along those lines.
Credits given, clear who did what, no implicit assumptions.
Is that marketing spin, lack of confidence, fear of blowback? IDK, but interesting to think about an academic world where that becomes the case
Even through after all my skepticism about AI in science, Anthropic proved they can search for and find non-trivial discoveries with their tech, they've done very little to dissuade my fear they are actively hurting the scientific community for PR.
We could treat AI as just a helpful fuzzer for open-problems that occasionally opens new interesting paths for the actual scientists, but we know the end-goal of these people is a future where there are no qualified people left to review, let alone make new discoveries independently from the machine.