AI is not a calculator, but it'll try to fake it - and above say, three or four digits, there's not enough training data to guarantee accuracy enough of the time. Labs probably train models on massive amounts of arithmetic, but still. It's not the right tool for the job.
Now, ChatGPT won't solve problems with code unless asked, but if you ask, it has its own full computer and can write code for it!
AI is powerful because it can use tools and can build tools for you to use or for it to use.
But it is not a calculator.
It’s both, and a million other things. Of course AI can do complex calculations today. I can’t even remember the last time I saw an AI make a mistake on something like that.
The problem is that people’s mental image of what AI can do always has an inertia of 6-12 months. Like you can still see people talking about how image generation models supposedly can’t draw hands, even though today we have 2K AI-generated video with perfect hands.
Creator did you a favor making their slop so detectable ;)
Quick!
What's the next top-k tokens after each of these?
"999 + 999 = "
"999.0 + 999.0 = "
"999 * 2 = "
"999 * (3-2) = "
Mathematically identical. But as a token sequence, not so much.
Modern models will of course have very similar next-token distributions across all of these. But not identical!
Now add a few digits and you quickly land in territory whose synthetic training data is much sparser.
Code has determinism and runs in milliseconds. It's the better tool for the job.
(BTW the last one was meant to be (3-1).)