Ok, but why did they chose Sonnet in the first place? Couldn't they have chosen Haiku or GPT Luna or maybe even some small cheap Chinese model?
At the time of Sonnet 4.6 it was probably the best choice. Haiku was pretty dumb back then. Luna didn't exist.
Sonnet 4.6 was released 6 months before Luna (and Luna's major price cut came a month or so after that IIRC). So it's likely they started with Sonnet long before Luna was an option. I assume they evaluated Sonnet 4.6 vs Haiku at that time and found Haiku insufficient.
After that, I'm guessing it was just fairly low priority to update a working system. Amazon's net income last quarter was >$20B. Saving millions is huge on the scale of one or two engineers spending a week tackling it, but it's too small for the organization to make it a big initiative. By the time engineers decided to update it, decision models like Jev were popular and are naturally a better fit for the task than an LLM.
I’m guessing accuracy. Smaller models are just THAT much more inaccurate to not make it viable.
loading story #50438331
Strands decider actually uses a small cheap/free Chinese model for its base architecture: Qwen
loading story #50440571
Not sure if it would even need to be a small Chinese model. Amazon would surely have the necessary hardware and infrastructure to run a bigger Chinese model with a low per-token cost too. They probably get their Sonnet at a discounted rate too since they do the inference "locally", but I imagine the savings for not having to pay Anthropic their share would still add up.