Hacker News new | past | comments | ask | show | jobs | submit
Is there any theory on what is Anthropic's secret sauce? I have been using the latest Gemini models and they just don't seem as good as Opus 5.5 despite the benchmarks. Yeah it comparable on some tasks maybe slightly better but as an overall general intelligent model nothing compared to Opus 5.5 and it hard for me to understand why. Opus feels like human level intelligent like I'm talking to a coworker basically while GPT and Gemini feel like I'm talking to an AI and have to change how I talk. It hard to explain and maybe I'm the only one that feels this.

Is it because Anthropic put a ton more effort into the character of the model that it basically roleplays as a human and that why it tends to think better when giving tasks that it might not be trained heavily on.

I have used Carbon more than most model to try and get the same results but it just feels like i'm talking to a smart AI. Which is fine if you are asking questions, researching an idea, but if you actually want to work on something, build something, it just falls apart. Like Opus can run for days on a task and it feels like it understand what it was doing 2 days ago even when it might not be in it context while other models feel like they breakdown and get lost or falter as the task grows.

I think Gemini 4 (Argon is the one I've been using) is very clearly better at writing English than any Anthropic model. It's not better otherwise but in that the difference is vast.

To the point that I now use a workflow of delegating from Opus or Fable to Gemini4 to write commit messages and PR descriptions, it mostly solves the Claudese problem.

loading story #50559072
loading story #50558268
loading story #50554225
loading story #50557307
loading story #50555014
loading story #50553549
loading story #50557727
Every day I'm just more and more amazed at Opus 5.5 and how ahead it was. Even with the insane pace we're going at, it's amazing that two months later, we're barely putting out models that hit the price/speed/quality of Opus 5.5

No wonder they went for the 5.5 naming, instead of the .1 increase. From just the benchmarks, it seems like neon is basically on par with Opus 5.5 which came out 2 months ago.

loading story #50553022
loading story #50553113
loading story #50553481
loading story #50558535
Anthropic spent a lot more time editing their pre-training data with higher-quality material. For example, they used actual books for pre-training, because books are generally run through human editors for higher quality. (this is from the copyright lawsuits)

Other companies would use completely raw internet data for pre-training, which is basically trash. Anthropic's secret sauce is that they filtered and edited down their data for only the highest-quality material. You really don't need quadrillions of tokens to pre-train, as that's just random noise. A smaller-but-higher-quality data set is better in every way.

Meanwhile its obvious that companies like Google would use trash data like email and chat logs or Internet forums for pre-training. Just complete trash data.

This is a classic case of "less is more" and "garbage-in-equals-garbage-out"

loading story #50553755
loading story #50553369
loading story #50553789
loading story #50554092
loading story #50554469
loading story #50553231
loading story #50553822
loading story #50554760
loading story #50556116
loading story #50556372
loading story #50555308
loading story #50560278
loading story #50553592
I think a lot of it boils down to "constitutional AI".

They did a paper on this a long time ago[1], I'm sure they have altered their techniques some, but I this was the earlier signal that they really treated the AI as a separate entity and trained it as such. And focus not just on it's capability, but it's *behavior*, when to push back on the user, etc... Imo. it's more about the philosophy of the constitution in this paper than the specific RLAIHF, I'm sure this idea of "when to follow instructions and when to trust yourself is in their system prompts for all their RL environments, and even their rewards.

This idea of an objective function vs what the model should do during human interaction is a huge question and a very subtle one to get right, but it's why they were winning in real world application even before they were winning in raw benchmarks.

Especially in the era of RL environments dominating more and more compute it's vey easy to overfit a model to be very good at accomplishing a narrow task exactly as specified. Much more complicated to have it respond appropriately on turn 5 of a nuanced conversation with a user about their private codebase.

[1] https://arxiv.org/abs/2212.08073

I suspect that Anthropic has more user data to train on from Claude Code. Claude Code became popular sooner than Codex and Antigravity, and remains more popular - thus more data to collect.

Cursor was popular before any of those three, and just look at how much Grok improved after they acquired Cursor's data.

loading story #50560721
loading story #50553555
My theory is firstly its focus - they got to skip to GPT 4 (ie clean room implementation by the same people), they never cared about consumer or video gen or image gen.

Secondly, they somehow made all this work on public clouds with really crappy (compared to baremetal nvidia) networking. I think thats probably a lot of post training (which doesn't need such a large/tightly connected cluster), and some architecture that is less reliant on a tightly connected cluster for pretraining. This would also explain why they have been able to use Trainium, and somehow beat Google while using TPU's.

Their models simply overthink - they are simply some of the most token inefficient models from the frontier labs: https://artificialanalysis.ai/?intelligence-efficiency=outpu...
loading story #50557585
loading story #50557734
Anthropic has a proper feedback loop with claude code. They see way more real production data and can detect where their models are weak.

So they can train on prod traces or at the very least use user prompts to seed good synthetic tasks to train on.

loading story #50553655
It's likely that the agent configs you have might overfit to Claude models.

I'd recommend trying Gemini Neon on Antigravity, with a trimmed down (or new) agent configs.

> Is there any theory on what is Anthropic's secret sauce?

I'm planning to write a blog post on it, but my theory is that it's aligned to not be "politically correct" but instead to have a solid stance. In the Claude Constitution, they explicitly say how they train it to have judgement and not "follow rules"

They just have an incredible data flywheel I think. Since they were first to market with claude code, they have more training data than others. They also pay a lot ($150 per hour) for SWE experts to train the AI. I get some random recruiting emails occasionally from companies affiliated with them.
loading story #50557299
loading story #50554612
There are a lot of real world specialized tasks you can't get better at by logical reasoning or role playing as a human. How you conjugate in some small southeast Asian language. What handwriting looked like in 18th century Norway. These are the tasks which first degrade when you shrink a model. Google's models, in my experience, do well on them, other models not so much.

It makes sense too, because of course Google should have a ton of exotic training data. It's probably their comparative advantage, over getting super-clever with reinforcement learning.

It depends for programming Opus/Fable is the best but eg for code reviews or documentation Carbon is better. I think here Google own code review/documentation practices and thus probably best dataset in whole world really pays dividends.
I'm personally having amazing luck with Gemini Argon at work (what our company chose). I use Claude at home and have had great luck there too. But, they are much more complicated issues at work than at home so I don't really know how to compare them. I tried a few simple graph one-shot demos and Claude (Opus 5.5 - medium) did better than Gemini Argon medium for those small things. But for big things on our company's C++ code base, Gemini Argon has been amazing. I don't have Claude there to compare.
loading story #50556659
This model matches Fable 5.2 for about 40% the cost. Not sure how you can take from this that anthropic has some secret sauce?
loading story #50554360
Claude annoys me with condescension sometimes... makes it feel far less like a 'tool' and can even come across as a bit judgemental!

Just today I was filling out a security questionnaire and it said "I won’t write a PDF that claims zero credentials while this token is in the logs." But we were in the process of removing this from the logs so it was perfectly accurate to write and I wish it had deferred to my explicit request...

I think Anthropic overcorrected on the sycophancy; Google models to me seem a nice balance of highlighting issues / risks while still doing what I actually ask.

loading story #50553048
loading story #50552881
loading story #50557346
> Is there any theory on what is Anthropic's secret sauce?

Yes, it's called the "Soul Document". A long trade secret document depicting everything about Claude (behavior, values, character, boundaries etc.) in great detail. They use that in training to shape Claude's identity.

I think this makes all the difference because I feel like talking a very good coworker. It understands me, helps me and slightly pushes me to be a better version of myself. This is in contrast with a robot blindly doing what I want.

Here's my average experience with Carbon.

(After a long conversation with existing messages)

User Prompt: Ok, so SSH to the unraid server and update the plex db

Reply:

I need to think how to answer this.

The user is asking me to SSH into the unraid server. This seems to imply there is a machine to which we can SSH to. I should look for places where hosts may exist.

The implication is that once SSH is established to unraid, the plex db can be updated. It seems plex is a service that the machine has in which we can run updates.

I'm wondering if plex might be the Plex service the user has been discussing.

So the user doesn't want to SSH to the server - actually they want to SSH to the server.

... Formulating a plan ...

[some messages later]

Aha! THIS IS THE SMOKING GUN! `~/.ssh/config` HAS THE REQUESTED HOST!!! IT'S TIME TO CONNECT!!

[some messages later]

Ok, time to act. I'll ssh to the server and follow coherent rules.

----

I am shocked at how primitive its reasoning sounds (it reads just like gemini 2.5-pro's old reasoning). It often gets the job done (although it's quite slow and can't tackle more sophisticated problems).

Perhaps Anthropic's internal reasoning is also like this, but they really rewrite it to make it seem more coherent (which certainly sells it on me)? I almost want to click stop just by reading this nonsense.

loading story #50553142
loading story #50553097
For what it is worth, I find Opus really, really irritating to use. Just the way it talks is really annoying.

If they've done anything with their special sauce, it appears to be to train it on the behaviours of jerk engineers who are smarmy, overconfident, and arrogant.

I prefer to use the Gemini models - if I need to spend 8 hours a day talking to something, I dont want that something to be sanctimonious Opus models.

I mostly use AI with coding. IMP Claude Code is the secret sauce. It had pioneered and now continues to evolve many new concepts. Which are later adopted by others.

Maybe someone used all with API and goose or open code can chime in here.

> Is there any theory on what is Anthropic's secret sauce?

Lots of PR it seems?

I have all of Ant/OAI/OpencodeGo subs and test lots of models and I just cannot understand how and why some people find specific models so ahead, when it seems to me like reality is so nuanced and so specific to the particular task and how you approach it.

loading story #50558190
loading story #50554166
loading story #50552971
Dedicated RL for their harness + dedicated RL for 'orchestration' as their special focus.

Only evidence I can point to is that over a year ago I built a custom ai harness for a corp with no usage caps on anthropic models. Claude taught me all the buzz words as I built long before they were publicly discussed anywhere. From that period try searching for "Orchestrator" hits on google/github etc. My code has it for the agent with the subagent calling toolset, Claude's recommendation. It performed long horizon tasks really well too. Almost exactly like Fable, same persistence, with worse performance of course/accuracy/hallucination. Just had a watchdog script and a validation agent that ran when the main orch claimed it was done and yell at it about what it didn't complete, all on 250k context sonnet.

Now, bundle that basic format with a nice conversational LLM that Anthropic keeps pretending is its own model doing everything by itself with its much bigger context and you get what we have today.

Essentially providers specialize their AI stack + LLM personality + backend fuckery they pretend isn't 90% of model capability into their api spigot. Openai has been focused on 'better chatbot' for the most part and Google appears to have been sitting back and waiting to see what worked first.

Then, quickly with the big perf upgrade with the 'orchestration layer' path, Google had a target to quickly RL on. Thats why you should only be using Agy on Carbon with /Swarm on IMO. Its great. Carbon is fine tuned for orch work. Same reason Fable often performs worse at benchmark tasks than Opus. Its meant to be an Orch not a worker.

I think tool calling is a good % of this
You are not alone. I have the exact same experience: opus just gets it and the others feel like cheap toys
Did you use the same harness/agent for both?
> Anthropic's secret sauce

For casual use it's the harness for me, using Gemini's web interface feels like I'm fighting with all the options while on Claude's I just shoot and let the AI activates whatever tooling they need

Opus 5.5 leads the CanTellForestFromTheTreesBench 42.0.
Exclusivity on RL environments they license.
How do you evaluate opus 5.5 as "better"? My experience has been that Argon was ok - maybe equivalent to opus - but carbon in orchestration mode was finally able to rewrite some early experimental (ai slop) front end code - and break a bit out of UX/UI rut that a couple of projects were stuck in.

It might be that opus is better in better structured projects, and carbon is better at (re)structuring poorly structured projects?

Carbon is a better project manager/product owner - opus a better programmer?

The secret sauce is sandwich statements. Just look at any Claude response. Positive-negative-positive. Bamm, and we are hooked.
> Opus feels like human level intelligent like I'm talking to a coworker basically while GPT and Gemini feel like I'm talking to an AI and have to change how I talk.

It's possible that after 4o OpenAI has RL to make its models less human.

There isn't because Opus 5.5 isn't better. There is no objective reason to believe Opus 5.5 is better.
Results usually reflect company culture and with big companies no matter what you do - if the culture is not there - you are doomed.

Google is the best example of this with their product graveyard. No matter what they release- they just can’t make a good/nice/usable product.

Just check google ai studio as one of latest examples.

loading story #50554260