Hacker News new | past | comments | ask | show | jobs | submit
Topfi | on: Gemini 4 Carbon
> Carbon ranks #1 on the recently updated v5.0 of Artificial Analysis Intelligence Index [0] at 59.

If you are going to hyperlink, maybe make sure that your model is actually on the page.

Also, the screen reader has a bug ([[duration]] minutes instead of the actual value) and you have once again announced a model we cannot use.

Edit: Both have been fixed after posting this. Maybe a Gemini 4 Carbon Agent has read and fixed the screen reader bug xD

I cannot fathom what motivates a lab, whose releases for the last year haven't been competitive, to shoot themselves in the foot like this. All this does is make it easier for competitors to snipe your actual launch, even if there was potential with this model (doubtful given 4 Argon, 3.1 Pro and 3 Pro), now everyone knows what exactly they need to release eval results wise to beat Carbon. There is this old idea of "Your competitor always is the first to have an idea what your next product will do", but in this case, Anthropic and co don't even need to guess, they got specific numbers to target.

Announce your models the moment they are available on API and in the subscription for all paying subscription users (cough Argon cough), stop with the cryptic tweets to drum up hype you know full well you won't live up to and make models that tool call, task adhere, task infer and handle long context tasks well beyond simple back-and-forth, chat style prompting. Reminder, during Ox Alpha, actual Google employees implied that was their model. How is this something anyone would believe is a good idea and how are they still allowed to tweet anything after? Please, stop that, it isn't helpful.

If frontier isn't your thing, stop wasting money on moonshot pre-train runs, fund research and open weight via Gemma and sell your inference hardware to other labs, that's far more lucrative anyway, as seen with Colossus 1.

[0] https://artificialanalysis.ai/evaluations/artificial-analysi...

gengelbro | parent | next
Oh come now, this rolled release is now the standard for frontier models. Also, the fact that AA hasn't updated their page is not a scandal, just a launch day hiccup.
loading story #50277324
Topfi | root | parent
There was none for Fable 5.1, Opus 5.5, Sonnet 5.5, GPT-6.1 Sol or GPT-6.1 Terra and while Astra had a period between announcement and availability, it was a single day, so not sure that applies either.

Gemini 4 Argon (at best Sonnet/Terra level) meanwhile took weeks to “come out” after their public announcement and when it finally did, paying Google AI plan users couldn't access Argon on Gemini or Antigravity even a week after. Three weeks from announcement to accessible for paying customers is not standard outside Google and this has been a problem with every Gemini release for the last year, so even using governmental requirements as an excuse isn’t possible here.

I will say that I dislike how OpenAI releases model only for Work and Codex but not via Chat, Anthropics "everywhere all at once" strategy is I feel the most user friendly.

| parent | next
{"deleted":true,"id":50277507,"parent":50276995,"time":1793208872,"type":"comment"}
Wheen | parent | next
AA has also lost all credibility to me with all of their index changes being unannounced and following an open weight model getting close to the top spot. I gave them the benefit of the doubt for v4.2, but to do this again immediately after Kimi K3.1 got into the top 5 is a little too suspicious for me. For Opus and Astra to stay at about the same score while everything else drops feels like they are adjusting weights specifically in their favor.

Even in the optimistic case that this isn't some conspiracy, it highlights how arbitrary their scores are. I had trusted their previous ratings and their new index shows that even AA doesn't agree with them. What's to say this new index is any better and won't just change drastically again?

causal | parent | next
I imagine there is immense pressure to announce taking the #1 spot while they still have it. It's probably better than releasing a model that hasn't been properly vetted, but I agree it's annoying.
Topfi | root | parent | next
Fair point, though if they are still vetting, that'd likely also entail some post training or at least the addition of a filter, should they find something they do not like. As we've seen with Fable 5 and Opus 4.8 Fallback, that can significantly affect user experience and benchmark scores.

So if they are still vetting and changes can thus happen, that'd mean the score could change for better or worse. Not sure they'd want the embarrassment of a #1 score changing after announcement.

loading story #50278291
odie5533 | parent | next
We're reaching the point where models are only going to hold the number one spot for hours.
Topfi | root | parent
I honestly still need at least a week before I have a decent picture of GPT-6.1 Sol performance. The announcement pace is truly insane.
mattlondon | parent | next
> If you are going to hyperlink, maybe make sure that your model is actually on the page.

It's on my screen. Try asking your agent to refresh the page.

Topfi | root | parent
As written, both issues were fixed ten minutes after I posted the comment.
Imustaskforhelp | parent
gemini 4 carbon has now come on the artificial analysis page. I think that I had checked it 5 minutes ago and it wasn't there.

For historians, The blog post is 31 minutes ago so we can say that it was true for 20-30 minutes.

Slightly embarrasing but it seems to be the case that the model is actually worth what they were saying on AA. It's just that it is not available to people and I think that Anthropic and OAI will release a better model to public before google might.

AA has tons of its flaws too and the recent shift to their new v5 has been nothing but controversial for many.