Hacker News new | past | comments | ask | show | jobs | submit
advael | on: Ripple-65B Preview: U.S. Open-Weight SOTA on Agentic Coding
Category narrowing to this degree really has diluted the term "SOTA" quite a lot

You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!

gpm | parent | next
Playing games like this also makes it very easy to accidentally say things that aren't true, like in the headline of the article...

> Introducing Ripple-65B Preview, a Frontier Open-Weight Model

It's not "frontier", it's not "frontier" even in the category of open-weight models (though usually that phrasing would mean frontier over all, and also open weight)... it's only "frontier" in the small sub-category of US made open-weight models.

GuB-42 | root | parent
It is technically frontier, if by "frontier" we mean the Pareto frontier.

Being at the Pareto frontier means that you can't get something better on one aspect without compromising on some other aspect. The trick is that it is easy to be at the Pareto frontier by having enough dimensions.

Here, we have 4: score, number of parameters, origin (US) and open-weight. You can get a better score or a lower number of parameters, but you have to compromise on at least one of the other aspects. That's a pretty weak frontier if you ask me.

derefr | root | parent
“Origin” isn’t part of the Pareto frontier because it’s not a quantitative dimension.

(You could say that “ability to be trusted by US government/military users” is quantifiable, but I would argue that by that metric, this model falls behind non-US-originated open-weight models that have more eyes on them!)

Routine8502 | root | parent
Discrete dimensions can exist in Pareto considerations. Not that I'm too fond of this game, but given that "Origin" is definitely part of the calculus of model deployment, it makes sense to consider it in a Pareto calculation.
bastawhiz | root | parent | next
It's just a different pareto frontier because you're filtering out items that don't fit your criteria. "Model was made by Anthropic" would be an equivalent dimension. You would never plot this, you're just reducing the subset of models you're comparing.

By that logic, any model is a frontier model. My GPT-2 clone is a frontier model when you compare it against models that were trained on my homelab.

kennywinker | root | parent
Which you're welcome to do and then people are welcome to disagree that is an important frontier to consider.
getcrunk | root | parent
Ahh I run into this issue all the time with ai when applying any sort of dimensional reasoning abstractly it always implicits continuous
quixoticelixer- | parent | next
Fourth most popular comedy folk rock duo
hkt | root | parent
..in New Zealand!
yorwba | parent | next
AFAIK the currently biggest US open-weight model is Inkling, which they include in the comparison, so at least their claim is not limited to a size category. Still they score 0.42 percentage points higher than the second-best model on their chosen benchmark, so it's hardly a meaningful lead.
msy | parent | next
It's like the Baseball World Series of Frontier models. It's SOTA in the same way Tesla are SOTA EVs - the best you're permitted to buy in the USA.
frinxor | parent | next
The US bit got me as well.

I guess it makes sense - there is a segmentation where users may want to run on domestic "frontier LLMs."

twelvechairs | root | parent
Why should US or China matter if its open weights? Its going to matter a whole lot more whether you are running it locally or where your provider is surely.

[Edit i think the preferred combination for those who actually want some data security at the moment and arent at the scale of self hosting llms is chinese open weight models served in germany or France where theres some better data privacy/security]

aftbit | root | parent | next
There are a number of reasons that you might not want to build your inference stack on a Chinese origin model even if you're running it entirely in house.

For one, you might not trust that the weights do not contain ideological biases (e.g. ask DeepSeek about tibet) or that the methods used to impose these biases may not have contaminated the model in some other way.

For another, you might not trust that the code the model generates won't be backdoored or attempt to leak information through your sandboxes.

For a third, you may not want to build your stack on a partner who you feel is a supply chain risk, who might become unaligned with your interests.

By far the biggest though, is that many industries will be forbidden from doing so by the government in an attempt to protect the US model industry (and thus create the demand for some model like this one).

I'm always excited to see new entrants to this open-models race, no matter why they're running.

dragonwriter | root | parent | next
There are zero reasons to have any less concerns about ideological biases, side effects of the mechanisms used to implement ideological biases, or code backdoors in a US model, and a supplier of open (as opposed to hosted) model weights is not an active ongoing partner with whom the idea of supply chain risk from possible future divergence of interests is coherent, so that concern is either invalid or, if the “partner” is the model itself, simply a restatement of the ideological bias in the model weights concerns which, again, is not diminished by the model having a US origin.

And your speculation about future protectionist regulation is just that, not an actual fact.

spockz | root | parent
I think the only point being is that if you see US as homogeneous and you are from the US then it’s okay to use an US model because it’s values will be signed with yours. But that is pretty weak anyway.
walrus01 | root | parent
> For another, you might not trust that the code the model generates won't be backdoored or attempt to leak information through your sandboxes.

I don't disagree with you on the severity of the concerns, but given the vast amounts of code that must have been generated in the past six months by the likes of GLM5.2, GLM5.3, deepseek v4 flash 0731 and newer, qwen 3.5-122, 3.6 and 3.8 series models, I would like to see any concrete examples of it building backdoors/intentional vulnerabilities/phone-home-to-china or similar unprompted. If it's been doing this and anyone with even a modicum of a clue has been watching it, surely someone has caught a model doing this? At least once?

Surely a model doing something like writing intentionally obfuscated code that obscures its actual purpose from a code reviewer, in the service of something malicious/backdoor-like, would have been seen and noticed by now.

mcnewcp | root | parent | next
I wish it didn't matter to me, but it does. I work in government and we self host. No matter how much I explain to them that by hosting weights on our own hardware, it doesn't matter where they came from, they still refuse to let me deploy anything from China.
dragonwriter | root | parent | next
> Why should US or China matter if its open weights?

Weights aren’t source that can be inspected for nefarious tricks that would impact use, so, to the extent that there is a dinension of trust associated with nation of origin, being open weights doesn’t obviate the concern (I would argue, though, that there is very little reason to for nation of origin to be a significant factor in trust alone, though a finer grained consideration of origin might be sensible.)

lynx97 | root | parent
> Why should US or China matter if its open weights?

Well, you might want to know what propaganda/censorship you are subjected to. Case in point, a regional government in Austria just learnt that their AI doesn't know about the Tiananmen Square massacre.

mettamage | parent | next
I’m SOTA on being exactly who I am.

Yep, you’re right. Quite easy to dilute the term.

visarga | parent | next
No, the term is well defined. It means being on the Pareto curve of performance vs size, such as: https://tinyurl.com/26sw35fw
advael | root | parent | next
Irrelevant. A term can be well-defined and used correctly while also being diluted. In this case, I am pointing out that it's diluted by narrowing the category in which the claim is being made. I have no doubt that in this narrowed category, the term is being used correctly. This has no bearing on the fact that by narrowing the category in that way, this claim is made remarkably less impressive or interesting to many people hearing it, including myself
vikramkr | root | parent | next
Terms can have multiple definitions and language is descriptive not prescriptive. If you intend to communicate that you mean "Pareto frontier" in the current context you need to include the word "pareto." If you don't then "frontier" will be understood as referring to mythos/astra/opus 5.5 tier models in intelligence (max capability regardless of tradeoffs). It doesn't matter that the second definition in the dictionary is the one you want to convey (charitably assuming you're not trying to mislead given the value of the hype of being able to call your model a frontier model) if your audience understands what you're saying to mean the first definiton.
XnoiVeX | root | parent | next
Frontier and SOTA usually mean that something is on the edge of what's possible. The bleeding edge.
bastawhiz | root | parent | next
Is it really on the pareto curve if you get to define arbitrary conditions for what other models are plotted on the curve?
O_H_E | root | parent | next
well yes but the y axis heavily depends on the benchmark you choose
epolanski | root | parent
By...whom? An ISO standard? Highly cited scientific literature?

A tinyurl pic linked by randoms over the internet?

swiftcoder | parent | next
For sure, but unfortunately, there's a non-zero chance the state department at some point makes the "US" part of this mandatory for a bunch of companies.
dev_l1x_be | parent | next
State of the AI-Apocalypse.
cyanydeez | parent
It means ot was released recently.

It was already meaningless when US labs used it.

It means new.