Hacker News new | past | comments | ask | show | jobs | submit
> So they admit that they could do more, but they won't?

Yeah, there's another paragraph:

> Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.

(emphasis mine)

It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field.

I'm betting this is related to China's Announcement No. 48 last week about AI export controls, since releasing an open-weight model is effectively export: https://www.jdsupra.com/legalnews/china-proposes-new-export-...

> Prohibited: high-performance detection technologies used to counter cross-border data transmission security risks, including deep-synthesis forgery detection (266401J); AI cyber offensive capability technologies, such as malicious code generation and mutation, automated mass generation and distributed large-scale deployment of anti-detection variants (266402J); large models trained using China’s Core Data (266501J); and high-performance AI agents designed for highly sensitive scenarios, such as critical information infrastructure, national defense and core government affairs (266502J).

  > It's clear that cyber is a touchy subject right now.
And that's why we need models that understand "cyber" or application and data security. The offensive models are powerful, and they are in the hands of script kiddies now. The models writing our software and designing our cloud infrastructure had better understand these issues as well, just to avoid coding in vulnerabilities.
They also show scores for a "Kimi model w/o cyber risk-reduction" https://statics.kimi.ai/kimi-blogs/kimi-k3-1/20261023/xdev-b... Presumably State Security will get access to that on request.
I think this is misguided. Considering how much code is AI written these days do we really want LLMs writing insecure code because we lobotomised them?
I think it's political. Certain companies are exploiting as much as possible to push the boogeyman of AI for regulation.
loading story #50214388
{"deleted":true,"id":50214613,"parent":50213912,"time":1792786159,"type":"comment"}
Anthropic: Touch too much security stuff and we ban you, and keep your money.

Kimi: We took out the worst of it so have at it for free.

Everyone else: Back to my ablated model on openrouter for 1/10th the price where I can brute force issues.

loading story #50214354
loading story #50214266
It’s not much of a deterrent to a motivated organization.

They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/

There is no deterrent to a motivated and well resources organization. Post training a model in general, let alone just for offensive cybersecurity, is well within the technical and financial capabilities of many cyber criminals. There's just no way out of that.
Classic “rules for thee and not for me.”

Agreed that this could be reinjected later. Makes me think of neo in the matrix:

“I know zero days.”

“Show me.”

What "rules"? They don't have to post-train the model on anything specific, it's their judgment call.
loading story #50213685
loading story #50213193
Governments also control who owns tanks and missiles.

Do you think that going around making cyber weapons isn't going to get you buried in a shallow grave?

{"deleted":true,"id":50214335,"parent":50212963,"time":1792784580,"type":"comment"}