Hacker News new | past | comments | ask | show | jobs | submit
joecot | on: Kimi K3.1 Pro model weights are out
The benefit is that you can use these models on a variety of different providers. You're not stuck with a single proprietary provider. Aggregators like OpenCode Zen and OpenCode Go can shop around providers on these. OpenRouter shows you the costs of a bunch of the providers, and you can route to the cheapest ones.

I regularly use together.ai with models like this, where they're fully self hosting them and I can turn off all data retention, at a fraction of the cost of proprietary models. I recently discovered abliterated models (models edited to not refuse to work on security stuff) and hosts for them like refuseless.com, and that's an option for open source models, not for proprietary ones.

penagwin | parent | next
> Aggregators like OpenCode Zen and OpenCode Go can shop around providers on these. OpenRouter shows you the costs of a bunch of the providers, and you can route to the cheapest ones.

Unfortunately due to their licensing requirements for modal-as-a-service providers, the pricing can’t go lower than Moonshot’s from what I can tell.

This is still great if like your example you trust together.ai more, or if different providers have better speeds/cache rates (though usually the original lab has the best cache hit rate).

And obviously if you’re a large enough service to afford to self host or finetune it then this is good.

athrowaway3z | root | parent | next
I didn't dig into Kimi's MaaS legalese, but i dont really understand how that knee-high fence is supposed to work long term.

All the service provider needs to do is provide the option to rent out VM's with access to the inference cards.

Have the customer provide the model.

Maybe provide a bit of a speedup, by letting the customer mount the model by hash.

Sure, that all sounded way too technical 10 years ago, and somebody would have said that's already too much annoyance to deal with for average customers.

Reluctantly i'd agree.

That was 10 years ago. Nowadays, with some proper docs, this entire set up can be up and running in 5 minutes.

knollimar | root | parent | next
Aren’t these only financially viable by sharing weights in some big setup?
joecot | root | parent
Maybe for the API rate, but for OpenCode Go they're regularly making deals to get users a lot more than their money's worth on open source models.
pigeons | root | parent
Sometimes it can be so hard to tell and track because of the cache differences.
nbk_2000 | parent | next
There's no need to use ablated models right now for cyber work. Several open weight models don't refuse cyber requests (DeepSeek, GLM, Kimi K3, etc). And as you said, OpenRouter has many 3rd party ZDR inference providers and several of those serve these models.
zozbot234 | root | parent
Ablated models are not any more inherently knowledgeable about "forbidden" topics, either. Since such topics are not generally trained for, you just get weird creative hallucinations.
stev1983 | parent
The problem with looking at different prices from different model providers is a lot of the cost is generally driven by cache hit rates. That’s where you can accumulate expense and the cache hit rate can very wildly from one provider to the next.