Hacker News new | past | comments | ask | show | jobs | submit
Amazing progress with this budget.

My only concern is that on openrouter.ai it says:

"To our knowledge, this provider may use your prompts and completions to train new models."

https://openrouter.ai/deepseek/deepseek-chat

This is a dealbreaker for me to use it at the moment.

Fireworks, Together, and Hyperbolic all offer DeepSeek V3 API access at reasonable prices (and full 128K output) and none of them will retain/train on user submitted data. Hyperbolic's pricing is $0.25/M tokens, which is actually pretty competitive to even DeepSeek's "discount" API pricing.

I've done some testing and if you're inferencing on your own system (2xH100 node, 1xH200 node, or 1xMI300X node) sglang performs significantly better than vLLM on deepseek-v3 (also vLLM had an stop token issue for me, not sure if that's been fixed, sglang did not have output oddities).

loading story #42769105
Fair compromise for running it for free. The model is open, so you can be 100% certain it's not pinging back if you don't want it to.
loading story #42769379
loading story #42768744
You don't need to worry about that if you are using the open weights models they just released on your own hardware. You can watch network traffic to confirm nothing is being transferred.
With distilled models being released, it's very likely they'd be soon served by other providers at a good price and perf, unlike the full R1 which is very big and much harder to serve efficiently.
Works great for us as most of our code is public and we can only benefit from more our code of our product or using it being available.

Also happy for any of our code expands their training set and improves their models even further given they're one of the few companies creating and releasing OSS SOTA models, which in addition to being able to run it locally ourselves should we ever need to, it allows price competition bringing down the price of a premier model whilst keeping the other proprietary companies price gouging in check.