Hacker News new | past | comments | ask | show | jobs | submit
hobofan | on: Progress on provable inference for large language models
If one were to listen to /r/claude or /r/codex where half the posts are complaining about models being "lobotomized", one could think that this is the most valuable thing there is!

In all seriousness, this is mostly just one of those defense in depth kind of things. Data exfiltration via a compromised provider is also a very different class of vulnerabilities than prompt injection via malicious tool calls.

grey-area | parent
But it still requires trust in whoever is running the server that tags version 'x' as x. All you have proof of is that they return the hash for a given set of weights which is the same as someone else's hash, and those tags/hashes will be changing all the time as the models evolve, so people are not going to be able to keep track of whether Anthropic say is dumbing down their model or not after release, because this is a zero knowledge proof, all it tells is that a certain set of weights was used (you have no idea what weights, how many weights etc etc).

So I don't really see this as useful, except in the very limited use case of using a third party proxy who is trying to cheat you (in which case why are you sending all your data to someone you can't trust?).

hobofan | root | parent
> and those tags/hashes will be changing all the time as the models evolve

I think that's where your assumption differs from stated claims.

According to Anthropics own docs[0], they don't alter the models at all after release. So for each model release you should be able to prove that a response was created with that exact model without dumbing it down.

Zero knowledge proofs also still allow you to have disclosed inputs. You can make one of the inputs be a publicly commited hash of the model weights, and have the ZKP validate that the inference was done with that committed set of model weights, in addition to all the inferences in the batch all coming from the same model.

[0]: https://platform.claude.com/docs/en/about-claude/models/mode...