Hacker News new | past | comments | ask | show | jobs | submit
wcoenen | on: Progress on provable inference for large language models
> The goal of provable inference (PI) is to enable a user with limited computational resources and without any information about the weights to verify that a response came from the requested model

Isn't this already solved by watermarking?

This page said they would provide an API to check whether a given text has the watermark for a Claude model:

https://www.anthropic.com/news/claude-text-watermark

hobofan | parent | next
That's two entirely different types of things that are being verified.

Watermarking checks whether a _specific text_ came from a provider.

PI checks that the text (that you already know came from the provider) was produced with _a specific set of weights_.

PI is mainly useful for AI supply-chain security.

seurimas | parent | next
Not to the degree needed for actually verifying the things this verifies. You could probably quite easily produce a non-Claude model that generates text that conforms to Claude watermarking.
wcoenen | root | parent
In the page I linked above, Anthropic explains that they use the techniques from SynthID-Text, where a secret watermarking key is involved. Only Anthropic knows this key, so only Anthropic can produce and detect the watermark. (But they also promised to provide an API through which other parties can check the watermark.)

SynthID-Text paper: https://www.nature.com/articles/s41586-024-08025-4

saifelse | parent
I think there are two gaps with the watermark:

- it only gives a high probability that the text was outputted from the model, whereas a crytographic proof is computationally infeasible to forge

- it cannot be traced back to the input. You could "wash" the output from one model into another and it would be watermarked by the latter. With the crytographic proof, if you have the input, output, and "commitment", you can verify your exact input was processed by the model (or so the model provider says).