Hacker News new | past | comments | ask | show | jobs | submit
For those skeptical, I've had decent success using formal verification for my Rust rewrite of Postgres. I wrote a compiler that can translate both Rust and C code into Lean. Once those two functions are in Lean, I have an LLM write a Lean proof that they behave identically.

So far, I've proven that about 12% of the code behaves the same when executed single-threaded. That works out to roughly 70k lines of C code. This has found very minor differences in behavior, such as how floating-point operations across platforms can produce different results.

I'm working on figuring out how to scale this up, but I don't see any fundamental limitation that would prevent this from scaling up further.

Every verification needs a spec, and in this case the spec is the Postgres implementation. I think we can and should do a lot better than taking 2.6 million lines of C code as the spec.

In general, the spec must be human readable, and I believe to achieve that we must do away with SQL. Putting aside the thousands of pages of SQL standard, even the core subset has a gnarly semantics due to issues like nulls.

A simpler semantics is one of the main motivations behind Prela [1], and despite its simplicity, we now know it’s powerful enough to express thousands of queries in the wild [2], as well as run doom [3].

[1]: https://prela-lang.org/

[2]: https://github.com/remysucre/prela/tree/sqlstorm

[3]: https://github.com/remysucre/prela/tree/doom/

loading story #50206746
loading story #50207410
loading story #50217325
> In general, the spec must be human readable, and I believe to achieve that we must do away with SQL

Eh. Carthago delenda est? Why?

loading story #50206013
> I wrote a compiler that can translate both Rust and C code into Lean

How do you verify this compiler is correct? To do that, I think you at least need a formal semantics for both C and Rust in Lean, both sound like Herculean efforts

loading story #50204469
loading story #50207782
loading story #50207085
Does your bilingual [Rust|C] -> Lean compiler fork rustc or clang, or other? 12% is promisingly high. I'm still hoping for a world where you can prove critical behavior w static analysis cheaply and practically. The overlap of LLMs and PLT feel under-explored to me

pgrust is incredible btw

loading story #50204511
loading story #50208790
Have you heard of Aeneas? In case you did, what are your opinions on it? Specifically, why you didn't choose it for your project.
loading story #50205450
I'm not skeptical, but I'm wondering how you can scale this beyond just code.

To me, it makes sense that we start being more rigid in how we specify so we don't ever need to look at the code generated by LLMs.

Which models do you see to be productive for the task of writing lean? Big fan of pgrust.
loading story #50203486
So far, I've proven that about 12% of the code behaves the same when executed single-threaded.

Dude, this is a database.

Concurrency is the source of all the hard stuff in databases.

Databases without concurrency are super-trivial.

I don't see any fundamental limitation that would prevent this from scaling up further.

The fact that you've dodged all the hard stuff is a pretty fundamental limitation.

loading story #50205248