I'm a little confused how this is even feasible to do in general. Most inference is batched and often implemented on heterogeneous hardware. In this environment, it's hard to get bitwise identical results, much less prove that the results came from a certain model. Is this actually practical? In the post near the end it doesn't seem like the authors think they have done anything for this and it seems pretty important for it to be used for any production environment.
This sort of result is about steady progress. It's not practical, yet. This system only allows for 1k context, not even enough for a system prompt. But the authors have contributed several orders of magnitude of speedup on the current state of the art.