Flide is extremely fast when one input carries many questions, as the shared input is computed only once. Even with a single question, it can still be up to 3x faster, depending on the model size and input length.
Unfortunately, the reverse situation, where the same question is asked about many inputs, is much more useful in practice.
Good point. In that case, the current setting may not help much. But I think we can extend the framework to put the question first, so all the states share one question. We'd decide dynamically whether to place state or question first.
loading story #50424563