Next sentence: "We expected the engine to return an error for any prompt it could not process, and our long-context tests returned successful responses, so we did not believe this to be an issue."
Which is pointing the finger at Sglang, who also weren't responsible for this.
At a minimum someone should have realized during testing that their utilization figures are off by 40%.
A much better title would be "Cloudflare configuration error leaked data between user sessions"
loading story #50255010
{"deleted":true,"id":50254339,"parent":50253459,"time":1793070831,"type":"comment"}
Doesn't matter, they could have marketed it as 1 million token window, you would then infer, get to 600k and get an error. You won't be able to finish your work, and you won't get credit for 600k tokens.
loading story #50254866
Bro, I had to read this sentence in the context of the previous one, 3 fcking times.. and it sill does not make any sense to me.
loading story #50253902