Why wouldn't capitalization, commas, etc do well?

These are text completion engines.

Punctuation and capitalization is found in polite discussion and textbooks, and so you'd expect those tokens to ever so slightly push the model in that direction.

Lack of capitalization pushes towards text messages and irc perhaps.

We cannot reason about these things in the same way we can reason about using search engines, these things are truly ridiculous black boxes.