Hacker News new | past | comments | ask | show | jobs | submit
UTF-16 in general was a good idea[1] at some point of time but that didn't last long.

[1] or according to some a rotten compromise, between those who though UTF-32 wastes memory and UTF-8 discriminates non-latin languages.

No it wasn't. Using UTF-16 for that reason is all about posturing and not based on reason. Even for mostly CJK documents or ther languages where UTF-8 happens to be longer on average, any ASCII-based markup will quckly turn the tables. And if storage is that much of a concern just compress the damned thing - then the UTF-16 vs. UTF-8 doesn't matter at all for size.

And to be clear, UTF-16 was not chosen anywhere for that reason. It only exists because people thought UCS-2 would be enough for everybody (even though that was always far-fetched and futilely trying to make it reality has permanently damaged Unicode) and UTF-16 was the easies quck fix hack to "fix" UCS-2 based systems. It's yet another short term solution that will keep causing problems forever.