Javascript is not enabled. This site can still works but it'll be more interactive when javascript is enabled.
loading...
Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
dosinga
49 minutes ago
|
on: Generative AI uses more energy on non-English text
roughly the tokenizer is optimized such that the training data (for the tokenizer) can be represented with the smallest number of tokens. and yes, English tends to be overrepresented in training data.
reply