Related: https://github.com/haykgrigo3/TimeCapsuleLLM

> A language model trained from scratch exclusively on data from certain places and time periods to reduce modern bias and emulate the voice, vocabulary, and worldview of the era.

Discussed here: https://news.ycombinator.com/item?id=46590280