One practical-ish thing you could reasonably do with this (beyond just better understanding how models work so you can make better choices/tradeoffs), is post-train some of the small-to-medium models, fine-tune them on your own data or priorities.

You don't need to change many weights, relatively, to add capabilities or modify behavior. Look up LoRas for instance.

I'm of the opinion that post-training existing models is an area with a lot of benefit and fun in the near future (it's already being done here and there of course).

I doubt you _really_ have to know everything in this course to know how to do that, but I'd want this as background personally for it.

You included machine learning in with LLMs in your root post. Try training a supervised model for some task, like an image classifier or a detector. LLMs have not replaced convolutional models, and even transformers can be used for non-NLP tasks. The idea this is somehow not professional is not true. Despite what people would have you believe, ResNet50 fine-tuned on a good dataset is still competent. In many environments there isn’t enough compute for bigger models, and large language models aren’t always the best choice.

You also don’t have to go straight into LLMs. You can learn most of the training mechanics via classic deep learning. It isn’t much different with LLMs except the training data and losses are specific and the models are much larger.

The comment below/above is right. Pre-training isn't the path (but you really do need to learn how it's done). Learn how it works, learn the architecture, then take models and do post training. Fine tuning and RL on them. There was a fun/clever/useless project recently posted here about building a terminal emulating model thing, as an example.