> We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x more efficient with time.
It's definitely an exciting time, but in terms of advancements in the state of the art, there is a lot of low-hanging fruit left to pick. There IS a bottom, however, as you can only encode so much "knowledge" in a small number of parameters.
This feels to me a lot like what the early days of what radio or aviation must have been like. Or, heck, microcomputers even.
It's definitely a core component of a bigger system. We are effectively trying to recreate intelligence and human life through models and robotics. So the key insights for me, the LLM is the cerebral cortex but we have a lot more to recreate. Once you map in sensory input continuously and give it physical robotics, things start to change. But even before that leaving these things in simulated realities is what will happen, and right now we have things that operate based on our commands, but a complete step function will be the things that act on their own and that will be a very dangerous time but also where we see some very surreal things happening. They might not necessarily be made in the same way either, they might operate on entirely different types of architecture.