Hacker News new | past | comments | ask | show | jobs | submit
asim | on: Gemma 4 12B: A unified, encoder-free multimodal model
We are now entering the closed loop game. Google doesn't need anyone else to accelerate their models. This is their bread and butter.

I'm both shocked but also not surprised that they continue to develop such efficiencies. Honestly it's like silicon and CPU architecture advancement. We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x more efficient with time. Maybe there's some point of decay but essentially the next 30 years will be more advanced than the last 30 and were going to be living in some sort of futurist blade runner scenario where gene editing is repairing ageing cells, organs and curing all sorts of cancers that haven't even appeared yet. Beyond our lifetimes people will live to 125 quite steadily and with great mobility and then obviously people will look to how do we get to living 1000 years, which of anyone is religious knows Noah and others lived to that age in a totally different era.

Anyway I'm going off on some tangent but look back 30 years. Now look forward 30 years. It's going to be insane. May God protect us.

loading story #48404661
bityard | parent | next
> We kept shrinking it and shrinking it and it kept getting more and more powerful and here we are with AI and it's only going to be 100x more efficient with time.

It's definitely an exciting time, but in terms of advancements in the state of the art, there is a lot of low-hanging fruit left to pick. There IS a bottom, however, as you can only encode so much "knowledge" in a small number of parameters.

This feels to me a lot like what the early days of what radio or aviation must have been like. Or, heck, microcomputers even.

asim | root | parent
It's definitely a core component of a bigger system. We are effectively trying to recreate intelligence and human life through models and robotics. So the key insights for me, the LLM is the cerebral cortex but we have a lot more to recreate. Once you map in sensory input continuously and give it physical robotics, things start to change. But even before that leaving these things in simulated realities is what will happen, and right now we have things that operate based on our commands, but a complete step function will be the things that act on their own and that will be a very dangerous time but also where we see some very surreal things happening. They might not necessarily be made in the same way either, they might operate on entirely different types of architecture.
0xbadcafebee | parent | next
1996 didn't look that different than today, in the US anyway. Biggest difference, besides the electric cars, is everybody has a phone but nobody uses it to talk to people.
staticman2 | root | parent
I agree the last 30 years in the U.S. hasn't changed all that much due to tech.

It's probably true that phones and social networks have altered the way people think, but not necessarily in a way that's qualitatively different from cable TV changing the way people in the 90s thought compared to people in the 60s...

loading story #48403140
Flere-Imsaho | parent | next
Yes I've taken the "must optimise longevity" route, taking priority over other things such as my career and hobbies. I want to see the future - all this AI stuff fascinates me.
Scrounger | parent | next
> May God protect us.

Today, data systems and algorithms can be deployed at unprecedented scale and speed. Unintended consequences will affect people with that same scale and speed

—Michael Chapman

btbuildem | parent | next
> which of anyone is religious knows Noah and others lived to that age in a totally different era

My favourite conspiracy theory lately is that the above isn't a silly fairy tale, that we actually used to live much much longer -- until the common cold came on the scene, and the sequelae dramatically shortened our lifespans. Today we dismiss it as "just a cold" unbeknownst of what it robbed us from.

ddorian43 | parent | next
> people will live to 125 quite steadily

Only after the current generation(s) of doctor(s) dies. And only if you make this in pill-form. Otherwise people will be people and won't even go to the gym.

It might be also the reverse, they develop a powerful+personalized drug that brings heaven on earth to your neurons (first time heroin experience + sexual gratification + childhood fulfillment + extremely addicting etc etc etc).

-----

Now that I think of it I'm gonna go with the latter.

ActorNightly | parent
Nope, lol.

Large models still are quite far ahead, don't be fooled that even Gemma:31b (which is better than the 12b overall) is anywhere close to big models.

There is definitely room for optimization, but fundamentally, for complex tasks, you need visible small gradients for accuracy that allow the model to be trained on (and consequently be followed during inference). For example, if you specify in instructions not to write code but ask coding question, Gemma will still write code. Whereas Gemini/Claude will pick up on that and follow your instructions better.

mitkebes | root | parent
It doesn't matter if Large models are undeniably better, if a local model is "good enough" to handle the task. With API costs ramping up, I think a lot of companies are going to want to look into what can be run locally instead, possibly only using larger models when the local models fall short.