Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.
The Taalas chips are not physically small. And part of their secret (if you look at the design) is just locating a bunch of memory soldered on the edges ( I belive higher amounts of SRAM ? )
loading story #49204555
loading story #49204725
I don't think this works out from a cost/silicon perspective. Small models already run pretty well in software (since the weights fit in cache) and big models require silicon area proportional to the size of weights. On a mobile device putting a chip like this is competing directly in BOM and power against a whole lot more l3 cache, and the l3 cache makes everything faster
What, even if it means you can run models without relying on the currently backlogged DRAM production?
loading story #49205227
My question is what changes about LLM use cases when you’re getting 1000 tok/s? Models in silicon might dramatically change how we think about them.
loading story #49204468
loading story #49204345
loading story #49203980
loading story #49205973
loading story #49203651
Works great from a press release perspective though.
The weights might fit in cache, if you're using a small model. If you wanted to have a 20B+ parameter model, that's just going in RAM. You could put more RAM in the device and pay the perf cost or have a dedicated chip. Most devices already have a dedicated chip, this just changes which silicon you're spending the money on.
loading story #49204361
That's actually a really good point... There's currently zero incentive to buying more hardware, and that's one very good reason do have a new one.
But this is already happening with iPhones. Apple is touting on-device AI and only the latest phones offer the full capabilities. Newer phones will be able to run better models, so the incentive is there as soon as someone makes the killer app that only makes sense when the model is running locally on your phone.
From what I remember, these chips are not mobile size yet
A small model would be. I think that’s more the point. It’s definitely not SOTA but it’s fast and energy efficient and local.
Nope, a small model would be larger than the whole iPhone SoC.
Slightly besides your point, but it's interesting how many here naturally ponder about how the current winner could or "should" keep winning, instead of how another company could become a competitor by doing the more clever thing the incumbent isn't thinking about.
loading story #49204443
Apple is somewhere between fashion company and second rate tech company.
They could have 9 year old AI and still post profits.
Not sure if it's my pixel or android, but I made a randos jaw drop with what the crappy AI on android can do.
When are we getting android OpenClaw?
loading story #49205495