Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.

There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]

Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.

With how expensive memory is, IDK if they can pull this off but I hope they do

[1] https://www.tomshardware.com/tech-industry/semiconductors/ap...

I don't think so. LLM inference enjoys huge economies of scale. At no point in the foreseeable future will a local model be able to compete with a cloud model in terms of price. Local models will have the privacy angle, but very few people will be willing to pay top dollar for that. There will be some people, for sure, but the vast majority will just use cloud models.

Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break over a decade in the future, for comparable cloud models. Yes, there's privacy, etc, but now you're down into fractions of a percent of users who care about the government reading the memory of their computer.

Apple and Microsoft and other companies will own that front-end experience with the customer, and it’s not going to matter to most customers whose backend gets used for what.

At some point, the real cost of inference/intelligence is going to come due, and the existing open source models will be good enough for 95% of tasks users want (or training their own model(s) will be inexpensive for Apple).

Apple will be there, as usual, to provide secure, on-device, faster inference that the vast majority of their customers will be fine with. Let everyone incur the R&D costs.

It's not comparable to huge models of course, but given the power/size ratio here it shows that serious gains are on the table for models that can run fast on a high-end laptop.

Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.

And Apple really wants to sell people who have no cred in the space their stuff because THEY have no cred in the space.

Its basically a club for stupid f**s