Hacker News new | past | comments | ask | show | jobs | submit
aurmc | on: Apple has new Mac app coming to make running local AI models easy
Many users would prefer to use a Sonnet-level (or better) LLM without sharing their data with others.

And with a device like this, you pay one single time to buy the product and then never have to think about paying for tokens ever again. For ordinary users that's silly, but for users or businesses chewing through lots of tokens, this means they can create their own turnkey replacement for "the cloud" where the only cost after purchasing the machine is the electricity to run it (which with these machines is very little). Mix that with the privacy benefits and suddenly it makes a lot of sense.

asdff | parent
Seems everyone in this thread is assuming I am saying go to private third party compute. No. Build your own cluster on metal you own on property you own, and have your team use that and just some thin clients to write with. Your compute goes way further. Your cluster can be going 100% of the time with jobs queued up by your team. All that memory you buy for your team in macbooks is idle and unavailable when your employee closes the lid and tosses it into the bag into the car on the way home. It could be used 24/7. On hardware you own. For cheaper to build I'm sure than apple hardware tax. On easier OS to manage and no doubt deal with anything involving any sort of heavy compute work than OS X. Are we really going to waste resources with docker when we could be using a node completely? Come on.

Like I said, seems HPC is lost art. All four sub replies missed that in my original comment.

eddythompson80 | root | parent | next
Your main argument is about better allocation for that RAM, which only makes sense in the conditions of a shortage. If RAM is plentiful, like it’s bound to be once the datacenters craze is over, that argument becomes “why does every apartment need a washer/dryer when we can have shared washers/dryers in a common area that are being used 100% of the time”. The answer is “because we can and it’s not that huge of a difference” plus you can have both. Like you can have a server cluster for large batch jobs, and personal ones for personal use.

For individuals though, it’s the same reason plenty of people pay for the 2tb or 4tb SSD for their laptops or desktops even though NAS has been a thing for 40 years. Maintaining a “server” of any sort is a completely separate endeavor for most. Just like a NAS is a no-brainer for many and non-starter for many more.

flaunf221 | root | parent
RAM was dirt cheap before and yet until M3 Apple made laptops starting with 8 GB of RAM. Why do you think Apple will suddenly become generous later?

The problem with LLMs outside of server is that people typically have many devices. So you can run LLM on your laptop. You want to make a query from your phone. Start your laptop? You want to do something from your smart home? Start your laptop? What about your partner? Will they need to buy another top of the line laptop for themselves? Your kids?

Alternative to server is of course that everything will have over 100 GB of memory, eat battery like crazy, be dumber and slower than single server in your household.

4 TB ssd was like $300 until recently. Even if RAM prices were to go to previous values, I don't think AI capability will be any comparable. Besides you also need proper compute, unless you want your prefill to be snail slow.

loading story #50451336
throwaway277432 | root | parent | next
It's investors making these decisions. They would rather have predictable cloud spend than trying any experiments with high cap-ex. Has been this way since AWS is a thing.

Not saying they're right.

monegator | root | parent
yes. People do not read.

I agree with you, and i would like to set up a server at work for local inference but we simply do not have the knowledge, it's much easier to get a couple of AI Max machines that are good enough for me and the boss which i convinced on the good points on running clankers locally and be done with it (though my plan is to use the AI MAX machines to figure out what the infrastructure and workflow will look like, and at some point ask for the investment)

rick1290 | root | parent | next
this is neat. how has it been going with the ai max machines? also looking to do the same as id like to get ahead.
loading story #50451090
throwuxiytayq | root | parent
> but we simply do not have the knowledge

How is this literally even possible in 2026? In a thread about LLMs, no less?

loading story #50451073
loading story #50452178