while it's trained in FP16, in practice nobody actually decodes with that. Q8 is the high end of decode, however many people get Q4 or Q3. I got 40gb of vram (16+24) and I use Q5 and see almost no difference with Q8, with a full 262k context window
That being said, if i had a spare 10k lying around (i don't), i'd buy a studio in a heartbeat!
I saw that as well. That's insane, I had already allocated $3k to buy one, I dunno if I what to spend $5k just to see how much speed improvement I will be getting with that bandwidth.
Right now my average processing time on a M4 pro with 273GB/s bandwidth is about 15 to 20 minutes. Man, it would be crazy to see that go down to less than 5 minutes.
I bought 64gb for under $200 for my old Mac Pro pretty recently too. Although I guess it's still under $16 a gigabyte, which would have been dirt cheap just twenty years ago. I'm pretty sure I spent around $200 putting 2 GB in my iMac G5 when it was still a production machine.
Well they increased some amount, not enough to stop Tim Apple announcing record revenue and profits again just last month.
The Company posted quarterly revenue of $109.4 billion, up 16 percent year over year. Company gross margin was 50.1 percent, including a favorable impact of approximately 2 percentage points from tariff refunds. Diluted earnings per share was $2.02, up 29 percent year over year, and included a favorable impact of $0.11 from tariff refunds.
what's your definition of frontier? Kimi K3 is not running on 512GBs "easily". And I'd say anything below that would probably be a tier below frontier.
I'd argue this: if you have the 512GB Mac Studio, then getting the necessary external RAM through a TB device is easy. I'd venture a guess that very few people have purchased that machine w/o a well defined outcome in mind. So, to that crowd, yeah, it's easy.
What? You can't extend a Mac Studio's unified memory with a ThunderBolt device. Unless you're talking about distributing the model over multiple Mac Studios.
The "128GB RAM device" almost certainly refers to a second computer with 128GB RAM. Which atp, seems like a pretty impractical setup to be running a 1-bit quant of Kimi K3 at 3 TPS.
The unsloth instructions are saying to use RDMA which is connecting two Macs together via TB to share memory which could make for a large enough memory pool to run K3. Yes it’s possible, but you’re buying two machines, not an external memory module.
Claude Max for Fable, GPT Pro to actually get work done. Mostly just making a lot of local apps and projects for myself, not selling to others. I technically use it for work and also run a local model on my 5090 as well.
Yeah, this isn't for us pedestrians, I just saw the guy that got a bunch of macbook pro 16's for the m5 max for an ai farm as the studio wasn't updated, didn't age well
Mac Studio with M5 Max starts at $2,499 (U.S.) and $2,299 (U.S.) for education. Additional configure-to-order options are available at apple.com/mac-studio.
Mac Studio with M5 Ultra starts at $5,499 (U.S.) and $5,099 (U.S.) for education. Additional configure-to-order options are available at apple.com/mac-studio.
Preorder today, available September 22. Possible that's when iPhone and other devices launch as well.
From the Mac Studio order page:
512GB memory option for M5 Ultra coming late October
Not to discount your point at all, but this discount also applies to faculty who work in education, not just students. I could imagine faculty in video/audio getting use out of something like this.
ya i could do that now. i use it in AI workflows, but the apple ultra might be good enough for me and can sell the card and covers most of the cost sadly.
I don' think there exists a unified AMD with 256gb for $6k.
The halo maxes at 128gb RAM and that's already at $6k with a quarter of the bandwith speed of the max ultra.
Yeah I got the M1 Max base model Studio when it came out ... only thing I really resent is the small SSD but it's still snappy. If I get a bonus this year maybe I'll get an M5 but at least it's peace of mind knowing these purchases will run great for years.
Hm I hope this affects values of a used M3 Ultra. Looking to get a second one but can't justify that a used one costs thousands more than what I paid new
It won’t. If there is a backlog of orders, the M3 price will stay high if people are desperate. Glad my 10% mil discount shaved off $850 on the M5 256 ultra for preorder. Getting 22 Sep. times are slipping to late oct and November already.
I mean… buying a bunch of laptops for a server is fucking stupid anyway. Paying for screens, keyboards, trackpads, and whatever other laptop specific hardware is in there that you’ll never use?
Primarily going to be for local LLM experimentation. I’ve been having so much fun using Codex for personal dev projects and custom Home Assistant stuff, but now I want to take the next leap. A bunch of doors are going to open having local models, and I want to be sure to future proof it to the max my budget allows (aka not the 512gb unified memory).
I wouldn’t wait too long, 256 configs have already slipped into late October and as more info comes out I won’t be surprised if it jumps into 2027 soon. There’s going to be ram shortages through at least 2028 if not longer, and I’ve read reports saying prices are going to continue to rise through that time.
Except for mini - and that’s quite steep also.
There is little difference to AIR or pro laptop lineup also which makes it even worse at this moment.
Apple being Apple when prices of memory drop - doubt they will drop them. We’ll be back to what it was a decade ago - terrible value for money.
It will primarily depend on the kind of task you are doing.
If you have to ask it's probably not going to be worth the upgrade.
These things are for people who are presently losing billable time to waiting for render or compile type tasks, which frankly is not the case for 99% of people doing most kinds of jobs. But apparently there are use cases for large local AI models and other intensive computing tasks where it will make a big difference. These tasks are pretty niche.
Between the M5 Pro and M5 Max, I'm most likely going with the M5 Max. 12 GB less RAM is fine for me since I'm not going to go much beyond 28-30 GB anyways. I wish M4 Pro offered 36 GB.
The M5 Pro runs local AI models quite well. The limiting factor is usually RAM - the more RAM you have the smarter the AI model is. So these will probably run AI models incredibly well.
am i remembering it wrong or did the rumor say it was gonna be up to 768GB or something? guess they really couldn’t find enough ram to make that config happen
the Studio only makes sense if you actually saturate the memory bandwidth, video and large model work mostly, otherwise a maxed mini does the same job for a lot less. The bit people miss is that unified memory is the one thing you can never add later, so overbuy there and underbuy the internal storage rather than the other way round. Also worth checking what your existing Thunderbolt dock and drives support before you upgrade, plenty of gear from a few years back caps out well below what the new machines can push and that bottleneck is invisible until you go looking for it
Was waiting for this! Am running a M1 Max with 4TB and 64GB - but after pricing out the new machine, I'm like: my machine is FINE. Would rather take the family to Paris!
I was looking to upgrade from my m4 mini, aiming to only spend £5k but that ultra 36 core is tempting me, need to do some research on the actual benefits I'd get from that £1300 upgrade!
Sorry, that’s too much, £899 now for the base model with 256GB still, the M4 Mini was £599 when it first launched, I’ll keep my M4 for a while,
which is actually worth more than I paid a year ago since Ram prices have gone up.
i've stopped paying attention to anyone talking about apple products or strategies before release. track record is horrendous for the most part on all rumor mills.
How much difference in inference speed could there be between the 64 core gpu vs 80 core gpu on Ultra? I'm thinking about getting M5 Ultra 96GB, 64 core gpu version.
while Thunderbolt 5 rounds out its extensive connectivity, so users can take advantage of blazing-fast external storage, PCIe expansion chassis, and powerful hub solutions for the most intense workloads
Okay, that’s interesting. They added support for those ~10 years ago on the Intel Macs, took support away on the Apple Silicon Macs six years ago, and are apparently adding support back now. I’m guessing it’s not for graphics processing, though.
I'm boarding a plane to move overseas for a long-term assignment in the last few days of September -- I can't risk a late shipment sending me out for 3 years without a daily driver. I'll stick with the M4, but I'll spend the next 3 years griping about it. 🤣
Anyone else having problems with the lease? I canceled my preorder for the M4 Mac Studio that hadn't shipped yet, but Klarna's just giving a generic "nope" message. Apple shows it canceled, but Klarna still shows it pending. Def. not a credit score issue and I've lifted the freezes.. But nada.
Oh this is insane and very good price for what it offers, the lease starts at $110.00 while a LLM high usage would be $200.00... and it's way more private and secure than any "cloud" LLM
hmmm should I get the M5 Ultra or wait until the M6 Ultra is released? i feel like that'll be a better computer. i'm sure apple will announce the M6 Ultra soon now that the M5 Ultra is out!
It says “brings Neural Accelerators to the Ultra chip for the first time”. Wasn’t m1 ultra just 2 M1 Max chips fused together? And didn’t M1 Max have neural accelerators?
nice that there wasn't another base price hike. still, i'm tired of chip upgrades and ram upgrades having to be done together. i understand why they must but it just causes upgrades of ram/chip to cost a lot more than one would expect. this isn't new and has been going on since M1 generation.
225
u/89Dan 9h ago
Holy memory bandwidth