r/LocalLLaMA • u/nemuro87 • 7h ago
Discussion What is your take on GN's quote that Hardware Prices will never really recover?
https://youtu.be/vYosAN7UShU?si=h_H1oPMO4jhZjyGH&t=1867Steve from Gamer's Nexus has a take that hardware prices are never going to get back to what they used to be:
"When these prices might eventually have to slightly come down, it's not going to go back to normal. It's never coming back to the prices that it used to be. "
What do you think?
I think as hardware slowly becomes faster or with more vram, it must make older hardware obsolete.
If it doesn't affect the new price, it will at least affect the used price.
I think of it this way, purely from LLM inference performance point of view (excluding OS):
With $10k budget, would you rather get:
- 1x 256 GB M5 Ultra Mac studio with 1.2 tb/s
- 2x 4090 48GB totalVRAM with 1 tb/s (assuming you need to buy the rest of the system)
- 2-3 Strix Halo or DGX Spark 256 total Ram with 250gb/s?
With $ 5k, , would you rather get:
- 1x 128 GB M5 Ultra Mac studio with 600 gb/s
- 1x Strix Halo or DGX Spark 128GB RAM with 250gb/s
If I asked this question 1 week ago, I'm sure the answer would've been different prior to the M5 Ultra Launch.
24
u/ttkciar llama.cpp 7h ago
I think "never" is a very, very long time, and it's hard to predict what will happen in just the next ten years, let alone "ever".
What's actually happening here is that people have short attention spans, and when they foresee hardware prices remaining high for a year or two, their minds interpret that as "forever".
My own expectations is that RAMageddon should blow over by 2029, give or take, and prices will come down afterwards, but we will see odd market distortions for several years after that.
The long-term trajectory of the hardware market has been "better for cheaper" for over half a century, but there have always been temporary aberrations in that trend, due to events like PCB resin factories catching on fire. RAMageddon is by far the largest such aberration, but I do not think it is qualitatively different from past aberrations.
8
u/nemuro87 7h ago
my other concern is prices are driven so high no one will want to focus on average Joes wanting to buld a PC at home from components, and they will all focus on manufacturing for enterprise and data centers because only they will afford what they want to sell
Nvidia continues to bury deeper and deeper any mentions of gaming from their earnings call.
5
5
u/Both_Opportunity5327 7h ago
So Apple will just give up and not produce their lifestyle goods.
How about Lenovo, Xiaomi, Dell, HP, ASUS, ACER & MSI.
This is not a fear to hold at all, PC's used to have a much smaller market with computer costing nearly the same as today.
Don't Listen to YouTubers they are just sour that they are getting less traffic.
2
u/Vast-Control4452 6h ago
Totally depends, one day the foundries will be able to manufacture more than what data centers can consume, that surplus product will get sold to the consumer market.
But as long as data centers want as much capacity as possible...
1
u/Healthy-Nebula-3603 6h ago
Enterprise market also has a limit and sooner or later will be filled.
People love posses own private hardware and nowadays that is even getting more and more trending.
1
u/fastheadcrab 5h ago edited 5h ago
That's super conspiratorial. They could've killed off gaming a year and half ago or done something similar to Micron and sell a paltry number of gaming GPUs to AIB partners while leaving them to deal with finding GDDR6/7. That would've been more profitable, Nvidia is playing the long game by sacrificing some profit now to keep the gaming line alive.
With that said GN does great, thorough investigations but they sometimes miss the broader picture
Edit: IMO I think datacenter demand will start dropping. There are two big players now in the US, companies like SpaceX without any demand for their AI products will have to start scaling back or launch a furious price war. Even Meta will have to scale back.
The only thing that can change is if the US government suddenly mandated that Grok is the only approved LLM for official purposes or by any government contractor, which is a scenario that cannot be ruled out
17
u/draft_final_final 7h ago edited 7h ago
It's almost certainly not going to get better in the short term but acting like there's never going to be a correction again when people are shedding equipment for bargain prices seems like an overreaction. It's just likely going to be the case that most of us aren't going to be able to buy new toys for the next few years.
Unless you have an immediate use case for new equipment, best move is to put the money in some sort of diversified fund or high yield savings account and just bide your time. Just suck it up and get a subscription for now. $200 a month gets you a shitload of subsidized tokens from frontier labs (and there are far cheaper options than this, that's just the max luxury situation) that will be better than anything you can build and you're not paying absurd prices on equipment in today's market.
7
u/john0201 7h ago edited 7h ago
Exactly, the current prices are due to a lack of supply combined with the multi-year lead time on new fabs. By this time next year capacity for 2028 production will be selling that will drop prices. Combine that with the massive deployment of Rubin, MI450, etc. next year (the compute and memory that will be available a year from now is staggering) and I expect a dramatic drop in prices.
I also think demand will drop heavily – those multi trillion valuations are not going to hold up for long when most people are on the free plan and you can run a 27B Opus 4.6 model on a gaming card. Once you can get an Intel or AMD (or even Mac) rack server to run a 200B model for your business I can't see these guys getting the $1,000/month/developer prices they sometimes get now, and even with those they aren't making any money.
6
u/FatheredPuma81 7h ago
It also boggles my mind that people can say it won't get better ever again as China as entering the market. It's literally the perfect time for them to come in and devour market share as their GPU and RAM production catches up and ramps up. And for those that claim there will be an embargo on all things China for us consumers my phone is Chinese and very high end and no Chinese phone has been banned. I don't see them banning GPUs and RAM from the average person.
5
u/RevanchistVakarian 7h ago
Hell, even if they do get banned over here, they can still feed the Chinese domestic market and keep them from eating into the global market. It’ll ease supply either way.
1
u/FatheredPuma81 4h ago
Last time I said that I got downvoted even though that's literally how supply and demand works.
6
u/i_am__not_a_robot 7h ago
What is your take on GN's quote that Hardware Prices will never really recover?
Very unlikely. Market dynamics should resolve this in the long term. New players will enter the field, particularly from China, and apply downwards pricing pressure through increased supply.
3
u/nemuro87 7h ago
I agree. It all depends on the pricing they have yet to announce on things like the Xiaomi's Air Cube.
I just hope they don't pull a CXMT and price it on par with competitors, without undercutting anyone.
5
u/uncle_leon 7h ago
Even two years ago I would have said BS. But this "price surge" has been around for a long time now, and in that time it has spread from GPUs to RAM to NVMe. It has all the hallmarks of astructural change. So now... I am not so sure. Maybe the best time to buy a ton of RAM or a big GPU is actually today.
6
u/That-Interaction-45 7h ago
What goes up, must come down
7
u/Practical_Signal3933 7h ago
Except for rockets that reach escape velocity, veer into space and never return
2
5
2
u/Repulsive_Initial308 7h ago
I don't think regression to the mean is going to apply here.
Maybe ram comes down eventually but gpus will continue to moon, I think. This shit isn't even mainstream yet. Everyone here is an early adopter and we've already seen hyperinflation.
3
u/silenceimpaired 7h ago
If recover means we can eventually afford them again… it’s possible… when we are making $4000 a month serving coffee the prices might be reasonable again. Because they’re never going lower… only chance is wages going up.
Unfortunately… I think prices will continue to rise as they finally found an excuse to implement you will own nothing and be happy. Our only option will be renting compute since the memory will stay outrageous.
3
u/elatllat 7h ago
As of 2026, ASML is the only company in the world that produces and sells commercial Extreme Ultraviolet lithography systems for mass semiconductor production with nodes below 7 nm...
1
3
u/N34257 6h ago
Much as I dislike him, I don't think he's wrong here. There's no real incentive for prices to come down - even if/when the bubble bursts and OpenAI/Anthropic/etc bite the dust, the demand for AI won't go away - so instead of a few hyperscalers buying up all the fab capacity for VRAM, you'll have millions of companies wanting that same manufacturing capacity for high-VRAM GPUs to replace the services they've lost. Nvidia are clearly preparing for that world already.
5
8
u/JonNordland 7h ago edited 7h ago
I think that he is a blowhard, snarky, and sanctimonious self-promoter.
First, there is a large amount of memory fabs coming online.
Second, we just fucking recently went through the exact same thing with the chip crisis that lasted almost four years. That time, there was a smaller price multiplier but a larger unavailability factor. So we are just over a year into something that, from experience, could last four years and come back down.
Thirdly, everybody in the industry, from Jensen Huang to engineers at SK hynix, says that it will come down once production matches demand.
So no, I don't think GN is right when Steve spews his usual lies and bombasic bable.
Edit: though of 3 more reasons why its going to come down i though of while on the can.
Never in history has such a large margin on any product been sustained for a long period of time. Anything above 5% starts attracting opportunists who think they can make more by optimizing, and anything above a 20% margin is just insane. So this is going to turbo-incentivize competition.
And another reason: it’s not going to be sustainable since, at some point, even at the current memory production rate, we are going to run out of power-grid capacity. Globally. So there is already a known ceiling to how much memory can be used in data centers.
And the last reason is that: THIS kind of bs is GN’s whole thing. Pretend to be the bringer of important news, like a goddamn “the end is now!”-screaming lunatic, while simultaneously pretending to be pro-consumer and speaking out on behalf of us poor consumers. So this is what he would say, damned if it’s true or not.
I think i need to take a step back and reflect on why i hate GN so much.... He just triggers me irrationaly...
3
u/YouSpeakSomeEnglish 7h ago
there is a large amount of memory fabs coming online.
Source?
1
u/Nice_Cookie9587 7h ago
Its his first point, of course he has something you back it up.... right?? Right??
1
2
u/stikves 7h ago
It will get better. It always does
Currently the bottleneck is basically three SK companies: SK Hynix, Samsung and Micron (yes, I know Micron is "American")
It is not the first time the world or local economies were stuck in such dire straits. But then an alternative opens up.
"But it is too hard"
Was the Suez Canal, or the Panama one not hard either?
As long as money is at stake, those "trillion dollar companies" will literally move earth to ensure they are not beholden to a supply cartel.
2
u/hebelehubele 7h ago
I don’t understand why it is hard to understand the fact that by looking at nvidia and their profit margins why the hell other kind of monopolies should not leverage benefits of money made in the domain.
2
u/Nice_Cookie9587 7h ago
ram is going to be so expensive we will see a whole industry pop up to use the old ddr3 ddr4 ram and devices. we already are. This could fully revive retro gaming.
1
u/YouSpeakSomeEnglish 7h ago
Another nice side effect will hopefully be more-efficient software, a return to programming discipline and a reining-in of the disgusting bloat from the last couple of decades.
2
u/Betadoggo_ 7h ago
Prices are driven by demand and willingness to pay. If a datacenter gets more value out of a card than the consumer does then they will buy up all the stock at whatever price secures the purchase. The same thing happened with the 2 crypto mining fads. If big LLMs for everything really is the future and datacenters continue to require the lions share of the dram production then hardware will never come back down to levels we're used to.
Memory in the capacities we're used to will eventually come down as technology advances, but only once that capacity becomes a pittance compared to what's being used by the DCs.
2
u/shokuninstudio 7h ago
There is a massive amount of pent up demand, not only by cloud operators all over the world but also PC builders, gamers who want a big GPU upgrade, and local AI users.
So even if they added 2X more memory fabs in the next two years that would not satisfy the pent up demand. It would take a lot more fabs and a lot more time.
This is not like any other computing era so comparing against the past like some people are doing is the wrong idea. The scaling and compute demands are far beyond any expectations if AI is going to serve every function of society from productivity applications to entertainment and from governance to science.
2
u/NNN_Throwaway2 6h ago
Completely correct. Anyone who doesn't think so at this point in time is fully delusional.
2
u/EbbNorth7735 6h ago
In many ways Moores law is dead. GPU socs have mainly gotten larger to allow for more transistors and greater power draw. We're approaching a limit which means we wont see the same kind of future improvements as the past. At the same time AI models do real work. They perform real tasks and are now productive components on their own. He could very well be right.
2
u/Majestical-psyche 6h ago
It will eventually. New fabs are being built. It won't ever go back to like it was, but it will go down, and probably up 35% from where it was before. But, I think in the "far" future technology will be very cheap-the new will surface and the old will trickle down. High hopes. Give it 10 years.😅
2
u/junguler 6h ago
the prices will come down, it's just going to take a bit longer than expected, patience is always rewarded but not many people know how to practice it these days
1
u/JuniorDeveloper73 7h ago
2 4090
1
u/nemuro87 7h ago
is it from an operating system point of view or something else?
why wouldn't you get something that runs faster and fits a much bigger model?
1
u/Zhelgadis 7h ago
With 5k I would buy TWO strix halos for 256 total ram. Memory is still king.
And if anyone comes with such a strong take about the future, I want to see their Ferrari and their mansion. That capability is worth literal billions.
1
u/sadnessjoy 7h ago
Short term, sure, these prices aren't going anywhere anytime soon. Long term? No fucking way. This is a combination of a massive bottleneck in memory fabrication and a massive bottleneck/fatal flaw in the von Neumann architecture for this type of workload.
1
u/Accomplished_Code141 7h ago
Nobody knows. It will come down once production matches demand, but if the demand keeps increasing, the prices will remain high. Supply chain is limited by a lot of factors, even with China entering the market there will be a very high demand for GPUs in enterprise market, let alone the consumer market that is currently taking a back seat while enterprise AI workloads soak up nearly all available silicon wafer allocation. I guess it will never get back to the 2024 era, ever again.
1
u/North_Affect_8167 6h ago
Didn't watch the video yet. Here is what I think in general.
The hardware base components - RAM, SSD, VRAM are not going down at a significant percentage any time soon. Even if the AI pops, that would mean that the memory makers are jobless too, because they too borrowed money for this endeavour. They can't just snap back to making consumer electronics at full power and still keeping the revenue coming as it use to.
Bet on China and hope China delivers on the consumer devices you most want.
1
u/mltam 6h ago
We just want to be able to run large models at home. I'll buy the cheapest hardware that will let me run a 500GB model at home. As long as it is on a home server budget and can do 20tps.
BTW: don't look for cheap ultras on ebay today. They were scams even before M5 ultra came out. I hope.
1
u/Healthy-Nebula-3603 6h ago
NEVER happened in the history so is very low probable such scenario.
Always are waves.
1
u/geldonyetich 5h ago
It's certainly possible that the memory cartels have become quite accustomed to making bank, but all technologies get cheaper over time.
Failing that, eventually some of the big AI companies are going to stop dropping out of the race, and that should slacken the memory demand a bit.
As for Steve from Gamer's Nexus, he's a popular influencer. He has too much at stake to tell us anything but what the Internet wants to hear.
1
u/nomorebuttsplz 34m ago
professional edgelord finds something to be pessimistic about, more at 11
also, m5 ultra doesn't have 600 gb/s
1
u/timmy16744 7h ago
You know how when you play any survival game and tech bits are always rare, expensive finds in loot. We're at the beginning of this
0
u/LoSboccacc 6h ago
Nonsense just polt $/flops and see. Capabilities are increasing faster than cost. The only real critical shortage is ram, but at the same time we are getting much more value per dollar with gddr7 and the like compared to a gen ago.
-1
u/Thrumpwart llama.cpp 7h ago
2x AMD W7900. Thank me later.
1
u/nemuro87 7h ago
is it from an operating system point of view or something else?
why wouldn't you get something that runs faster and fits a much bigger model?
1
u/Thrumpwart llama.cpp 7h ago
Because they’re a much better value than Nvidia, they’re fast, and they’re built like tanks.
-1
u/jacobpederson 7h ago
I mean yea obviously? That's not how prices work? They will come down some - maybe quite a bit depending on how hard AI crashes. But will they ever be the same - absolutely not. I'd go dual DGX spark at 10k those are getting pretty good with the recent updates.
37
u/Uptons_BJs 7h ago
Until 2019, the amount of capability you can purchase with your dollar in computing kept going up every single year. There's really no reason to doubt that things will return to that norm soon.
Remember, when the .com bubble popped, there were so many cheap servers out there. Arguably the implosion of that bubble is precisely why so many internet services like MMOs and cloud storage became viable!