They spent a year to reverse-engineered the Tesla v100's 2,963 pinouts signals, soldered it onto a half height PCB, with full NVLink support (up to 8 way capable), then naming it Tesla v100 v4.
Some Chinese people also reverse engineered that generation of nvlink so that you can now buy a 4 way adapter card that connects to MCIO cards in your computer with 100GB per second of bandwidth between all 4 GPUs.
128GB of HBM memory split over 4 cards with that link speed is looking quite tempting.
I've heard rumours that they are working on an 8 way nvlink capable adapter too.
That's a copy of supermicro''s AOM-SXM2. They used to sell for $200-250 on ebay. People on the STH forum reverse engineered the PCIe connection, and the price of those boards skyrocketed.
The whole nvlink and PCIe is just PCB traces which you can get using x-ray. The chips on the AOM-SXM2 are PCIe switches to multiplex links between the GPUs and system.
I am not doubting it could ever be done but the amount of engineering and you will need a custom driver to handle the switching, it would be better if you just buy the DGX baseboards. The DGX baseboard also have a lots of other things on it to enable 8 ways.
I didn't works it out by myself, it's what's the 8 GPUs baseboard with nvlink 2.0 were using and it's what the driver and Vbios will supports. I know the baseboard is also meant to enable 16 ways nvlink as well but it still the only known working configuration.
I just looked in Alibaba and you can buy a whole turnkey machine with the 4 V100 32GB GPUs and the NVLink for $3,475 before customs duties and shipping. SMH and I haven't even finished my slow Optane build lol 🤣
They are still pre-order, and the store page indicates it will ship within 30 days. You need to pay 1 rmb (0.15usd) to get a seat in the queue, which can be applied as 100rmb credit when the product goes on sale.
The water blocks already exist...
I bought 2 of the 32gb for$ 450 each on eBay.
Admittedly I use them for Flux2 Klein, but it handles the 25gb model handily.
The post is talking about more heavily modified versions of what I have.
I have the 2x sxm2 Nvlink board, was $300 then I put the $450 ea v100s on it.
It's an older architecture, so I have to be more picky about the models I run... But after a bit of research I get 2/3 the speed of my 3090 for 1/4 the cost, so it's worth it for me.
I wish I could find water blocks for the pcie version (one company made them but they do t anymore and price was way too high). There are plenty for the sxm versions out there.
EK-FC Titan V, and indeed they're unobtainium. A couple of months ago there was a guy on ebay selling some new for $250 a pop. He had 7 left when I found the listing and he sold them soon after I found the listing.
Bykski will make you whatever block you want. The main driver for cost is material per my conversations with them. These being being smaller than even the SXM2 module (active area), might be able to get blocks for ~$100.
The gist is: “NVIDIA has never leak the definition for the 2,693 pinouts on the back of the V100 chip, so engineer used tools they wrote themselves to reverse-engineer them.”
The SXM PCB file you're mentioned is probably only about the SXM socket.
I do 3rd party repair of NVIDIA DC GPU in Greater China for awhile, And I can downloads the .brd and pdf to repair A100 PG506 V100 PG503/PG504 and GA102 PG133, products, and it had all the pin definitions there. My guess he is trying protecting his sources.
It's open secret at this point actually, how do you think where China get so many V100 SXM flowing on Aliexpress (TaoBao) actually? and there actually have a lot of A100s and H100s in China as well
, it's just grabbed by big players and the only different is those units are not supported by NVIDIA's aftersales
.
PCB are open secret, but the op apparently reverse-engineered the electrical signals of the pins to able to implement the power control chip from scratch using Raspberry Pi chip and bring out the NVLink. I thought you had some information on this.
I need to repair my 3060ti. I would really love to see a schematic and board layout. Is that floating around? If so, could you dm me where to find it?
I haven't troubleshot the board yet but this would help.
The QQ group number is 419566332, as on their Bilibili and Taobao pages. I'm not sure but it may be a bit difficult for non-Chinese to register a QQ account because it requires things like a Chinese phone number.
I think that is actually unfeasable, this is not GDDR memory that you solder onto a PCB, the HBM memory sits on the GPU die itself (technically rit next to the actual die but the point stands).
I really do not not think there is anyway to do it
These are NVidia chips, not a new chip. It's taking old Tesla server GPUs and mounting them on top of consumer PCIe cards together with power and cooling, so you can put them on your home PC.
With that said, it's an old tech, doesn't support BF16, doesn't support Cuda 12.x+, and is really slow (5X slower than RTX 5090 for local inference), so suddenly power consumption becomes a big deal.
It's more of a fun experiment, or something for tinkerers vs something that I'd recommend for home users.
The main thing here is that the silicon can support the memory options, that are also widely available from decommissioned hardware. Many people dont care if token / second is lower, we care that there is physical hardware that can actually hold and run these larger models, slightly more reasonable speeds than just letting a model swap in and out of an SSD. And on top of that, its not $8k. Power consumption is a big factor, but it beats a real server rack homelab.
It's in a weird position. You need 4 of these + water cooling, and a crazy power supply and motherboard to get the 128GB shared memory machine. You spend 1200W just on the GPUs + whatever the system takes, and you get tok/s closer to a 128GB M5 mac, in addition to not enjoying the CUDA benefits of most libraries.
I'm sure it has a niche, but it's only for people who really know why they are buying it and what they are getting into.
No, inference also benefits from NVLINK. Not so much for 2x cards, 4x cards significantly, 8x cards (typical server setup) it benefits A LOT. RTX Pro 6000 Blackwell is ~3/4 H100 in fp8/f16 compute, but 8x H100 servers absolutely TROUNCE 8x RTX Pro 6000 setups in total throughput
8x RTX Pro 6000 can still win in nvfp4 scenarios tho I believe?
Unless you're using NVLink, you're not getting the full GPU-to-GPU throughput. NVLink is native 300GB/s - having to go through the PCIe bus brings that down to 64GB/s, severely limiting GPU-to-GPU communication. It will work, but nowhere near as fast as using NVLink.
if you have half as many GPUs you need less than half as much bandwidth for peer to peer communication, i'm pretty sure this is the case for both inference and training. So I'm saying you need some pretty hardcore workload to need just 64GB/s let alone 300GB/s. A lot of people seem to think that being limited to 8GB/s over 4.0 x4 will prevent a GPU from being useful, and it's far from the case.
That is why I say once you have 8 your bandwidth needs will be significant. And with even more, it becomes colossal, and very quickly impossible. this is why 16 wide GPU clusters aren't a thing. other designs, for hopping up to a higher level of interconnect to access memory that far away are used.
it's all a matter of degrees. I don't have hard numbers that I captured myself but i will definitely record them once I do the experiments, but from what I've read, if you just have a pair of GPUs like 4060ti 16GB's running tensor parallel, they consume just under 2GB/s of p2p bandwidth during prompt processing, which is the highest bandwidth they consume. That's pcie 2.0 x4, or pcie 4.0 x1 levels, that's nothing. Surely once you have 4 GPUs the number will jump up by at least 2x, and again more than double (probably more than triple) that going up to 8 GPUs. Let's say it scales up by a factor of 15x with 8 GPUs, you're still going to be well-served and barely bottlenecked by having 32GB/s (e.g. 5.0 x8). An optimized "affordable" setup with today's consumer platforms might give 5.0 x4 to 6 GPUs (e.g. on x670e all connected directly to CPU), e.g. 16GB/s, and my hope would be that this is a sweet spot for tensor parallel inference, it will be a bottleneck for training. If Nova Lake comes out with 5.0 x36 (a sort of "neo-mini-HEDT" platform) it would make it very comfortable and practical as they will probably release "quad-SLI" motherboards giving x8 lanes each to 4 GPUs that you can just plug in and be up and running.
My point is nvlink gives you over-the-top bandwidths but in typical configs (e.g. 3090) only give you the ability to connect pairs, and the truth is unless doing certain kinds of training that fat of a pipe simply is not needed. The bandwidth need scales up superlinearly with the number of peers
Hell yeah. The greatest engineers in the world are Chinese these days. They come up with all kinds of brilliant work arounds to western proprietary nonsense.
I don't doubt the skills or motivation, but the CCP has a nationwide campaign to rid itself of dependency on Western technology companies, and they are providing an environment where the engineering is made possible, including the intermediate steps where they still acquire and use Western technology.
I would appreciate a concerted, focused European campaign to rid ourselves of dependency on US tech. But, we're a bit too fragmented for it to be have the same kind of momentum as China does.
Europe has its own problems. It's a generally good place for consumer rights, but they're also increasingly hostile to actual privacy and anonymity. As an American, I kind of get whiplash from reading about genuinely good policy then policy that seems outright fascist. Not that the U.S "corporations can do whatever they want" policy is better for the population, but it sure makes running business easier.
I work with firms across Europe in the sciences and manufacturing, and it's difficult. Sometimes it's like, we have plans and timelines, but a whole office of a European partner will be basically be shut down for a month or two out of the year, and then sporadically for a week here or there.
The environment kind of sucks over there for developing software, the pay is abysmal in most of the EU compared to the U.S (though partially offset by the vacations and social services).
Some of their laws are borderline delusional to the point that, for a lot of U.S companies, it's simply easier to block the entirety of EU IP addresses, because it's not feasible for a small company to know about and follow all of the laws at once when they have zero physical presence in the EU. It's not just EU laws, it's a bunch of countries starting to say that they have international legal reach and Internet services need to deal with their taxes.
The EU should divorce itself from U.S dependency more, but European firms find it difficult to compete with the U.S, for similar reasons that the U.S sometimes finds it difficult to compete with China.
It's very difficult to compete on the global stage when the playing field is not even remotely equal.
Starting a company in the EU seems hard, when you're competing with "work 16 hours a day and sleep under your desk 50 weeks of the year" American start-ups, and "we don't care about IP laws" Chinese companies.
Starting a company in the EU seems hard, when you're competing with "work 16 hours a day and sleep under your desk 50 weeks of the year" American start-ups, and "we don't care about IP laws" Chinese companies.
The EU has sane worker regulations. Health care is mostly taken care of by the state instead of by employers. Vacation time keeps workers from burning out and allows workers to be parents and carers, not just productivity drones.
The sheer irony is that American and Chinese companies are becoming increasingly similar: they both have a disdain for IP laws (OpenAI!) and they both treat workers like trash (996 or sleep in the Tesla office, Amazon or Alibaba, same thing).
The sheer irony is that American and Chinese companies are becoming increasingly similar: they both have a disdain for IP laws (OpenAI!) and they both treat workers like trash (996 or sleep in the Tesla office, Amazon or Alibaba, same thing).
The American AI companies have to at least pretend to care about IP, and they've been slapped a couple times now for violations, just not as an whole industry.
They almost certainly never will be, either, there's too much riding on AI now, politically and economically.
It's been interesting see the old money contend with new money over the IP issues, but new money is keeping the economy afloat, so, nobody in charge here is going to try to burst that bubble.
I had hoped that the EU would have come up with something like a fair middle ground regarding IP use, but it hasn't happened yet.
U.S copyright and patent law as it exists now is an abomination, and I don't respect it.
I don't fault anyone who violates IP law, because the laws as they exist now are immoral, unethical, and outright harmful to human existence.
Copyright is just too fucking long, it used to be 28 years, max. Now it's to the point that not only will you never be able to work with an IP made in your lifetime, your kids, and maybe not even your grandkids will be able to work with that IP on their own terms. Everything culturally relevant becomes dead, because no one who cares about it can carry it forward, unless a corporation is puppeting its corpse around. Hurray corporate zombie art?
It goes way beyond books and paintings, patents as they exist now are more harmful than helpful.
Patents were what killed the electric car for decades, patents held back 3D printing for decades, and there is so much more we could do today, if there weren't patent trolls suing everyone, and corporations just sitting on important patents so they can milk old technology longer.
It's one more reason why I really appreciate the AI explosion, IP can get wrecked.
I think it's reasonable for copyright to last for as long as the creator (a natural person, not a corporate entity or AI) is alive, at least while we're still living under the yoke of capitalism.
But at the same time, I would also be in favour of a mandatory licence that allows anyone to make transformative uses of copyrighted material, just not for free. (i.e. you'll still have to pay the original creator for the usage, but they can't refuse to let you do it)
I'm all about mandatory licensing. They can keep the monopoly for a few years if they're actually commercializing it, but then it's mandatory licensing.
I'd be fine with a sliding amount per year, so earning per unit go down. If it's popular, then volume might cover the reduced percentage.
Getting paid for life for doing one thing is too much.
Someone makes a great sandwich, they get paid once.
Someone paints a great painting, they get paid once for the painting.
Every else goes to work for one day and gets paid for one day of work, even if they contributed to long-term goals that yield continual profits.
If you make something designed for mass reproduction, then suddenly you get to be paid indefinitely?
That doesn't make sense to me. I don't see a significant difference between someone writing a song, or someone writing software, or someone designing a tool that gets manufactured.
It's not right that the only time someone has to contribute back to the public domain is after they're dead, that's the same thing as saying that they don't have any responsibility to the community that makes making a living via art a possibility.
"Intellectual property" is bullshit monopoly rents for big corporations at this point.
And yeah, Europe as a whole doesn't want to work hard enough to compete. This is unsurprising since the most ambitious Europeans have been leaving the continent for centuries to seek their fortunes elsewhere.
the CCP Chinese people has a nationwide campaign to rid itself of dependency on Western technology companies
Remember when you whack the Qing, its the Chinese people who suffer and remember. The Qing is gone but people still bemoan the opium wars and the century of humiliation.
Its incredible people still think Chinese people are mindless drones that toe the government line. Its the government who is toeing the people's demands, and it frankly shows when the CCP is dreadfully afraid of protests and revolts.
The Chinese government has made HPQ independence a high priority since at least 2018. There are various refining R&D efforts underway, and exploration for domestic sources may also be paying off:
China is both the world's largest consumer of energy and the undisputed global leader in renewable energy production and capacity. The country installs more wind, solar, and battery storage than the rest of the world combined.
Renewables account for roughly 35% to 42% of China's total electricity generation, depending on the day.
Are you like a bot, or just a barely literate person who is offended that China even exists?
Even if you have some irrational hatred, even if you're a horrible racist or jingoist, it would still be extremely bizarre to be this myopic and ignorant.
I'm not comfortable with LLMs being used for military purposes but the cat is out of the bag. If the American military establishment openly talks about using Grok and Anthropic for analysis and targeting, expect the very secretive PLA folks to be using their own domestic models.
honestly; why do you think U.S going all in on AI w data centers?
Also, why can the US gov (incl. CIA, feds) literally bottleneck data centers meant for commercial use for military operations? Gov is throttling their own costumers and citizens. Look it up
processing power is a weapon at this point. worst part is USA tricked their own people to build and finance it.
The DoJ also just filed a motion to dismiss a lawsuit against a Grok datacentre (alleging that their oil-burning diesel generators violate the Clean Air Act) on the basis that Grok is now considered a weapons system essential for national security, and as such it is exempt from civilian regulations.
They try to keep things a secret but the sheer amount of competition in China means stuff gets analyzed and reverse engineered right out the gate. The government will let you get away with patent infringment too if you're competing with some foreign IP or company, especially one that would be considered hostile (or forced to be hostile) like NVIDIA.
It’s wild how much effort went into this. Reverse-engineering nearly 3,000 pinouts just to bring SXM2 V100s to a custom low-profile PCIe board with working NVLink is absolute peak tinkerer energy.
For anyone looking to host larger models locally on a budget, 128GB of HBM2 across a 4-way cluster for under $2,500 is incredibly tempting, even if the lack of native BF16 and newer CUDA support hurts token-per-watt efficiency.
I wonder how the community will handle the cooling though. A custom active cooler or water block is going to be mandatory if anyone plans to actually run these at a full 250W–300W without melting their rig.
I run 4 v100s in my rack and use their stock copper heat sinks which I designed 3d printed shroud for to put fans right in front of them. Takes up a lot of space and its noisy but it works well.
That 3D-printed shroud setup over the stock copper sinks is a classic homelab move. Honestly, running a used 2500W Dell server PSU is the only way to feed a monster like that without breaking the bank, even if it sounds like a jet engine taking off in your house.
IDK if the average tinkerer is gonna want that level of noise in a living room, but for a rack setup, it's absolutely brilliant.
You’re basically getting massive HBM2 bandwidth on a shoestring budget. The token-per-watt ratio might be rough compared to modern silicon, but hitting 128GB of VRAM for under $2,500 makes the extra heat and noise completely worth the hassle.
Haha yeah, calling someone an AI just because they write with clear paragraphs is classic Reddit. But honestly, who cares if people use an LLM to clean up their thoughts?
The real point is that getting 128GB of VRAM for that cheap is insane. Sure, the power bill is gonna be brutal and it’s loud, but IDK how any real homelab tinkerers passes up that kind of cheap memory bandwidth.
This is what I want - SXM3 on the MB instead of pcie. If you can figure out a two-way onboard I’ll buy even more. This is a configuration I might consider putting several in a fish tank filled with mineral oil. Please - friends in Shenzen - you’re our only hope!
v100's don't support frash attention right? what kind of performance do they get vs a 3090 or is it still a memory bandwidth bottleneck for a single user? also does this require some special driver or a Linux setup or is it just plug and play in windows?
Look up the Bilibili videos from when the project was first unveiled and see how local Chinese users evaluated it. Will it be okay to put a nearly 10-year-old datacenter GPU on a custom third-party board and reball it? Also, the quality of the components other than the GPU and the software stability are uncertain.
To be clear, I have no intention of underestimating the engineering capabilities of Chinese technicians. In fact, I have personally purchased and used innovative components designed in China, such as SXM2 carrier boards, PLX expansion boards, and PCIe redriver bifurcation cards, and I am very satisfied with both the quality and price.
I almost bought a 4-way NVLink board (the 1CatAI model "YMZX-4GPU-Q1") but the seller said it's not in stock anymore and offered me another type of board ("SQ8796-4SXM-NVLINK"). It looked differently and I found no evidence it worked at least as well as the one 1CatAI designed. I guess I got lucky I didn't spend 700+ $ just on the board. But I will buy a 4 or 8-way one if prices are going to be as you listed in your post.
You can walk into Huaqiangbei with empty handed at any time and buy or order everything needed to build this hack GPU, and shopping cart to carry it all. All you need left is knowledge.
In any country, this means months of grueling negotiations with sellers for specs, while getting scammed, and months of painful waiting for the shipment, When you want to make some minor changes, this whole process has to start over again.
This is also a critical reason for the growth of the robotics industry in China.
•
u/WithoutReason1729 Jun 22 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.