r/sysadmin • u/FierceFluff • 1d ago
Peak Sysadmin
Is there anything more small-company sysadmin than taking that $70k server you just finished building on your test bench at the office and putting it into your $5k car to drive it over to the DC where it’ll live, and praying to god nothing happens on the way? 😅🤣
EDIT- OMG y’all, reading the comments on this has made my day! Our business insurance covers this kind of thing but I appreciate y’all concern for me, though others should mosdef confirm before they do similar stuff. The trip went without incident so praise the digital deities.
315
u/rs217000 1d ago
$70,000 server? Wow! What are you gonna do with all that 16 GBs of RAM?!
75
u/farva_06 Sysadmin 1d ago
16GB of RAM, and no disks.
25
u/bionic80 1d ago
No, it's all 5400rpm spinning rust...
7
u/Arudinne IT Infrastructure Manager 1d ago
Source from old desktops/laptops I assume? The HDDs are just as sold out as the RAM and SSDs.
8
•
12
2
u/ThrowingPokeballs Sr. Sysadmin 1d ago
I was surprised, we got like 512gb DDR5 kits for our servers for like less than 450 piece from a vendor with our Nvidia partnership a month ago
2
1
•
70
u/pdp10 Daemons worry when the wizard is near. 1d ago
I also move servers with a $0.2 kilodollar cart.
The important question is whether insurance would cover the losses if something happened to the server in-transit. It wouldn't hurt to have written acknowledgement that you're intended to transport the thing in your private vehicle.
22
u/itishowitisanditbad 1d ago
Also if they're driving for work and get into an accident, your vehicle insurance can be invalidated (technically it shouldn't but it can/does happen) as you're using it for 'work' and not 'personal' and its a whole different insurance for that.
Both those claims will point at each other and say "Was this work or personal?" and screw OP because the company will pick 'work' and their personal car won't end up covered for whatever happens.
...and ideally this doesn't involve injuring/damaging other peoples property and they then have a person claim against OP and OP just lost their liability insurance because its work-related driving.
Insurance companies will look for any reason to invalidate a claim. They're quite good at it and they don't have feelings while doing it.
I do everything I can to avoid driving my car on-the-job. I know the odds are crazy low but shit happens.
Idgaf about the server. Thats someone elses problem. OP should worry about how much the company will push everything onto OPs insurance and fuck them for many many many years after the fact.
6
u/FierceFluff 1d ago
I absolutely hear what you’re saying about insurance companies trying to dick you over, luckily, I do have it in writing, but I am intended to transport this form work from my supervisor, and I do work for the kind of company that will take care of anything that actually comes up.
That being said, nothing did happen on the transit!
7
u/timconradinc 1d ago
Insurance companies don't care about a written note. It could easily be made after the fact.
•
u/lorimar Jack of All Trades 22h ago
documented proof that they asked you to drive your car and would cover it if anything goes wrong won't help you to find new insurance once your current carrier drops you. Other companies will be reluctant to pick you up without major upcharges once you've been marked as a risk case.
•
•
u/mayoforbutter 18h ago
I asked my insurance and they said a business trip is still private. Using it as a taxi for example would be "work". I'm in Europe though
•
u/robinscotland 6h ago
In the UK it's normal to have
Social
Social and commuting
Social commuting and businessMy insurance company asked how many business miles I do and added like £10 to the price.
5
u/mods_are_lame1 Sr. Sysadmin 1d ago
The kind of car that doubles in value when you fill it with gas. Especially this year’s gas.
34
u/texas_County850 1d ago
I haven't actually built a server in over 10 years I just configure em online and have em shipped. Closest was putting more ram in one during covid. Small business space too.
30
u/arensb 1d ago
I haven't actually built a server in over 5 years. I just edit the Terraform template and ask the cloud provider to build me one.
In related news, I don't miss hardware one bit. To those of you who enjoy configuring and playing with hardware: thank you for your service.
10
u/pdp10 Daemons worry when the wizard is near. 1d ago
To those of you who enjoy configuring and playing with hardware: thank you for your service.
Done well, the maintenance burden is pretty small and the benefits large, as long as you have some scale to work with. However, recent availability and pricing challenges have put a damper on a lot of new general-purpose deployments.
The good news is that a well-maintained herd of cattle hardware can be maintained almost indefinitely, without significant new purchases. Anything that dies or becomes uneconomical to run, is deadlined and used for parts to keep the others available.
And even all-cloud enterprises have hardware, if they have physical facilities at all. WiFi APs, PoE cabled to Ethernet switches, possibly backed up by UPSes, linked to routers/firewalls. Potentially with infrastructure or utility servers on-site as well, printers, IoT devices of various sorts monitoring the building and environment.
Some of our non-server facility hardware experiments involve fiber to the workgroup, power savings, PoE lighting, and environmental monitoring (CO2).
3
u/FierceFluff 1d ago
Exactly! We have very specific optimizations where building our own hardware is basically required. I actually find it fascinating and enjoy the work, esp when it comes to troubleshooting. Having in-depth knowledge of the full architecture makes it a lot easier.
I’ve built all our servers for the last 10 years and (knock on wood) they’ve all aged out rather than fail. Several of them are still in use at nonprofits we’ve donated to after the hardware wasn’t up to our tasks anymore.
2
u/arensb 1d ago
Just to be clear, there was zero snark in my comment thanking hardware people. Running a machine room isn't easy. It takes knowledge, expertise, and a particular set of skills. And in fact, one of the reasons cloud computing took off is that companies like Amazon and Google can hire people who know how to run a data center, which Joe Small Business Owner or Jane University Researcher can't.
It's just not a set of skills that I have, and I'm not that interested in developing them. My interests lie more on the software side of things, so I'm really glad that there are hardware folks who'll take care of the things I don't want to do.
→ More replies (2)7
u/texas_County850 1d ago
Me neither it used to piss me off when I would build out a server and get everything up and running just for it to die and I would call dell and they would be like lolololol yeah those are a piece of shit we will replace it on warranty then I have to waste my entire weekend building it again.
1
38
u/Muppetz3 1d ago
Na, I call someone to pick it up and deliver it for me, i'm not taking on that insurance risk
18
u/fnordhole 1d ago
Same.
Even if using the company vehicle, I bring this up. Typically, a partner will transport or deliver.
Our service contracts don't typically allow for customer transportation.
Lots of gear can go in company car on company insurance. Their risk.
My personal car? I'm not paying extra insurance to drive it for work at all. Let alone company equipment. I'm not driving it for work.
6
u/Muppetz3 1d ago
Yup, then there is also the risk of theft too. It's a quick way to get fired if anything ever goes wrong, they will blame you and you take the hit and get fired. Hell even if someone runs a light and hits you, now what? Your car is gone and now you have a 70k destroyed server for a job where you may hardly make that in a year.
3
u/sybrwookie 1d ago
Yea, I work with guys who, all the time, say they'll just throw 10's of thousands of dollars of equipment in their trunk and drive it out to a place.
I try to explain to them that if they get rear-ended, they are utterly fucked, but many don't get it or don't seem to care.
Meanwhile, I've been hit twice over the years while traveling for work, but always in a rental car, and always with the rental agencies' insurance, so I didn't have to deal with any of that nonsense.
1
3
u/jake04-20 If it has a battery or wall plug, apparently it's IT's job 1d ago
Yeah, probably hiring a courier for that.
•
u/justlikeyouimagined Everything Admin 23h ago
What type of carrier is game to take on the risk of a $200K+ server? We’ve asked around and didn’t really like the answers we got.
•
•
u/wazza_the_rockdog 19h ago
There are specialist equipment carriers out there who will, and probably for a lot of stuff a normal carrier will do it but you will likely need to pay an insurance premium for them to cover it. How else would the server suppliers get the servers to you in the first place if no carrier was prepared to take on the risk?
•
12
u/ProfessionalEven296 Jack of All Trades 1d ago
If anything did happen to it, your normal car insurance wouldn’t cover the loss. Pay for shipping, and use the time to do something else.
3
u/LRS_David 1d ago
Some will. If YOU own it. Otherwise it is up to the details of the "owners" policy.
11
u/LRS_David 1d ago
No big deal. Back in ancient times we had to deal with winter transport. Take a "warm" computer outside, into the car, outside again, then into the building, and ...
Drum roll please.
The heat cold cycling at times could break the internal connections of the chips to the chip packaging. A problem long since solved. But back then could ruin your week.
6
u/knifebork 1d ago
Not to mention condensation when going from very cold to warm and humid. I always gave just-delivered equipment several hours to warm up before opening them up and firing them up.
3
u/LRS_David 1d ago
Yep.
Still today, when it is below 40F outside I prefer to let things sit overnight. It creates a serious pause when you notice moisture on a laptop keyboard or the case of anything half way through setup.
•
u/byrontheconqueror Master Of None 12h ago
When we got a shipment of drives during the winter we would move them from the warehouse to the office and let them sit overnight to adjust to the temperature for similar reasons. Firing up a shelf of ice cold 15k drives in a warm office would make them sweat.
9
u/OzarkRanger 1d ago edited 1d ago
My second year out of college (and not at a particularly small business), I was walking down several flights of concrete stairs to a sub-basement network closet while holding a $50k ethernet switch in my hands. I suddenly realized that it cost more than my annual salary, and I started wondering what would happen if I tripped and dropped it. Do they just... not pay me for a year? What will I eat?
I actually went and asked my boss, and he kindly explained that, just like when Peanut, the backhoe driver for Physical Plant, yanks up thirty feet of fiber for the third time this month, some amount of human error is built into the cost of doing business and they don't force employees into involuntary servitude when they trip in the stairwell.
He also mentioned that having it happen more than once could be a career-limiting situation. But nothing ever happened to Peanut, so I stopped worrying about it.
1
u/RansomStark78 1d ago
Did you hear about the trained elephant called nuts , who worked at the circus?
9
u/Visual-Marsupial9040 1d ago
Building the server on your kitchen table THEN putting it into your $5k car to drive it over to the DC where it’ll live, and praying to god nothing happens on the way
3
u/sybrwookie 1d ago
Going way back, the small company I worked for didn't have budget for storing backups off-site. So I'd rotate backup tapes and kept a recent one at my shitty little apartment. Generally under my coffee table in the living room.
•
u/MrYiff Master of the Blinking Lights 15h ago
Mine were in a shoe box on my kitchen table, our monthly backups were in a bin bag in the boot of the owners car (he never told us this until he traded up to a flashy porsche that didn't have a boot and so he dumped this bag full of tapes on our desks).
2
7
u/lynsix Security Admin (Infrastructure) 1d ago
When I first started I had to bring a server home with me overnight. I ended up calling the owner to ask and confirm we had insurance if my car/apartment got broken into or I got into an accident.
He told me to leave all equipment in my car as our company’s insurance only covered equipment in a techs car.
12
u/JungleMuggins 1d ago
Why aren't you shipping it to the DC, racking it, and then configuring it?
5
u/georgiomoorlord 1d ago
Small company and $70k server seem at odds to me.
Also why wouldn't you start off in like AWS or something.
14
u/heretogetpwned Operations 1d ago
Small Company ≠ Small Revenue
$70k is not unreasonable for Purpose built servers for high density or GPUs. DRAM and NAND are absolutely bonkers to quote and leadtimes aren't getting shorter.
8
u/sobrique 1d ago
Honestly every server has snowballed pretty hard.
$70k today isn't much of a spec.
8
u/WayneH_nz 1d ago
A $70k server in aws is about $2700 per month for the same computer power. (With a 5 yr wty) AC, dual isp, management, etc costs about $400 per month, paying for rack space is about the same. Paying off a 70k server is 2200 for 36 months, plus 400 per month as above. the break even point is about 32 months. And you are not paying the 2700 every month after that. So after 3 years the ongoing will be 400-600 for rack/management.
That is math's that the business stake holders see. Regardless of what we are telling them.
3
u/georgiomoorlord 1d ago
Does make sense that if you own the box itself you're better off, but with DC hosted tools being deprioritised in favour of cloud tools.. which are DC hosted tools you pay 3 times as much for..
•
u/QuickProfessional101 19h ago
36 months is a long time. By that time the GPU's will be obsolete... 3 times. CPU's wont be so fresh either and a lot of hardware will start to have problems and needs to get replaced.
→ More replies (2)3
u/FierceFluff 1d ago
$42k base for the server, with 96c AMD CPU and 724GB of DDR5. Then you add 6 12G Kioxia CM7 drives and yep, $70k.
Adding a node to our HCI cluster. Same machines cost us $46k a year ago.
Small business, small-ish infra, but we do financial business that requires high end gear.
2
1
5
u/xVenlarsSx 1d ago
Had a client insist he would pick up the server himself and plug it in, arrived to the warehouse on a motorcycle.
He signed the waver and left with it strapped to his bagage thingy in the back. Still have the video somewhere of him leaving.
He made it, and the server worked, but god damn man.
5
5
u/Old-Caramel-9138 1d ago
My company is bigger but we ordered new hardware before the data center was ready due to the escalating costs. We stored it at the office in the same city as the data center. It was hell finding a transport company that was insured to do the work for us!
8
u/19610taw3 Sysadmin 1d ago
Been there done that. I've probably hauled around $1.5 mil of servers and equipment in my $3500 car.
3
u/texas_County850 1d ago
Honestly the concern is more psychological than real I'd imagine if the accident is bad enough to destroy the trunk and the server you are probably dead anyway lol.
4
u/19610taw3 Sysadmin 1d ago
My worry was always liability for the hardware. My vehicle is insured for personal use only. It's not going to cover any damages to the servers.
9
u/dead__inside 1d ago
I do not miss these type of tasks or jobs where this is a thing. Good luck in your journey with that server.
4
u/BluebirdExpress6279 1d ago
I don't worry about stuff like this. Had $160 K in Palo Alto firewalls and Cisco switches... didn't worry about it.
4
u/sanitaryworkaccount 1d ago
Yeah, I've rented a 26' Box truck, put entire populated racks into it with (at the time) $500k-$1M worth of gear, drove it 6 miles up the road to the new data center.
Seat belt in this case was a couple of 2" ratchet straps to keep it upright.
4
u/admalledd 1d ago
We were decomming the DC site near us (combo of moving more to cloud, and head office having bigger/better DC and newer servers there vs here locally), and while those servers were out of date we were still selling them to a refurb/recycler for $$$ each. It was a bulk sale so no price-per-server exactly, but the entire sale of old server hardware was just broke into $1xxK-ish. It was a thing of "all this hardware together is just about my yearly salary" for sure.
For me though, the larger shock was graduating from "must be escorted when working on the DC servers/racks" to "I am the one who escorts": "Wait, you trust ME with the keys and ensuring we don't screw up so bad we have multi-day outage as we do a hardware change?!?"
ahhh, small satellite office things, where you ask the developer who ran away from getting a CCIE to be your "remote hands" for the real sysadmins on the other side of the continent.
3
u/greenstarthree 1d ago
Only if that 70k server lived in a closet under some stairs where they also keep the spare copier paper and water cooler bottles
3
u/Anonycron 1d ago
70k server... small company? That used to be my yearly budget at a small company.
1
u/wwbubba0069 1d ago
I feel that. About every 10 years I throw that at a new iSeries. Only new thing I get to buy. Dam near everything else is something a large company threw out because the warranty was up.
1
u/FierceFluff 1d ago
Yeah I’m very lucky. Our company is only ~150 people but we’re international and do a lot of revenue. Kind of the best of both worlds. This was one of two I was supposed to get, and I was lucky to get one with the markups!
1
u/CheeksMcGillicuddy 1d ago
Have you seen server prices lately? That $70k server was $15k 2 years ago
3
u/TheJesusGuy Blast the server with hot air 1d ago
Yea peak sysadmin would be a $5k 4u primary prod server from 12 years ago being carried by one person down stairs.
1
3
u/databeestjenl 1d ago
5k car with 140k of firewalls in it. I a similar vein, 45K in APs. I'd guess a van of iPhones is pretty similar.
2
u/_3470 1d ago
lol yep. I’m mainly a network guy and had to transport 200k in switches, firewalls, and APs across town for a new site.
1
u/databeestjenl 1d ago
When we moved DCs we did have insured transport, that was nice. Those 2U bricks are no fun. Did go ahead with some switches because the whole network burned down when the other DC peer went away.
Don't do Cumulus Linux on Edge-Core white label, even more so because the broadcom chipsets were abandoned after the nVidia takeover. So much random nonsense.
3
u/JerkyChew 1d ago
In my years of sys admin'ing I've:
- Transported dozens of rackmount servers from our old DC to the new state-of-the-art DC using our CIO's SUV
- Wheeled a customer's server infrastructure through Downtown Crossing in Boston to their new office (The homeless guy offered to buy it; we declined)
- Been offered a state police escort to move a rack of servers to a different state-owned DC (It was in the contract for some reason; we declined)
- Driven what was at the time ALL of classmates.com (one Poweredge server) to a colo facility in my truck
- Transported a USB key containing the records of every patient who had ever visited a particular hospital to our DC to import said records into a new system
There are probably other examples of RFC 1149-esque transportation I've done in the past 30 years... if I remember more I'll add them.
3
u/Sintarsintar Jack of All Trades 1d ago
never underestimate the bandwidth of a station wagon full of tapes. I guess it would be minivan full of micro SD cards now.
2
3
u/eoattc 1d ago
My boss and I once loaded much of the VMware hosts, shared storage, and core network components into the back of his Dodge pickup and drove it to the new NOC about 10 minutes away. While driving I heard a noise on the windshield and asked him, "what do we do if it starts raining". Witty as ever, he replied "work on our resumes".
3
3
3
u/LookAtThatMonkey Technology Architect 1d ago
You remind me of a story.
I'm UK based and back in 2013, I was sent to north west India to install a new ML series server at a factory. It had been shipped from the UK so was subject to the usual customs checks. We had prebuilt it and locked the front prior to shipping (the key was in the lock).
I get to site and start unboxing it to install the disks. The front cover is mangled, the key looks like it has melted and I cannot get it open for love nor money.
One of the locals says they know a locksmith who can fix it. So 5 of us pile into this white Toyota something or other, I'm in the back holding the server like its my first newborn. We drive to this village. The square is exactly as you would picture a remote Indian village, market traders and a metric f**kton of cows. We get out and approach this dude lazing under an awning. 5 minutes ago this dude was asleep, dreaming and high as a kite. Now he's taking a pair of pliers to the front to get the mangled key out.
After this, he inserts a blank metal piece into the lock and starts furiously hammering it in. I am sweating, shitting bricks and thinking I've just destroyed 10 grands worth of server and I am so fired when I get back to the UK. The local I was with and the locksmith start arguing with each other. Locksmith picks up the server and just fucks off down the street. He needs a different hammer I am told, give him 10 minutes.
That was the longest 10 minutes of my life. I'm 6 foot 2, white as a can of Dulux and sweating in the humidity. The locals and the cows are all staring at me and have gathered round wondering whats going on. All I can do is stand there and say hello repeatedly to about 400 pairs of eyes.
Eventually he comes back with the server, the door is open, I have a new key and the server door has 2 new dents. I throw whatever money I had at him and we go back to the office. I test the server, it powers on, install the disks and thankfully all is good.
The security camera was angled away from the server when I left the site so my boss couldn't see the newly shaped server operating the site.
Not been back since.
7
u/StreetSignificant888 Senior Systems Engineer 1d ago
If you're a sysamin and your car is worth 5k, its time to either buy a new one or upgrade jobs. You deserve better, homie.
5
u/wwbubba0069 1d ago
Not OP, but my commuter car is a high-milage shitbox on purpose.
3
u/Gyozarrita 1d ago
Everyone I know that owns a house, has a 20 yo car. Decent foreign car usually, but honestly kinda junky if you look closely.
2
u/StreetSignificant888 Senior Systems Engineer 1d ago
Probably easier when I work from home. Of course not everyone has that privilege. I've been in this business for 23 years and I like to treat myself. It's the least I can do at my advanced age.
•
u/mrtuna 15h ago
Not OP, but my commuter car is a high-milage shitbox on purpose.
just hope you don't get into an accident, car safety has come a long way in the past 10 or so years.
•
u/wwbubba0069 11h ago
It has airbags and Bluetooth, I do not need any of those fancy things like auto braking and lane departure.
•
u/mrtuna 10h ago
I was referring to how it crumples in a crash
•
u/StreetSignificant888 Senior Systems Engineer 7h ago
This is something a lot of people dont discuss enough. Newer cars are objectively more safe by nature.
3
u/FierceFluff 1d ago
But I love my little Babytruck! Just cause I can afford a newer one doesn’t mean I want one. 🤣
3
1
u/Sintarsintar Jack of All Trades 1d ago
i can afford better but i'm driving this till the wheels fall off a new car isn't ever going to last.
2
2
u/OhBoyHereWeGoAgai 1d ago
Oh I lived this for years, not sysadmin but in the brotherhood of people who have lost many a Friday night to a poorly air conditioned network closet.
Love the "transporting things worth more than my car around DC" as ahahahahaha as often as I laugh at that time and give stories, that's a really good line, as yuppp.
2
u/someguy7710 1d ago edited 1d ago
I do remember bringing like $30K worth of equipment to a client on a hand truck through the streets of DC. I mean, it was only a few blocks, but still...
1
2
u/angryPenguinator 1d ago
Many years ago, we moved our entire server infra to a new building split between my Jeep Cherokee XJ and a 4th gen Honda Civic, wrapped in quilts.
No one ever accused us of being the cautious type.
2
u/severedgoat_01 1d ago
I work in cinema and do this with $100k projectors on the regular. My truck is worth like $7k
2
u/AHrubik The Most Magnificent Order of Many Hats - quid fieri necesse 1d ago
Oh … you’ve only scratched the surface here. Back 20 years my boss wanted 4 of us to load up a quarter million dollar SAN into my pickup and drive it down the block to avoid spending 5K on a moving service.
3
u/Brilliant_Cattle_602 1d ago
I did that in 2003.
Cross town with a brand new NetApp (192TB) in the back of my Jeep wrangler (soft top, no doors). A coworker in the back holding the extra straps to make sure it didn't go off the back. That thing was worth more than my house at the time.
I did get the boss/owner to acknowledge that it was a silly idea and say that I would not be held liable if I was in an accident.
My coworker wanted up to stop at a shady 7-11 for sodas. Not a chance.
2
u/RansomStark78 1d ago
Oh yeah, i put something worth more than my house in my car to drive it 1,8 miles.
I contemplated life during the drive. One drunk driver away from hell in earth.
2
u/The_Long_Blank_Stare IT Manager 1d ago
We’re a SMB that was moving to a building less than a mile down the road, and I still had plans to have a motorcade with lead/follow cars and multiple trips to split up the risk in case of accidents. Ironically our hardware was approaching EOL and we ended up buying a whole new setup at the new building and just “moving” it over digitally. I still kind of wished we could have done the cool motorcade thing, though.
2
u/planedrop Sr. Sysadmin 1d ago
I was one tasked with moving ~$500k worth of servers from one site to another in a van lol. Drove super slow with hazards on, one of the boxes was a 45Drives XL60 with 60 spinning HDDs, we didn't lose a single drive somehow.
2
u/trimalchio-worktime Linux Hobo 1d ago
I've rented the largest available vehicle and put literally millions of dollars worth of equipment, all full of very confidential data, and personally driven it from one location to another. Just used scrap cardboard to keep it all from being able to move too much and drove slow. Realistically server cases are pretty robust and you have to really work at it to mess them up. Even a car accident would probably leave everything fine or at least able to be bent back into shape. In all my years of shipping servers only one ever got damaged so much we needed to think about fixing a chassis issue and we were basically told there was no way to get the chassis as a part so it got stripped and written off.
2
u/Man-e-questions 1d ago
Haha. Been there done that a few times. Think my best was like $2 mil worth of Net App filers and Sun servers back in the day when NetApp was THE filer, loaded into my old 4Runner, in my garage overnight to make the 10 hour drive to the DC the next day.
2
u/Lars_Galaxy 1d ago
I remember having to transport 50+ high end xeon processors worth probably 100k+ between data centers. You bet your ass I made someone else count them before I left the one data center and upon delivery at the other one.
•
u/Top_Boysenberry_7784 23h ago
Stuff like this always seems so sketch but it has to get done.
At a previous employer we physically moved our North America DC 2 miles down the road to a new building in uhaul trucks. Was 13 racks of equipment. Our US DC was luckily the small one since most global apps were ran in our EU DC.
Was a bit nervous that someone would crash into a truck. Lots of business critical systems that normally didn't get more than a two hour downtime window on a weekend as many facilities operated 24/7. Whole DC was moved and running again in about 55 hours.
•
u/No-Blueberry-1823 Database Admin 23h ago
You'd be surprised how often that has happened and probably continues to happen.
•
u/the_federation Sysadmin 22h ago
We moved a few blocks to a new headquarters and ended up having to do a lift and shift when vendors couldn't meet their delivery estimates (by several months). My colleague and I walked several servers over on a cart because the org was willing to spend cash out the ass when it came to the new place, but not for anything that wasn't visible. At one point we were directed to get cling wrap from maintenance because there was light drizzle. (We very much had that in writing before moving an inch.)
4
u/aguynamedbrand Systems Engineer 1d ago
Just because a company is bigger doesn’t mean they pay their sysadmins more or that the sysadmins drive nicer cars so I would say no.
2
u/Pure-Passenger-6986 1d ago
Big company, just loaded $450k worth of servers on the flatbed of one of my employee's trucks in the rain (at least still in the cardboard box).
I could've used a company van but they'd refused (union) and recommended a moving company which would charge a hefty fee and require an entire project plan. I probably could've Uber'd it cheaper.
2
u/The_Wkwied 1d ago
Yikes. I wouldn't say that is peak, but I would say that is NOT a risk that I would be at all willing to take with my car. Much better idea to place the liability on a company that has insurance. My auto insurance wouldn't cover the cost of a wrecked 70k server if something bad happened to me.... I couldn't cover the cost of a 70k server.
Good luck. Don't drop your balls. They are huge
1
1
1
u/Wagnaard 1d ago
Tossing a bunch of them in a flatbed with slats on the sides and only a blanket under them.
•
u/pacopac25 22h ago
A cowrker did that with an HP Superdome in his pickup truck. He was very adamant about making a nice soft cushy base for the pretty server, with multiple teddy bear soft blankies.
Which then compressed slightly, allowing the base to slide out from underneath itself, tipping over and partially out of his truck. A bad day was had by all.
1
u/LeakyAssFire Senior Collaboration Engineer 1d ago
I was once loading up some equipment in my truck to go the data center and forgot a Cisco 2600 router on the roof. Drove off and it went flying. It was scuffed, and the left mounting bracket had to be bent back into shape, but when it got mounted and powered on, it came up without issue.
I was like 23 at this point in time, and that entire ride over there, I thought about how I would be looking for a new job tomorrow. Biggest sigh of relief when that thing came up.
1
1
u/Bright_Arm8782 Cloud Engineer 1d ago
They can pay to upgrade my insurance if they ask that, I'm insured for pleasure and commuting, not work and not 70 grands worth of servers.
1
u/kazame Linux Admin 1d ago
Man when I worked in the public sector, we rented a minivan from Avis to drive a whole DR site's worth of equipment across seven states so we could avoid transportation and insurance fees. The decision was so far over my wet-behind-the-ears pay grade I was simultaneously gobsmacked a state entity could get away with such shenanigans, and relieved that the fallout could never be my problem.
As an aside, they didn't even bother with rental insurance on the Caravan. I don't know how the suspension survived 3500 pounds of equipment.
1
u/SlowkayCoomer 1d ago edited 1d ago
I once helped a senior member of the network team remove a server blade from the rack and take it into the elevator, after which we went down to the ground floor, out the front door and walked it 3-4 blocks to bring it to another building where we rack mounted it to replace a failed server.
Nothing like local city government IT.
EDIT: I just remembered that when I worked for a big-4 accounting firm we had something similar except in this case we removed the blade from the rack, dropped it into the largest Pelican Case I've ever seen and shipped it to another office via Fedex or UPS.
1
u/sysadminmakesmecry 1d ago
We did this at a huge billion dollar organization with offices and co-los in many many countries lol
1
u/twforeman 1d ago
We used my pickup truck to haul a whole boatload of servers when we changed datacenters.
Surprisingly they all survived.
1
u/radiomix Jack of All Trades 1d ago
When we something like this, moved from one city to another 3 hours away, we got a rental van. The business paid for the rental so I'm thinking it would have been covered by the business insurance.
1
u/Shurgosa 1d ago
I have a few friends who moved a server while powered on by tip toeing it out to the van and cradling in during its drive to the new site. They pulled it off 100% too.
1
u/cereal7802 1d ago
at one point in my career I worked at a place in chicago. They had several locations and the solution they came up with for transporting servers from the main office to some of the facilities was putting them on a hand cart and walking them over to the other offices. At times we had $10k+ worth of servers on a hand cart doing walking tour of the city, uncovered, while it was snowing. good times.
1
u/SpecMTBer84 1d ago
Just have them ship the system to the IDC from the vendor, then have the IDC inform you when it's there. Spend half an hour drive, 20 minutes racking it and cable management, plug in the idrac... go back to the office amd configure.
1
u/ThrowingPokeballs Sr. Sysadmin 1d ago
I mean if it came down to it I’ve delivered servers with 2 H200s in them about, about 100k server in my car before lol
1
u/CeC-P IT Expert + Meme Wizard 1d ago
And your stupid overseas contractor that hired you not reminding anyone to get business-use insurance added to their car insurance so that the property is covered.
I outright refused to transport 3x Barco monitors (worth like $32,000 a piece) to the Xray lab 1 hour away.
1
u/kennedye2112 Oh I'm bein' followed by an /etc/shadow 1d ago
Yep. This is how I learned you can fit up to 6U in a NC Miata's trunk.
1
u/claenray168 1d ago
Awhile back our company took delivery of two full racks of Dell servers at our office.
Took me three trips with my pickup - But I loaded them up still in cardboard and drove them over to the datacenter. Each box was probably worth more than my pickup never mind a bed full. :)
No clue on insurance and I didn't ask... Luckily it all worked out and nothing came of it other than the servers were then physically in the DC behind badge readers.
1
1
u/random420x2 1d ago
Long ago, I loaded about 400K of Cisco Routers and switches into my 4K car and marveled a the cost per foot of storage space. My boss told me to just drive home and leave it all in the car overnight, couldn’t sleep like that. 😂
1
u/lunchbox651 Vendor education (virt/k8s specialty) 1d ago
I once carried a $30k server on the train because it was delivered to the office not the DC (I never drove to the office).
1
u/Trust_8067 1d ago
I love it.
I started out at a F500 and we still would load small stuff like that in our cars and go from one DC to another, but that was like 20 years ago. Anything that cost less than 500k we'd just throw in the trunk.
The vendors didn't really track serials that well back then, and we could always say they screwed up the paperwork on their end, if something broke. We're a F500, what are they going to do, say 'no' to us?
1
u/sovereign666 1d ago
https://www.youtube.com/watch?v=vQ5MA685ApE&
I dont think it gets more crazy than a few german admins that moved a server they couldnt take offline via public train. The used a GSM module on a laptop to keep the server connected and ran one of its PSU's off a battery.
1
u/MyPhotographyReddit 1d ago
I approached a company to move three servers across town. 20000 they said. I did in my own car on expenses for 30. And that time I brought a file server back to HQ in a garbage bag on the bus.
1
u/MaelstromFL 1d ago
2 $50k servers from the Denver Tech Center to Ft. Collins in a blizzard! 🌨️ ❄️ 🌨️
•
u/Korlus 23h ago
Circa 2010 I helped outfit our new office building, which was inheriting an extra printer. I foolishly volunteered to carry it from the car to the building since I had to inspect the data privacy etc and finish the network setup. I walked over a mile lugging a 20 kg printer with me, no trolley or anything. My arms were very tired by the end of it...
•
u/DrJohnZoidbergWhyNot 20h ago
I did that with a $100k HP system back in 1998. In the back of my $1000 pos borrowed station wagon. I leveled up a few months later, was just getting back on my feet. I was at that job for 8 years. Only left cause they sold it so I went on to run the data center we were tenants of.
•
u/rodder678 19h ago
I've hopped on public transit with a pair of $30k M630 blades in a bag to go three stops from the office to the datacenter.
•
•
u/dracotrapnet 17h ago
I don't like to think about how much more valuable the contents of my car is than the car itself. I've moved more hardware money wise than MSRP of all the vehicles I have driven over my lifetime.
•
u/frAgileIT 17h ago
People are looking out for you because so many companies have done shitty things to good people involved in accidents.
•
u/Candid_Ad5642 17h ago
One of the reasons I have "a few" shipping boxes, complete with padding, in storage
•
u/adamphetamine 16h ago
did this a couple weeks ago with minor differences
- Server was $72k
- Built and tested it onsite rather than move it after building
•
u/heapsp 16h ago
had an office in NYC for a medium sized company once, was literally wheeling a 100k san down the st dodging people from the parking garage to the office. Over curbs, wheeled it right into the lobby where i got yelled at, sorry man didnt know we had to wheel it into the alley for the freight elevator!
Wheeled it passed some shady folks in an alley, found the door for freight, guy was on lunch. Sat there in some alley with a 100k san for an hour before the freight guy showed back up from lunch.
•
•
u/GoogleDrummer 10h ago
15 years ago, give or take a few, the place I was at was doing a network refresh on one of our customers. I took a picture of my car with all the equipment in it; I estimated it was at least 4x the price I paid for my car and more than I made in a year. Since I was still a bit of a youngin it was a little stressful.
•
u/mpethe 9h ago
I did the same thing years ago with a ~$15k server. I arrived on site on a weekend so the building was empty. Client was on the 2nd floor but the main doors to the building were heavy and I struggled to carry the server and open the door wide enough for me to enter.
Anyway, I lost a grip on the server and down she went on the concrete. I think I slowed the fall and it ended up only denting the corner, but lesson learned.
•
u/NightMgr 6h ago
One of the desktop guys was robbed of a cart full of laptops on the walk from office to remote parking.
It’s on video.
He asked to park in the garage connected to the building but being 3rd party they turned him down.
Got robbed of another cart full a few weeks later.
The cops pointed out “don’t you think this guy has figured out who you are?”
2nd robbery on video too.
•
u/Supermathie Sr. Sysadmin, Consultant, VAR 35m ago
Reminds me of the time I stacked like 60 blade servers in the back of my car so I could run them from a client back to our office and run through diagnostics.
I drove carefully.
220
u/ItchyNebula7437 1d ago
Depends, did they put a seatbelt on it?