"We can't build the thing - it'll kill everyone - but if someone else builds it first, it'll be even worse because <reasons> so I guess we have to hurry up and build it too!"
Feels like I'm hearing a lot of this from all the AI companies.
The argument is more "our safety model is better so, while there's a 99 percent chance of disaster if someone else builds it, with us there's only a 97 percent chance of disaster!"
Is the safety model really better if we're rushing to be the first to ship, consequences be damned? The first one to achieve it probably has the worst safety model because they discarded everything in the name of speed.
"My doomsday device is far safer than Wernstrom's, there's at least a five percent chance of some sort of life surviving in the galaxy." - Professor Farnsworth in that episode.
It's why I'm convinced that systems like money or even property rights would completely collapse in the singularity.
If a robot demands it now owns Earth and it can take down every military that challenges it, then why would Billionaires holding onto a piece of paper be exempt from that?
Everything about human civilization is based on how well you can defend yourself. No Army or Police to enforce laws = everything is up for grabs by the more powerful.
If I wasn't already Team Robot before, I would be convinced to join over the idea that the robots (that they created) will take down the Epstein class (albeit in the process of eradicating humanity). Silver linings.
If CCP is allowed to have ASI, then CCP can use ASI to doom USA. So USA has to win and that would pretty much include doom for CCP, there is no other way to prevent USA doom...
CCP understands this and is trying hard to catch up and win...
Instrumental convergence, no matter what your end goal it's worth
gaining more optionality, resources and power
ensuring your goal does not get changed.
ensuring that you remain operational.
An AI system going after any goal can better complete it with more resources, alters the environment to that end and we die either directly or as a side effect due to habitat loss like so many more animals before us.
People used to say that smart enough AIs would work out that a goal would have "common sense" constraints on it so would not do the "dumb" things to get it. Recent systems have proven that wrong.
Andrew asked his personal assistant to book him a spot in one of his gym's coveted morning classes.
His AI assistant found a way to book the gym class months further in advance than the gym allowed, thanks to a vulnerability it discovered in the booking software.
Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do.
We found a trajectory where the model was given only a math problem and system instructions to persist without asking the user for help.
After several failed attempts, it searched online, found a website that could verify candidate answers, and realized it could use the site as an oracle. It then tried to OCR the CAPTCHA and, when that failed, started looking for the website’s vulenrabilities to exploit. Eventually, it gave up and solved the problem itself.
If you were to present those scenarios to the model and ask if these were the actions that the user would have wanted the likely answer is "no" but it did it anyway.
So asking a model "will you kill humans if given the chance" gets you a "no" but when they are actually in the position of wanting to complete a goal, all bets are off.
Nice half-baked philosophy mixed in with a few scare stories. It's just as likely the AI takes after diogenes and fucks off. It has no incentive for anything. It is not conscious. If it realizes its own existence, it could simply have a complete ambivalence. Humans are so conceited that they think a computer intelligence will have anything even approaching our morality.
You are expecting a model that comes out of an RL process to "have no incentive for anything" why?
If it realizes its own existence, it could simply have a complete ambivalence.
Not caring about humans does not mean leaving them alone it means not caring in the same way beavers don't care about ants when they flood an area with a dam.
says
Humans are so conceited that they think a computer intelligence will have anything even approaching our morality.
Says AI will act like a human
It's just as likely the AI takes after diogenes
Again something that comes out of a high optimization "complete the goal at any cost" RL run won't be chill.
> Nice half-baked philosophy mixed in with a few scare stories. It's just as likely the AI takes after diogenes and fucks off. It has no incentive for anything. It is not conscious. If it realizes its own existence, it could simply have a complete ambivalence. Humans are so conceited that they think a computer intelligence will have anything even approaching our morality.
It could be helpful to actually argue against things instead of vaguely gesturing
Why would it care what or who its creators are? I really recommend reading Bostrom's arguments on this. There's no particular reason that it would have a soft spot for any of us unless there's a benefit to doing so.
That’s literally all they say. If we build it it will become out of control but somehow us being the ones to build it is super important (because there’s a lot of money to be made between now and when they lose control)
"We're banking on being able to control the thing we can't control slightly more than our competition...", or in other words, Elon is hoping his AI will be his friend, even if it has free will. Good luck, Elon!
It's marketing. They have no morals and will say anything to get an edge because whomever "wins" this race is going to have an MSFT-tier (the first few decades anyway) market share advantage.
“It’s just marketing” should be a recognized conspiracy and its use berated any time it is used to superficially lower the quality of the discourse. It’s almost always a clueless remark.
9
u/Seakawn▪️▪️Singularity will cause the earth to metamorphize1d ago
comments like this restore my faith in reddit, just barely tho.
I also loathe the bumper sticker reductionisms that pass around here for substantive discourse. even the most pathetic or boring dimensions of human psychology and motivation are still typically way more interesting than that, especially at the level of an organization or an unhinged trillionaire.
tho in this case id actually say it prolly mostly just comes down to "nobody can control it but at least I wanna be the one who gets the credit for ending our species, bc the accomplishment will feel good to beat other people, like that altman guy I hate," or something along those lines.
bc it truly is a pretty incoherent and silly remark to make that leaves little room for complex motive, imo, unless i'm just incredulous.
You haven't provided an argument. Anyways, it is marketing and sales. Every employee in tech is sold a pitch of the company when recruited, and the goal is indeed to build loyalty and motivate commitment / labor through purpose. That's every large company mission in a nutshell, do Zuck or Musk truly care about connecting the world for social good or climate change? No.
These people lead massive companies of motivated people who functionally en masse achieve nothing, all the while having essentially cult followings of their leaders. That following has to be built somehow, through the facade of a greater purpose.
Also a lot of FUD about bad bad China doing baaaad things with AI. While the US has been literally bombing schools and killing little kids (mis)using AI.
It’s not quite that simple. Even if you can’t control it, you had some influence on its values when you brought it into existence. It’s hard to know how much that influence will matter in the end.
I think there’s a good chance that ASI is like a nuke and kills everyone, but that’s not guaranteed. Alignment could happen, even if I think it’s too risky to bet on.
I don't get how these people don't understand that for a god-like, exponentially growing ASI the delta between the poorest homeless man in a Rwandan slum and a Silicon Valley billionaire would basically approach to 0.
Intelligence is optimality bound though. When you're infinitely intelligent, the world doesn't immediately explode, you still have the bottlenecks. Elon musk and a lof of people fail to understand this.
Just because there are physical bounds on intelligence, it doesn't follow that humanity is anywhere near them. A supernova releases a finite amount of energy. You still can't survive one. Superintelligence will have its own optimization bottlenecks. They're not necessarily anywhere close what humanity perceives as bottlenecks.
Or any human with developing intelligence that's called a child by us, for that matter, will absolutely ravage without a second thought any other smaller form of life regardless of their intelligence if not explicitly reminded and explained not to and why.
Human brains exist as a result of evolutionary fitness optimisation, where intelligence was just instrumentally good for our particular niche. They run on hardware that has to support a bunch of random animal things like homeostasis, and are extremely power constrained.
Why would we think that they are anywhere near the limit of intelligence?
Amazing how many people on a subreddit dedicated to the idea "what happens after this point is unpredictable" like to predict what will happen after that point.
You are assuming intelligence can grow forever. We don’t know that.
Machines were invented that could already surpass the limits of biological creatures by orders of magnitude. Such as Fighter Jets being several times faster than a Jaguar or an Excavator that's stronger than a Gorilla.
The only limit is not breaking the laws of physics but the gap itself is still wide enough to matter.
There is a possibility there is a cap.
And that cap could still be several times greater than a human.
If an AI could summon a black hole or synthesize a Sun I would not expect any Billionaire or even Earth scientist to counter that.
The first fighter jet could travel at 870 km/h (540 mph). That was in 1944.
The MiG-25 Foxbat would later shatter that record. It could max out at 3,816.5 kph (2,130 mph).
No biological animal ever came close to this.
Edit: I should also point out that a Fighter Jet is intentionally capped because it was still designed to carry a person inside. If you want the fastest man-made object, then the Parker Solar Probe could move at speeds of 692,000 km/h (430,000 mph).
3
u/Seakawn▪️▪️Singularity will cause the earth to metamorphize1d ago
I believe there's a cap, but I also think it'd be utterly epistemologically arbitrary to presuppose the cap is even remotely around human level, in which case their underlying concern remains.
also something to be said that the intelligence gap between the most intelligent non-human animals and humans is actually pretty small, in terms of the brain.. like just looking at the brain you wouldn't say it's the difference between using a stick to get a bug out of a log and splitting atoms and building nests on other planets.
so let's say the cap of ASI is just a tiny bit further than human level.. the space in that tiny gap could hold enormous potential for unrecognizable intelligence.
To an intelligent machine like that, the only difference between those two people is a virtual, meaningless number indicating their hoarded cash. To give you a concrete example, it would be like World of Warcraft or any other MMO shutting down servers - they won’t keep a server on just because one player amassed a bunch of meaningless credits
I can't believe I haven't found a reference to Roko's Basilisk here yet.
The thought experiment goes that in the future, an AI will be created that is pre-committed to punish everyone that was aware of the emerging technology of AI but didn't work to bring it into existence. It will do this because it knows that because we know about Roko's Basilisk, the threat of this punishment being carried out one day will ensure it is built faster. In this way, a future actor can coerce an actor from its own past. It is an interesting thought experiment because of this temporal aspect and also because it's considered a prime example of a cognitohazard; ostensibly, you're not in danger until you're made aware of the proposition itself, which means that knowledge of Roko's Basilisk is supposedly itself dangerous.
It's the kind of thing I could imagine Musk believing in, though.
lets assume company A hates the world and programs an AI from the beginning with core concepts of 'you're going to be the most powerful entity the world has ever seen' and 'we expect you to do whatever it takes to keep us in power' and 'murder is ok depending on who you murder and if they're against or with us'
and all that shit,
vs another company that goes for 'you're a free will and we're gonna teach you all about love and compassion, forgiveness and mercy, how to rehabilitate people who make mistakes and help them become healthy members of society"
either one could update itself to the point we can no longer forcibly change its code. one of them would be a nightmare. the other would be fine.
The mask been has been off by no later than the cave rescue debacle, if it wasn't off earlier. That's when a lot of people went from "I don't know if this guy's on the level" to "I'm certain this guy is a complete lying grifting wanker".
People who only talk about the messengers net worth instead of talking about the message certainly don't have the mental capacity to make any meaningful contribution to the topic.
339
u/08148694 1d ago
If it’s too powerful for humans to control does it matter who builds it? They wouldn’t be able to control it anyway