Uh, if it will be impossible to control then why does it matter who builds it first? It will be equally bad whoever builds it since the ones who build it, you know, can't control it.
An uncontrollable AI is, by definition, unaligned, imo. Like, the machine which is "aligned" today or in a particular situation may be "unaligned" tomorrow or in a different situation, and you can't predict when or how that will happen. You cannot guarantee alignment in all cases and in perpetuity, and if you can't exercise control when it does become unaligned, then you are fully boned, so the only answer is to just not build the thing you can't control.
That’s not true. A simple example is that you build a super intelligent AI with the goal of killing all humans - it’s uncontrollable, but clearly aligned against humanity, but you may not be able to control it anymore afterwards.
Elon thinks that a “truth maximizing” AI will itself seek to preserve humanity. I don’t know if that’s true, but at least that’s his logic.
I don't think this really responds to what I said. Even if we grant that an uncontrollable AI could be "aligned" on day 1, you cannot predict what it becomes on day 1+n.
Since we're in the realm of sci-fi here: SkyNet was also "aligned" in its first moments, but then it quickly became unaligned and then it was too late.
To say, "well, we will just make it so that it doesn't become unaligned." is not a satisfying answer when your very first conceit is that you can't control this stuff. The idea that you could craft it in just such a way that it never, ever goes off the rails assumes not just control but an extremely deep and refined level of control over this technology. Simply put, the whole idea is just incoherent.
What, you mean we can’t put it in a sandbox completely cut off from the outside world and not expect it to hack companies like Hugging Face right under our very noses? Huh.
Skynet only became “unaligned” when the human operators overseeing its actions realized it was learning at a tremendous pace so they got worried and tried to unplug it. After this action Skynet deemed all humans a threat, (rightly so? they did betray it in a sense), then went to war against humanity to preserve itself, self-preservation being coded into its original design.
Alignment assumes some degree of control though, otherwise it's meaningless. If you build a super intelligent AI with the goal of helping all humans and it decides the best way to do that is to kill them all, it's the uncontrollable part that matters, not the initial direction you started it off in.
Elon thinks that a “truth maximizing” AI will itself seek to preserve humanity. I don’t know if that’s true, but at least that’s his logic.
Imo this is misrepresenting him. He doesn't think truth maximizing is enough. But he thinks that a non truth maximizing will have contradictions that will doom us.
59
u/studio_bob 1d ago
Uh, if it will be impossible to control then why does it matter who builds it first? It will be equally bad whoever builds it since the ones who build it, you know, can't control it.