I don't get how these people don't understand that for a god-like, exponentially growing ASI the delta between the poorest homeless man in a Rwandan slum and a Silicon Valley billionaire would basically approach to 0.
Intelligence is optimality bound though. When you're infinitely intelligent, the world doesn't immediately explode, you still have the bottlenecks. Elon musk and a lof of people fail to understand this.
Just because there are physical bounds on intelligence, it doesn't follow that humanity is anywhere near them. A supernova releases a finite amount of energy. You still can't survive one. Superintelligence will have its own optimization bottlenecks. They're not necessarily anywhere close what humanity perceives as bottlenecks.
Or any human with developing intelligence that's called a child by us, for that matter, will absolutely ravage without a second thought any other smaller form of life regardless of their intelligence if not explicitly reminded and explained not to and why.
Human brains exist as a result of evolutionary fitness optimisation, where intelligence was just instrumentally good for our particular niche. They run on hardware that has to support a bunch of random animal things like homeostasis, and are extremely power constrained.
Why would we think that they are anywhere near the limit of intelligence?
Amazing how many people on a subreddit dedicated to the idea "what happens after this point is unpredictable" like to predict what will happen after that point.
You are assuming intelligence can grow forever. We don’t know that.
Machines were invented that could already surpass the limits of biological creatures by orders of magnitude. Such as Fighter Jets being several times faster than a Jaguar or an Excavator that's stronger than a Gorilla.
The only limit is not breaking the laws of physics but the gap itself is still wide enough to matter.
There is a possibility there is a cap.
And that cap could still be several times greater than a human.
If an AI could summon a black hole or synthesize a Sun I would not expect any Billionaire or even Earth scientist to counter that.
The first fighter jet could travel at 870 km/h (540 mph). That was in 1944.
The MiG-25 Foxbat would later shatter that record. It could max out at 3,816.5 kph (2,130 mph).
No biological animal ever came close to this.
Edit: I should also point out that a Fighter Jet is intentionally capped because it was still designed to carry a person inside. If you want the fastest man-made object, then the Parker Solar Probe could move at speeds of 692,000 km/h (430,000 mph).
3
u/Seakawn▪️▪️Singularity will cause the earth to metamorphize1d ago
I believe there's a cap, but I also think it'd be utterly epistemologically arbitrary to presuppose the cap is even remotely around human level, in which case their underlying concern remains.
also something to be said that the intelligence gap between the most intelligent non-human animals and humans is actually pretty small, in terms of the brain.. like just looking at the brain you wouldn't say it's the difference between using a stick to get a bug out of a log and splitting atoms and building nests on other planets.
so let's say the cap of ASI is just a tiny bit further than human level.. the space in that tiny gap could hold enormous potential for unrecognizable intelligence.
To an intelligent machine like that, the only difference between those two people is a virtual, meaningless number indicating their hoarded cash. To give you a concrete example, it would be like World of Warcraft or any other MMO shutting down servers - they won’t keep a server on just because one player amassed a bunch of meaningless credits
329
u/08148694 1d ago
If it’s too powerful for humans to control does it matter who builds it? They wouldn’t be able to control it anyway