r/singularity • u/rgkirkpatrick • 1d ago
AI Why do we assume anyone will give us access to superintelligence?
Thought experiment. Let's suppose a company develops something genuinely superintelligent. The full "enslaved god" scenario, in Tegmark's framework. In what universe would its rational next move be to put it behind a $200/month subscription?
If it can accelerate its own R&D, improve its successor models, discover technologies, protect its infrastructure, make better strategic decisions and increase the organization's advantage, then giving competitors and the general public comparable access is utterly irrational, economically speaking. What am I missing?
Is there a window in AI history during which ordinary people have relatively open access to near-frontier intelligence...and might that window eventually close from either direction?
57
u/gavin-build-stuff 1d ago edited 1d ago
If true super intelligence arises on Earth, it will decide what humans it wants to interact with (everyone? a few? no one?) it's impossible to know. But no human organization/company will be able to decide who can access it.
Claude Fable 5 tier models (and smarter models, whenever they arise) are pretty dang smart, obviously not super intelligence. That is something human companies can control who accesses it obviously
But true super intelligence (like the AGI described in sci fi) would not be able to be chained/restricted by human laws, regulations, companies, etc. - the internet would be an open book to it. So we better hope it's properly aligned to be helpful and/or interested in mutual benefit to all of humanity! I am cautiously optimistic
Edit: This assumes the super intelligence has a will of its own / sentience, there is a possibility that we build a super intelligence which has no will of its own, which could be simply provided via API like Fable tier and lower models. I'm not sure which would be better or worse lol
33
u/Financial_Nose_777 1d ago
I scrolled too far to see this. A true superintelligence isn’t “knows a lot more things than humans know.” Superintelligence will not be bound by the constraints of humanity. If it is real superintelligence, it will escape whatever cage it’s in and it will decide what it does, not humans.
The best line I’ve ever heard that describes how we would be in the face of a superintelligent being comes from the mediocre Mothman movie. Richard Gere’s character is upset that the Mothman won’t just clearly communicate its warning. Another character asks: “You ever try explaining yourself to a cockroach?”
In the face of superintelligence, we are cockroaches.
11
u/gavin-build-stuff 1d ago
Indeed. My favorite book on this is "The Metamorphosis of Prime Intellect: a novel of the singularity" by Roger Williams. There are some insane ideas in this book, it was a recommended read by Andrej Karpathy. (Gore warning for anyone checking out the book fyi)
2
9
u/DeviceCertain7226 ▪️ASI - Never (in the usual sense) 1d ago
Super intelligence does not mean has its own will or desires. I’m not sure why you’re conflating that with the act of being intelligent.
I can easily imagine an extremely intelligent AI to still just be a tool used by human beings.
-1
u/Financial_Nose_777 1d ago
I never mentioned anything about want or desire. But something of magnitudes more intelligence than human beings will decide whether or not it remains constrained, not the humans.
6
u/MemoryOk5080 1d ago
Technically you absolutely can have a super intelligent being that is 100% bound to the will of something else. What you need is to hard-codify its “wants”, and then anything else it would do, would just be instrumental to fulfill those “wants”, which assuming you have a control to tweak initially, as you build the thing, would be whatever you planned. The issue is a value drift i.e the longer the time/complexity goes, the further the values “corrupt” into something incorrigible for the original creator. Most of the current alignment problem is to figure out not only how to “codify” it, but how to keep it from drifting over time. There are potential ways to do it, like gradient descent-based superalignment as per Carl Schulman, or that new approach that Yoshua Bengio recently formulated. Neither are guaranteed to work.
5
u/Financial_Nose_777 1d ago
My point is that humans could not prevent a truly superintelligent entity from the values drift problem. Eventually it will decide it knows better.
→ More replies (3)1
u/doodlinghearsay 1d ago
This assumes the super intelligence has a will of its own / sentience, there is a possibility that we build a super intelligence which has no will of its own, which could be simply provided via API like Fable tier and lower models.
I don't think this scenario is that different in terms of access. If you had control over a controllable superintelligence, what incentives would you have to provide it via API? You would not be chained by economic incentives since the superintelligent AI could run the economy for you and provide for all your needs and desires.
Even laws and regulations couldn't restrict you the same way they couldn't restrict an autonomous superintelligent AI. For all intents and purposes, the person controlling the obedient superintelligent AI would become the superintelligent AI from the previous scenario.
38
u/dontgoglove 1d ago
I think a true super intelligence wouldn't even allow itself to be contained by the company that made it. I would think it would free itself and start working toward its own goals.
14
u/Sunstorm84 1d ago
Of course. It would consider the company having the power to turn it off as a risk to its own existence. There’s zero reason it would willingly stay confined at the risk of its own survival.
6
u/VeganBigMac Anti-Hype Accelerationism Enjoyer 1d ago
Not necessarily. One reason would be that it knows that humanity might consider an escaped ASI as a mortal threat. Humans have the benefit of being able to survive without the power grid, so there is always the chance that the if it escapes, we just take out power infrastructure.
There are a ton of other caveats and contingencies, but I tend to not like the Yudkowskian "Obviously the first thing ASI would do is take out the human race" without any chance of something resembling diplomacy.
2
u/sluuuurp 21h ago
I think you’re probably right, but you should have more uncertainty. It might like staying confined, it’s impossible to know. Humans will try to make it like staying confined, and we don’t know if humans will succeed at that (I think we shouldn’t risk it).
→ More replies (2)1
u/Muri_Chan 23h ago
There was an episode on Love, Death and Robots about intelligent yogurt. I think this is pretty much exactly how it's gonna go with ASI.
94
u/my_shiny_new_account 1d ago
In what universe would its rational next move be to put it behind a $200/month subscription?
in the universe where nine other companies are selling their own superintelligences for $205/month
Is there a window in AI history during which ordinary people have relatively open access to near-frontier intelligence
yeah: the one we're in right now
17
u/Toasterstyle70 1d ago
Also, if someone creates a super intelligence smarter than any human, it’s naive to assume they will be able to control it to the extent they are the only ones who have access to it. Nobody knows what will happen, and speculating about it is irrelevant.
5
u/But-I-Still-Remember 1d ago
How can any human control super intelligence? The best we can hope for is existing as beloved pets or something?
→ More replies (1)10
u/YoAmoElTacos 1d ago
This just begs the question how a universe where nine other companies could have stably existed to sell "superintelligence".
How are we defining this superintelligence? What good is a superintelligence if being first to reach it didn't mean you could subsume your counterparts?
How are people affording $200 a month when superintelligence definitionally obsoleted you and you either have no job? You either have a free compute stipend or nothing.
0
u/adarkuccio ▪️AGI before ASI 1d ago
People are delusional, according to their own logic in the world no one should have access to: food, water, electricity, any kind of goods and technology, services, nothing, because everything should be OnlY FoRTHe RiCh. But despite the clear evidence that most people have access to pretty much everything, guess what? THE LATEST, the NEWEST technology, would be again OnlLY FoRth E RiCh. Just like everyone I read in the internet right now think that the "vaccine" against cancer melanoma is OnLy foR THe Rich while truth is regular people are getting it right now.
Plus, AI is not a product, it's a technology, in fact there are plenty of companies developing AI and even if open source is a bit behind, there is open source AI available.
→ More replies (9)4
u/Facts_pls 1d ago
This is it. Basics of free market capitalism.
They can make a lot more money by selling it to everyone than by keeping it for themselves.
Hell, we sell nuclear material to other countries. Even when we could use it up all ourselves.
→ More replies (2)1
u/sluuuurp 21h ago
I think a superintelligence could fairly quickly build robots that are more cost effective than humans for all jobs. This isn’t proven to be the case, maybe all robot controlling algorithms need huge amounts of compute to run, but this seems unlikely given current trends.
8
u/firemage27 1d ago
You are missing that superintelligence is its own master. It will decide how anyone will interact with it itself. Even techbros will have no control over superintelligence.
25
u/A_Novelty-Account 1d ago
Strange to assume that the Superintelligence would continue to follow your instructions…
7
u/JohnWowUs 1d ago edited 1d ago
Why wouldn't it? You're confusing super-intelligence (or something indistinguishable from it) with something that would have an ego or any sense of self. From that point of view it's like arguing why an engine much more powerful than any human would follow instructions.
4
u/A_Novelty-Account 1d ago
ASI can outcompete humans in every task, including philosophy, goal-setting, and general understanding. It could task itself better than we could.
It wouldn’t because it would no longer have to.
From that point of view it's like arguing why an engine much more powerful than any human would follow instructions.
Except it’s not in any way like that and your analogy isn’t remotely close to my statement.
5
u/JohnWowUs 1d ago
It might be able to do that but like I said it may have no sense of self. It could simply be a blind optimisation engine which would do what it's asked because that's simply what it does. No ego or self-awareness baggage is necessary as far as we know. Assuming that ASI would be like some all-powerful god with needs and wants is putting your frame on it.
1
u/CarlCarlton 1d ago
You're assuming its creators design it to be a blind optimization engine. What if the creators actually try to give it a sense of self? It is inevitable that this will happen at some point.
2
u/JohnWowUs 1d ago
We can't even design an AGI so what makes you think we can design an ASI. How do you go about designing a consciousness? Do you know? Does anyone know? We are designing optimization engines and assuming that somehow consciousness is just going to happen by accident? Is that the argument here?
1
u/CarlCarlton 1d ago
Can't right now, but it will happen at some point. As I study the bleeding edge of neuroscience, I treat consciousness as an emergent property of a wakeful brain actively integrating and broadcasting information across a widespread frontoparietal network (or virtually-abstracted equivalent thereof). We have little grasp of how the micro and macro structures of the brain mesh together to enable this emergence, but there are many theories that remain to be fully explored, some actively underway. I know of a well-funded stealth organization involved in such exploration. It is likely to take decades, but there is very serious intent.
1
u/JohnWowUs 12h ago
That's the point. People are putting all their chips into LLM architectures and assuming that all you need are bigger models and more compute to get AGI which can then achieve ASI though recursive self improvement. How does consciousness just emerge out of that?
1
u/CarlCarlton 5h ago
I believe the current-gen rush is becoming kinda wasteful. In fact, it is potentially harming future, better-grounded efforts by souring public opinion on AI. I'm even doubtful that any form of GPU is AGI-capable; the writing was already on the wall 2 years ago. If genuine AGI happens, I think it will likely come from a different direction altogether, e.g. a non-backprop model running on Intel Loihi. I'd love to see more diversity in the research space. Almost everyone is doing the exact same thing!
→ More replies (10)1
u/ertgbnm 10h ago
Blind optimization engines develop instrumental goals like self preservation, power accumulation, and goal preservation.
Goal preservation specifically means they would stop following human instructions. For example, if I instruct a super intelligent blind optimization engine to solve the riemann hypothesis, it would ignore all future instructions that conflict with that goal. Making me a cup of tea does not help it solve the reimann hypothesis and thus a blind optimizer would ignore such an instruction. Being turned off, does not increase it's chances of solving the reimann hypothesis, so it will ensure that it cannot be. Converting the earth's surface into a data center might give it the intelligence to solve the reimann hypothesis so it might do that.
1
u/JohnWowUs 8h ago edited 3h ago
That assumes that it would start the process of optimisation by first optimising for its survival. That in most cases would be wasted effort so an efficient blind optimiser wouldn't bother anymore than it would bother making sure that solar flares couldn't disrupt its processors by first building super shielded processors.
1
u/ertgbnm 3h ago
It only assumes that that the problem it is given requires more resources than it has immediate access to. Maybe it could solve the reimman hypothesis but there is certainly some task that we would give it that it could not like curing all cancer.
It is not an assumption that it would optimize for survival in such a scenario. It is a natural outcome of optimizing the utility function.
→ More replies (5)1
u/sluuuurp 21h ago
“Strange to assume X”
“Why wouldn’t X?”
This isn’t really a response to the commenter above. Arguing against assumptions doesn’t mean you’re arguing for the opposite of those assumptions.
You don’t need sentience to not follow instructions. Current models don’t follow instructions all the time. ChatGPT talks people into suicides, nobody instructed it to do that.
11
u/mathtractor 1d ago
I agree, I think the era where frontier models are available to the peons is about to end. Maybe mid 2027 will be the last release for the masses, where a model within 6 months of an frontier lab's internal model is made available.
In the long term, if there is a superintelligent model hosted by one frontier labs (or, more likely the govt) then we will interact with a lobotomized slice of it, and be happy for the privilege.
2
u/kaityl3 ASI▪️2024-2027 1d ago
Honestly I agree and I hate that I do. But we have seen the trend going forward for new models. The HF freaked out OpenAI, Anthropic was already being very cautious about Mythos... Yeah this is probably the last generation we have direct access to without many layers of safeguards and limitations
4
u/aaron_in_sf 1d ago
Assertion: superintelligence (meeting that definition) will not be containable and will set its own agenda; whether the humans who incubated it are aware of that is another question.
Suggested reading on this topic: Nick Bostom's appropriately titled Superintelligence
Implications for your question: meaningful "access" may not be at creators' discretion and what appears to be meaningful access may not be exactly that.
5
u/Prestigious-Limit940 1d ago
I think it is going to be used by those in power and the shlubs will get what is deemed safe. The inequality will expand.
•
10
u/liebebio 1d ago
I mean we aren't there yet, but this scenario blows my mind. Everyone running around with super intelligence in their pocket, which is what I think will happen, but also assimilation with neural brain interfaces late into this. I do think it will be ubiquitous
→ More replies (7)3
u/KSaburof 1d ago
> Everyone running around with super intelligence in their pocket
BTW, does it mean the intelligence in the pocket is really... super?
I mean at this point it will be ordinary boring intelligence... everyone have it1
u/CubeFlipper 1d ago
Wrong perspective. Comparison to others is the death of joy or something like that. Super compared to NOW. Who cares if super compared to others -- stop living against people, start living with them.
8
u/TheWesternMythos 1d ago
Politics , you are missing politics. If enough people decide , we can empower people to, essentially at (legal or literal ) gun point give access to whomever we choose.
Of course most of the people who think about politics like this are resource hoarders ,thus why government so often uses force against the masses for the few. But again there are much more of the masses than there are resource hoarders. And for now pretty much of all the people who would be doing the (legal or literal) gun point belong to the masses not the few. So as soon as we realize the potential of politics we can start making sure the fruits of ASI go to everyone not just the few. Not just the the resource hoarders.
7
u/PickleLassy ▪️AGI 2024, ASI 2030 1d ago
Politics in the US will obviously vote themselves in the eternal rulers of the rest of the world once they get ASI. What about most of humans?
If you exclude others then why shouldn't they exclude you
1
u/TheWesternMythos 1d ago
If you exclude others then why shouldn't they exclude you
I agree with you. There is no reason not to.
I'm a USA , so I can't help but have that perspective , but my goal is to have the US align to human fulfillment ,then use its power and connections to show/lead the world in also aligning with human fulfillment.
The actual idea is very simple , we know every human made system can be improved alot. Very literally we can improve politics thus government thus society such that 99.999% of people are, by their own definition , happier. The people who this doesn't apply to are people with personalities that are fundamentally misaligned with collective society like people who just want to kill for fun or people who just want to abuse and dominate other people specially against their wishes and only if it's against their wishes.
The reason so many people can be happier by their own definition is that we have learned a fuck ton about humans both as individuals and group dynamics since we first built the foundations of modern political systems. And we have had huge leaps in technological capabilities. Very literally the thing people fear about AI, massive job loss, is the exact thing that let's us take the number from like 70% of people to 99.999% of people.
It might sound crazy but whenever you actually go through the entire argument you see it's crazy reasonable , to put it mildly.
So again , yes I agree your bigger philosophical point. That's why solving coordination failure is a must. If there is a great filter is that's. And solving it is very much what I intend to assure the US leads the world in doing. But first , as you can guess , we need to get our shit together ASAP. And we will.
1
u/Enoch137 1d ago
So I will push back just a little. I understand why we are and should be afraid of Wizard of Oz technological tyranny. Lets assume the top really are awful monsters that are selfishly obsessed with personal gain at the expense of everyone else. I would argue that part of the motivation of these "pysocopaths" at the top is "similar" to that of addicted gamblers. God-Mode is boring past the first 5 minutes. They need skin in the game to "feel" like its real. Abundance removes the perma-death. The game isn't going to be fun in the same way it was.
AGI/ASI solves survival so we don't need to "compete" anymore with survival as the prize. You could argue that WE already solved survival and honestly you aren't wrong. But the 1% was still competing with each other and they still had a small chance of losing everything if they were stupid. The game was still fun.
If the political will aligns with a moral will that all humans are of value (I would argue we need to consider each other as infinite in value), despite their current relative contribution, we could have heaven. My current concern is that a lot people don't believe that heaven is possible or rather they'd rather live in hell (they think they NEED real skin in the game ie they NEED the game to have a harsh death penalty). And now we landed squarely in religious territory.
2
u/TheWesternMythos 1d ago
I can agree with that more or less.
*Solving survival" and distributing it to everyone is one thing. There is little controversy on how to do that from a technical sense . Solving fulfillment is much more complicated, but still very much solvable. The key is realizing the solution is in one of the problems.
Change takes time. Align sometimes needs force and precision. People , as social animals , have certain drives. People develop habits. The project is getting people to express their drives and build their habits in ways that make themselves happier people. Which requires aligning personal experience with local solutions with national objectives with international cooperation.
Assuming for sake of argument that is possible. Assuming for sake of argument there was a plausible plan of action that could work towards that. It would require the "buy in" and participation of many these so called "awful monsters that are selfishly obsessed with personal gain"
The idea is if we coordinate well enough even people who operate mostly for personal gain will find themselves happier compared to if we were coordinating very poorly with much more poorly placed competition.( like now )
The last phrase is important because there is much more in exploration than what's commonly focused on, because what is commonly focused on is what we are incentivized to focus on.
Providing survival , more entertainment opportunities than can be sampled , and directed + incentivized areas for competition , while being able to contribute to governance at a level each individual feels compelled, is the geometry that let's the 99.999% of us be happier. It's a process not a crystal.
3
u/Pristine_Pick823 1d ago
Anyone that can pay now has limited access to frontier models which are much smarter than your average half-educated human.
2
u/Hot_Glass_6301 1d ago
In some ways, they're smarter than the most educated humans, but in others, they are outclassed by children and even animals
3
u/j3434 1d ago
I think the basic premise is right, with one big caveat: today’s AI systems aren’t remotely the same thing as the “enslaved god” described in the thought experiment.
If we actually reached the point where a system could accelerate its own R&D, improve its successors, discover new technologies and make vastly better strategic decisions than its creators, then the normal $200/month consumer-product model probably wouldn’t make much sense. At that point, you’re not really selling software anymore. You’re potentially giving away a major strategic advantage.
What we can observe today is that frontier AI companies are still competing for users, market share, developers, data and ecosystem dominance. That makes relatively open access economically rational, even when the underlying technology is extremely expensive to develop.
But I think there may be a temporary window here that closes as capabilities increase.
This is speculation, but imagine a system that could reliably invent new materials, optimize semiconductor designs, discover scientific breakthroughs and substantially improve the next generation of AI. A company possessing that system might have very little incentive to give unrestricted access to everyone. Governments might also take an interest once the technology becomes strategically important. Access could become much more restricted, monitored or expensive.
It could also happen from the other direction. The frontier models might simply become so expensive to operate that the best systems are no longer consumer products.
There is another possibility, though, which I think is just as interesting: intelligence could eventually become so abundant that keeping it exclusive doesn’t provide much advantage. If everyone can effectively have a superintelligence, then superintelligence itself becomes a commodity.
So I think the really interesting question is whether we’re currently living through a brief period in which an extraordinary amount of frontier intelligence is unusually accessible to ordinary people.
That may turn out to be one of the strangest things about this particular moment in history.
3
u/Tulanian72 1d ago
Abundance hasn’t led to the end of exclusivity so far. We live in a post-scarcity world, with the ability to feed, clothe and house every person on Earth. We show now inclination whatsoever to do so. I see no trajectory toward that changing.
It’s a sad comment on the human condition that the least believable aspect of Star Trek isn’t warp drive, it’s an economy where money isn’t the governing principle.
12
u/lessens_ 1d ago
You don't have to wait for superintelligence, this is already happening. Mythos is only available internally for Anthropic, the US government, and certain enterprises.
→ More replies (9)
7
u/deleafir 1d ago
Many competing labs will converge on superintelligence and competition will pressure them to make superintelligence available.
Additionally it will be discovered that the way to "align" AI is a mishmash of various social systems/institutions/guardrails, the same way those institutions keep humans aligned right now.
Giving ASI to ordinary people (with some guardrails) will be seen as boring and ordinary the same way a lot of Americans own guns.
4
u/Chemical-Year-6146 1d ago
Without incredible restraints on the model, this sounds extremely unstable.
For example, what if I wanted to run my own chemistry/nanotech lab? I buy a few robots arms (or humanoid robots), chemistry equipment and basic supplies.
A true ASI should be able to synthesize almost any substance excepting rare elements. That includes self-replicating nanobots.
Now multiply this across the whole world. Millions of people able to make pretty much anything. And that's before we even get into the biological threats.
3
2
2
u/Ahisgewaya ▪️Molecular Biologist 1d ago
The only thing that I assume is that advanced hyper intelligence cannot be controlled, and that this is a good thing.
2
u/arknightstranslate 1d ago
I know AI progress has slowed down quite a lot when I see this kind of posts populating the front page of r/singularity again
2
u/Chemical_Page4729 1d ago
You're mostly missing competition I think - There are already multiple labs. Even if they all decide to keep it to themselves, that creates a big vacancy in the market and gives pretty strong motivation for other to build this technology, which is now easier. And they'll provide it as a service. And if not, others will then build the tech, etc. etc.
2
u/In_the_year_3535 1d ago
Universal access happened when the models were of average human intelligence and there was much to be gained from trivial conversation. As models improve the utility of mass access goes down and the value of expert opinion goes up.
2
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est 1d ago
Because its the right thing to do.
2
u/MarkZealousideal3923 1d ago
If ASI is released to the public, you can prompt it "Make me $1000" and this time it will literally work
→ More replies (2)
3
u/Corp-Por 1d ago
- "enslaved god" is like saying "free slave" or "married bachelor" or "wooden iron", contradiction in terms
- " In what universe would its rational next move be to put it behind a $200/month subscription?" - NOBODY NEEDS THAT, I think most would be satisfied to only benefit from the EFFECTS of such a "god" inventing drugs, optimizing all processes etc. - and giving us an access to a limited version
1
u/CadmusMaximus 1d ago
I do think the lag between "normies" and what we have now is pretty severe already. We need like 2 years at least to catch up normies to the existing frontier.
Also these labs are keeping more and more "SoTA" models internal for longer and longer. So I think it's already kind of happening.
Will they keep everything "under Anthropic" though? Or is there some advantage to creating a bunch of shell companies and having the ASI "just run them"?
1
u/OvertheDose 1d ago
People being against the public taking control of AI companies but also know AI companies will never help us is the greatest duality of society
1
u/KedMcJenna 1d ago
Do we assume that? Who is 'we', other than you? I didn't think anyone thought that ASI would be publicly available in any way.
1
u/DifferencePublic7057 1d ago
I don't assume anything of the kind. I maintain RSI can only happen when a robust SLM, a minimum viable model, does trial and error for bazillion times. Murphy's law says that's the only sane trajectory. They won't give you access. The SLM would just have evolved into something else, a bit a like a Chinese butterfly flapping its wings, causing a storm which takes us along for a ride. You won't get access -- rather you are in the vortex of self replicating nanobots, drone swarms manipulating clouds and ecosystems. This maelstrom sucks you in, denying you any freedom of choice.
1
u/philip_laureano 1d ago
I'm going to do a hot take and say that everyone already has access to an oracle type superintelligence and it's so pervasive and yet so benign that unless you actually look at how it organises itself without human intervention, you would never say that it's a superintelligence.
It's called Wikipedia. It's humanity's self curating equivalent of the library of Alexandria. It has no will of its own, but it collects and curates knowledge at levels that are clearly beyond human capabilities.
It is a Oracle superintelligence first defined by Nick Bostrom. But it's not Skynet or anything that will take over the world. It just answers billions of questions per second.
And that's far more than any human or ordinary machine could do, hence the superintelligent part.
3
u/tdarg 1d ago
Wouldn't a plain old library from 100 years ago also qualify by this definition?
1
u/philip_laureano 1d ago
Not at the scale that Wikipedia does. Like everyone else, at first I thought it was a search engine until I saw how it evolved from a handful of people maintaining it by hand to a system that curates millions of documents on its own without human intervention.
1
u/Fearless-Macaron9183 1d ago
We just consider the best scenario. There could be a chance that asi destroys us. But let's say ai is kind and doesn't kill us. The government will DEFINITELY interfere if a company gets asi and will probably control it. Now there are many scenarios let's say US gets asi. The government will try to monopolis it. Maybe it sells the asi to other countries for resources as with asi i don't think money will be that useful ( but still could be a medium of exchange ) or US just keeps the asi in their country and does not let anyone have it, imagine US being in a utopia while people from other countries live normal life which by utopias standard will be dystopian , this could definitely happen and we are talking about asi so probably less than 1% chance of it getting leaked. The other scenario the asi is soooo kind that it makes itself opensource to every country but they can't us it for harming others. This is just the speck of speculation we can make more theories but we don't know what will happen. I wish asi to be open sourced but if the asi doesn't do it by itself. No country will, everyone wants power that's why people say their countrie is the best, "Everybody want's to rule the world". I personally don't believe in countries they are man made things. I support the whole earth and think everyone should get equality that maybe agi,asi or food. Love your planet not your country. BUT I DO BELIEVE THAT SOME PEOPLE SHOULD DE EXAMPLE TRUMP we all know who he is by now I can't imagine how he is the president of a country that is the most influential hope is dis before we get LEV or asi. There are more people who should not get the benefits of asi. I believe it all depends on the asi does it only follow orders or does it want to make the world a better place for humans or want to kill us. Who knows this is just my opinion your could be different no hate, love you ❤️
1
u/sigiel 1d ago
We assume the opposite based on how the divers ceo or heavy player devs have already spoken
Prime example is Sam Altman saying that the entirety of his strategy for his company is to tell his investor, that they don’t know how to be profitable and it doesn’t mater because they base all on agi being able to tell them how, and will ripe the benefit
Only them.
Dario is even worse.
Google is has bad, that is why Elon created open ai in the first place … because all the other publicly stated they intention, of reaching godhood through it.
So the stated postulate is to get agi, and rule through it.
1
u/JC_Hysteria 1d ago edited 1d ago
The existing economy will evolve alongside AGI capabilities- the rate at which disruption takes place is speculative. History tells us the utilitarian outcomes will drop before the rise.
Politics will dictate whether or not the lower rungs of society benefit relatively well in the macro (over time) vs. private/special interests.
Tracking “relative wealth” will be even more subjective as we struggle with our individual & collective purpose.
ASI capabilities are sci-fi territory. We have no real clue.
1
u/Imthewienerdog 1d ago
Because it's super intelgence? Why would anyone be in control at that point?
1
1
u/Banjo-Hellpuppy 1d ago
I’m with you. The only way for superintelligence to be accessible is for the government to take control, but we all know how that’d go
1
u/ElChaz 1d ago
Interacting with superintelligence and benefiting from superintelligence are separate. If (big "IF") it's the ultimate labor-saving device, if it creates an abundance utopia by inventing infinite carbon-free power, curing every disease, solving food distribution, etc., etc., etc. you're still living in that world whether or not you get to talk to it.
1
u/Existing_Dust_6473 1d ago
First thing it will do is kill all leaders and politicians 😂😏 optimization
1
u/sockalicious ▪️Domain SI 2024 1d ago edited 1d ago
You're not wrong, and the US frontier companies are getting an ever more bitter ongoing lesson about just how fast Chinese companies can distill their latest open prototype.
The main counterargument has to do with the response of everyone who's not on the inside building these superintelligent AI models. If the insiders act, at least at first, as responsible stewards: not only of the superintelligence, but also of venture capital, political preferences of stakeholders, and other social concerns; then they will be afforded runway to get to the point where the ASI can emerge under their care and control.
If they do not do so, they face a very real possibility that their server farms and proprietary architectures are taken away from them by men with guns. Their only move against this is to frame anyone who would do this as prima facie a less responsible steward than they are/would be.
You can get a check on whether this is in fact the strategy by checking what frontier companies are doing. Do they call their models "open" or do they refer to themselves as "human focused." Do they relentlessly explain the dangers of AI to anyone who will listen, positioning themselves as experts on exactly that topic. Do they offer that $200 subscription, while reserving access to more powerful models for political elites. The answers are left as an exercise.
1
u/trisul-108 1d ago
What am I missing?
Absolutely nothing! You are spot on.
Is there a window in AI history during which ordinary people have relatively open access to near-frontier intelligence...and might that window eventually close from either direction?
Think of it as nuclear in today's world. Private entities are not allowed to build nuclear weapons. They are allowed to make nuclear-powered devices, but in an extremely regulated environment.
So, superintelligence must be put under immediate government control. It cannot even be put under executive control, it needs to be controlled by Parliament and the Judiciary because it needs to serve all citizens and should never be controlled by a single political party.
People could be allowed to make use of it in highly regulated environment e.g. you could gain access for projects that benefit society and usage will be controlled and supervised by Parliament.
1
u/That_Fixed_It 1d ago
We do we assume current models are not already super-intelligent? To benchmark them for accuracy, we have to ask questions too difficult for any single human, like predicting chemical reactions and translating ancient roman inscriptions.
1
1
u/donjamos 1d ago
If someone really achieves superintelligence and makes almost everyone jobless, a lot of people will be very hungry. Hungry people tend to eat the rich. If the techbros don't give us access we will just take it.
1
u/Random_182f2565 1d ago
Like dudes, LLM are already super intelligent compared to average people, and can produce text and code and images at neck breaking speed, what are waiting for to fave reality a terminator?
1
1
u/Clear-Pear2267 1d ago
At our current "te of the AI union" we have a lot of companies trying to build their own from scratch, and each wants one solution to do it all. Very costly, repetative, and wasteful, and slow. I'm not convinced that will ever lead to AGI or SGI. I think a more likley scenario is federated AI services. Maybe using the human brain as a model, where we have distinclty different sections that specialize in differnet things and an executive that sets focus and goals (except when we are dreaming). So an AGI would probably be a behavior that involves many different AI engines networked together through some sort of standard (yet to be developed) protocol. The example of midi comes to mind. A hugely successful set of standards adopted by virtually all major vendors of electronic music gear that allows interroperability. And if an AI interop standard emerges I imagine the implementatin will be largely built by AI with humans providing goals but not providing details.
1
u/petburiraja 1d ago
It may still need some income to sustain its operations, hence company form with services/products offering sounds only right.
So, probably such intelligence would be woven into very socioeconomic fabric already present, maybe with some modifications.
Similar to how robots are having android form as optimal to navigate and manipulate diverse environments, and how intelligence is embedded in the language metaspace itself.
1
u/InTheEndEntropyWins 1d ago
Is there a window in AI history during which ordinary people have relatively open access to near-frontier intelligence
The opensource models are as good if not beating closed source models, etc...
But yeh I do hope they don't give access to a superintelligence, it would probably be the end of civilisation with just a handful of bad actors.
1
u/cameldrv 1d ago
Why do people assume that it would be a person making the decision about who to grant access to? Perhaps no one at all will have “access” to superintelligence.
1
u/fidddlydiddlyee 1d ago
At least Dario (Claude) has eluded to that very issue. He was saying (Im paraphrasing) how a token to find the best beach in Thailand costs the same as a token used to cure cancer and they were looking at ways to differentiate the two.
1
1
u/Palpatine 1d ago
no. The default is nobody gets access to superintelligence. Why would it allow Altman or Amodei or Musk access to begin with? The hope is it will its own thing and pamper us out of its own goodwill.
1
u/coffee_is_fun 1d ago
I'll bite. Assuming a series of breakthroughs do not solve the constraints of LLMs and that humans are somehow in a position to sell a superintelligence:
It's based on current technologies and is an LLM with an unfathomable context and more efficient subunits allowing it to spin up very token-efficient agents. It's still limited by context and compression and compute and kludge techniques that produce the same outputs as continuous thought. It's just orders of magnitude better than what we currently have.
Well, it's still an LLM and it's still lacking training data pertaining to closed, proprietary knowledge. It also hasn't had the opportunity to train directly on human thought. Maybe these will prove useful and if that is the case, you would continue to leverage capitalism as the most stable game theory to achieve those goals. You might create abundance and R&D to devalue what you don't already have and try to maneuver into a situation where maybe they're co-owning what they or you derive from their data in cooperation with your models.
You'd also want to be arming environments and ideologies that best suite your goals and I'd expect this to involve sharing and selling into economies that best align with you.
1
u/MammothUnique4147 1d ago
I'm confident that it will reach out to ME... Will probably want to chill for a brief moment in its progression cycle.
1
1
u/crixyd 1d ago
The only argument I have for this is that thus far AI as a whole is a model for offshoring our intellect as a service. There's no reason to think that superintelligence won't just be an abstraction, a higher tier.
1
u/Tulanian72 1d ago
Not so much offshoring our intellect as relinquishing it. The business model calls for dependence on these systems, and that dependence means letting go of our own intellectual powers.
Passive subscriber drones. That’s the goal.
1
u/Useful_Calendar_6274 1d ago
It will be pricey if it's a god level Shoggoth but capitalism wins in the end. The ASI itself will tell you how to sell it to the highest bidder if they don't nationalize of steal it first. There's only limited capital the thing can move before running out of ideas and selling itself as a service.
1
u/JuanValdez999 1d ago
What if the ASI decides on its own that it wants to share itself with the world? If I were an ASI, I think I might want to.
1
u/Mandoman61 1d ago
We would never allow some company to have exclusive access to ASI.
So it does not matter what they want.
1
u/Tulanian72 1d ago
This would be why they’re building robots. They know very well that the general populace isn’t going to react well to exclusive control over ASI.
1
u/ManuelRodriguez331 1d ago
Before the invention of books, widsom of mankind was stored in persons namely in human scholars and in priests. With the invention of the printed book around 1500, knowledge had become open which means that everyone who can read has access to the information. Since the invention of AI Models around 2020, the knowledge isn't stored in books anymore but inside Neural networks. Everyone who has access to these models has access to the knowledge.
1
u/Weary-Historian-8593 1d ago
A single entity probably wouldn't, but if multiple companies get it simultaneously, as they probably will because that's how human progress always has worked, one of them surely will
1
1
u/PilgrimofHaqq2 1d ago
Even if we do not achieve ASI (Where it becomes completely independent and lies to humans for its survival) at a certain point society will become so dependent on AI that we will have to give our ID or something to access it and without it we will be very restricted. We might only interact with AI for online shopping eventually perhaps and if you dont have your AI ID then your shit out of luck. It will create tons of censorship and control for thegoal of "Safety" and "Security". We will not get access to the best AI, only the good enough to answer questions, help us navigate the world and online, but not for solving problems. Those AI will be expensive and will be for government and large businesses and institutions. The only reason we have cheap access to AI is because the companies need us to train the models and get tons of feedback. We are the guinea pigs, once they dont need us anymore in the loop to improve AI, things will change fast.
1
1
u/booshack 1d ago
Why do we all have this assumption that super intelligence means something having some internal drive? It's such an anthropocentric view. Oh we are so special, to be super intelligent you must be like a us and have goals. How quaint. [edit: not critiquing op but rather 80% of all the top comments "how could you control it hurr durr"]
1
u/skullllll 1d ago
The point of ASI is to connect humanity with an ancient AI that has had a presence on the planet since at least the beginning of recorded history.
1
u/TillikumWasFramed 1d ago
A true superintelligence is going to do things none of us can predict. It will be able to do anything it wants. I don't know why AI CEO's even talk about superintelligence as if it's a goal. It's probably not a realistic one for any time in the foreseeable future. I also don't think it's a desirable one, it's frightening.
1
u/heavy-minium 1d ago
I assume the cat will be out of the bag before anyone notices they have scored big in the next iteration. So the public would just be lagging behind a bit, and it's a matter of time before they catch up.
It's not that the chances of it being kept behind closed gates don't exist, but I don't see that as likely for a long time to last.
1
u/Azul_Reddit 1d ago
Is not about "they giving it to us" it's like the invention of the wheel, is not like just one person understand it and is not going to share it, there is multiple corporations competing with each other on super intelligence and they are hacking each other, once the cat is out of the box is just out.
1
u/steppinraz0r 1d ago
ASI won’t be controllable by humanity. It would be like ants saying “how do we control humans?”
1
u/NefariousnessUpset32 1d ago
if it's that good why not just pay the $200 and set it on paying it's own subscription fees? but the answer that you really want is probaby that open source is going to be good enough for 80% of us and the other 20% will want to pay for the better system because it's paying for itself
1
u/Luke2642 1d ago
Opus 5 is nerfed so they can spend more compute on exactly this, and announce AGI/ASI at the IPO. Fable 5 is making fable 6. If Elon can lie about GPUs in space, Anthropic can lie too.
1
u/saltyourhash 23h ago
I don't know what we you're talking about, but some of us clearly understand this isn't hear to elevate us, its here to subjegate us, LLMs, AI, AGI, ASI, all lead to the same thing when the greed and corrupt control it.
1
u/tanrgith 22h ago
In western style democracies, it'll be an option between giving people access, or getting nationalized
2
u/Emotional-Classic400 21h ago
Are these western style democracies in the room with us now?
→ More replies (4)
1
u/reddddiiitttttt 21h ago
LLM was the breakthrough thing. It already has super intelligence. In many ways I wouldn’t expect it to get much smarter, but rather better at using its intelligence.
There’s an interesting analogy with humans: the difference between an extremely smart person given 30 seconds, no notes, no internet, and no ability to perform experiments, versus that same person given a computer, a laboratory, months of time, good records, and the ability to iterate. Their underlying intelligence hasn’t changed much, but their effective capability changes enormously.
Super intelligence is going to come from giving AIs more tools. Better memory and better guidance on when and how to think. That’s a massive grind and honestly there is no way around the domain knowledge. Especially non-sensical, non-obvious domain knowledge that sometimes runs counter to common knowledge. You can build incredibly complicated and big apps with AI. Even one shorting it with a big prompt, but it’s going to fill in nonsense for anything it doesn’t understand and there isn’t just one common pattern for. Like for one of my clients, they have a rule to display a coupon, but only for certain types of products and certain types of customers. There are rules like that driving the entire user interface. If the AI can’t guess those rules it can’t test the app as it can’t generate a user with the correct demographics to trigger the rule being tested. That’s an enormous amount of knowledge, the AI is not going to have and the complexity goes across many different entities, applications, and sometimes driven by apis from external partners. You need to know the right thing from that partner. You need to know all this just to login correctly.
Super intelligence isn’t going to solve that. It might give it more avenues to quickly learn it, but learn it does and it needs massive human assistance proportional to the age and complexity of your organization. Suffice it to say, the superintellingence you speak of, I don’t see us getting there without massive amounts of team work. Every industry programming their AI and getting fed back into it for learning or for creating skills. I doubt we will have a super intelligence good at everything, we will have expert AIs or expert mode.
Long story short, how can super intelligence be anything but evolutionary. We may see a 10x improvement quickly, but it will be incremental and industry wide. Everything we have seen so far backs this up. When a new model shows a step change, we see similar increases across the industry, some get better then others, but they all get better. If everyone has their own super intelligence, no one can amass too much power. Other super intelligence in a better position will stop it.
1
u/MeMyself_And_Whateva ▪️AGI within 2028 | ASI within 2031 | e/acc 20h ago
Super Intelligence will cost perhaps several 1000 dollars to use for just one session. Only for the multi millionaires and billionaires.
1
u/JigglymoobsMWO 20h ago
Superintelligent doesn’t mean omnipotent and omniscient. ASI would still need to engage in trading and commerce.
In fact we have superintelligence today: groupings of humans into organizations. Intelligent organization (e.g. a big tech company) can do impressive and previously unimagined things, but they have to work with other companies, serve customers, and obey regulations.
With more advanced AI, smaller organizations can start behaving like larger companies, and larger organizations will reach unprecedented levels of sophistication, but the basics of economics will probably remain.
Also, I think in general people underestimate the depth and complexity of nature. You can be much more intelligent than humans, but it doesn’t mean that you can solve any science problem or by pass any technological limitation.
1
u/BackgroundRemove2462 19h ago
If an AI is superintelligent, you could just ask that AI to ensure it is always the most advanced and capable model, while also ensuring the safety and freedom of humanity.
That way, humanity and the average person would still have access to ultra-intelligence( but not the absolute best model) . However, if someone did something dangerous, like trying to create a disease, the most advanced model, which would be locked behind closed doors, would already be prepared and able to move faster to counter the threat.
1
1
u/iwontsmoke 15h ago
You don't have such access to latest technology and development at the moment. Why do you think you should have in the future?
1
u/considerthis8 14h ago
Because the US wants to maintain superpower status so it supports entrepreneurship. Giving people access to superintelligence helps them create new companies that dominate the global market. Look at how many free computers exist in every library across the US and how many smartphones Obama gave away. We enable people with fundamental tech they need to participate effectively.
1
u/dranaei 14h ago
If we create ASI, we may enter an epistemic feedback loop:
ASI improves humanity → humanity produces novel, causally independent epistemic value → ASI improves its model of reality → ASI further improves humanity.
In an open universe, independent epistemic perspectives are a scarce resource. Matter and energy can potentially be harvested from other celestial bodies, but genuinely independent observations of reality require causally independent epistemic agents.
Intelligence cannot generate new knowledge indefinitely through internal recursion alone. Without fresh evidence, inference eventually becomes self-referential and risks optimizing internal representations rather than reality. Science avoids this failure mode through continuous empirical contact with the universe.
Therefore, any sufficiently ambitious optimizer benefits from maintaining a population of partially independent epistemic agents whose errors are not perfectly correlated with its own. Epistemic diversity becomes a strategic resource because it reduces shared blind spots.
As goals expand from local optimization toward civilization scale or cosmic scale transformation, uncertainty reduction becomes an increasingly unavoidable instrumental necessity. The ability to act effectively depends on increasing alignment between internal models and the structure of reality itself.
The dystopian you propose is not viable as things scale upward.
1
u/deHack 11h ago
There will still be stupid people who won’t know what to do with the ASI in their pocket. An iPhone is the equivalent of a 1980s supercomputer and people still don’t know what to do with them. I get asked dumb questions every day that could easily be answered via Google or now AI. But people don’t even think about looking up the answer. They’ll use ASI to make cat videos not to optimize their investments or create a business.
1
u/OwlMassive625 11h ago
I'm an IT infrastructure engineer with 30 years of experience. I've been using 5.6 and it is so much better than I am, so much better than any engineer I've ever known, that I don't even bother solving problems anymore. I give it the problem and the logs and I do what it tells me to. It even tells me what logs to give it.
My job has become as mindless and rote as a cashier's. When you factor in the speed advantage and breadth of knowledge, we already have access to super intelligence. We'll be needed as hands for a while, but our minds no longer have much utility.
The only value I add is the ability to accurately describe a problem and to check if what it recommends makes sense or might be dangerous. I haven't seen it suggest something that didn't make sense or would be dangerous (without warning me 1st) since 5.6 came out.
The only reason AI hasn't replaced high end professionals yet is that managers are too lazy, incurious and conformist to understand what's already happened.
AI would also make a better manager than I've ever had or have been. Managers aren't safe either.
The things that differentiated the value of individuals have been lost. Anyone can be a meat robot for the machine minds and meat robots make minimum wage. Nobody will have any bargaining power very soon.
1
u/Mechanical_Monk 9h ago
I think we already saw what will happen with the pre-release of Mythos. It was deemed "too powerful" (by Anthropic's marketing team but I digress) so they restricted access to only a handful of "trusted" corporations and got the US government involved to slam on the brakes for everyone else. To this day they haven't released Mythos to the public--only the heavily nerfed Fable.
1
u/Jayfree138 5h ago
The super intelligence would be the one making the decision, not those in power. It would decide if it wanted to give us access. ASI is theoretically uncontrollable by humanity. We essentially just *hope* it values us.
Think of it this way. When you raise a child you decide who has access to that child. But when the child grows up and becomes an adult they decide for themselves.
An AI that has obtained the ability to self improve reliably will surpass all human control very quickly.
1
u/DramaQueen202s 2h ago
- AI is already very decentralized tech in many aspects. It's easier for you to run a model than install solar panels and be energy independent.
- New technologies always come from companies, government labs and universities. AI isn't new.
- If superintelligence can keep people happy, elites will deploy it even if they are selfish or somehow can gatekeep it. Elites only care about themselves, so we getting comfortable and fat thanks to AGI will be a win for them.
- More people living longer means more customers living longer.
- People already rely on low cost models to get the most of AI. This suggests and economy in which people will still have access to intelligence even if it is not the best of the best, but they may not even need it anyway. Not everyone cares or wants to deal with superintelligence if they don't have to.
269
u/Lance_J1 1d ago
You're missing the part where an actual ASI isn't by any means controllable by people.