AGI inequality is an existential threat
written
in the circles i'm in, there's a lot of talk about AGI[1] creating a permanent underclass.
as the idea goes, AGI will make capital far more important than it is today while making almost[2] all human labor no longer economically useful. without radical intervention, AGI will split the world into a caste system determined by how much wealth you had in the pre-AGI era.
but, this isn't the whole story.
even if equalizing measures are taken by governments, AGI will make any corruption far more dangerous, as it acts as a force multiplier for those in power to hijack systems for their own benefit. in a worst case scenario, we could end up with one person controlling the fate of all of humanity.
Power tends to corrupt, and absolute power corrupts absolutely.
once the transition away from human labor is complete, there will be no incentive for those in positions of power to care about the lower classes. this creates the possibility of an autogenocide driven purely by greed.
the "best" case scenario, where everyone is provided with livable UBI by a democratic government, may not be stable. a lack of upwards mobility in society will likely lead to political instability, and an opportunistic actor could take advantage of the power vacuum created in order to enrich themselves.
today, we need to start thinking about how to tackle the societal issues that AGI will create so that we will be prepared to face them tomorrow. failing to do so poses an existential threat to humanity[3].
a call for action
the issue is twofold:
- there's far too little awareness, especially in the general public, of the societal dangers of AGI.
- we lack viable proposals for how we will build a post-AGI world that actually benefits everyone.
we need work towards addressing both.
first, we cannot risk this staying a niche concern for any longer. the risks posed by the post-AGI shift in the balance of power are just as great as the far more publicized risks of misaligned AGI.
second, awareness is useless if it doesn't lead to change. we need to determine a viable path towards creating an equitable post-AGI world, and once we have a plan, we need to act before it's too late.
we still have time to change our fate.
recommended further reading
footnotes
[1] in the context of this piece, AGI (Artificial General Intelligence) refers to a future AI system or set of systems capable of performing all economically valuable human labor.
[2] even in a post-AGI world, there will likely be a small number of humans employed to serve people in the upper classes. i think that the size of this market will likely be small enough to be irrelevant.
[3] a small group of rich elites who screwed over everybody else so that they could conquer the universe does not count as "humanity". i am not willing to debate this.
even if AGI inequality did not pose an existential threat to humanity, it is still worth preventing systems from strangling human ambition. total stasis is a form of death.