Artificial General Intelligence (And Superintelligence) And How To Survive It
Part 1 - Overview
Artificial general intelligence (AGI) poses grave, even existential risks rivaled only by the degree to which it can empower and uplift humanity. An artificial intelligence which can easily outrun us, and which could achieve superintelligence even to the point of exceeding the collective intelligence of the entire human race, could either create a utopia on Earth or kill us all.
AGI also does not have to achieve superintelligence to become a grave threat, particularly as an unanswerable weapon in the wrong hands, or simply an unparalleled disruptive force in far too many hands, including the wrong ones.
How we develop and govern this technology while we still control it will probably determine which outcome we achieve.
We are so far along in the development of AI to resolve any serious arguments about banning it entirely.
We can’t.
But we’re still early enough in the process to exert immense leverage over how AI evolves and who will control it when it does.
This is the golden moment. Yes, eventually multitudes will have access to some form of powerful AI – arguably, they already do to a degree via models like ChatGPT and downloadable open-source models.
But right now the most-powerful systems are still running on server farms full of masses of advanced processors typically designed for AI applications. And these tools, much less anything running independently on your smartphone or laptop, are far from omnipotent.
The adversarial regimes of Russia and China have stated their intent to become dominant in artificial intelligence, and will not stop because we do. Nor will other dangerous actors.
However, advanced democracies have decided advantages in this field.
First, a handful of nations are critical to the production of advanced semiconductor chips – America, Japan, the Netherlands, Taiwan and the Republic of Korea. The first three are already cooperating to block Chinese access to advanced chips and the tools, technology and expertise to make them.
Second, most of the world’s advanced democracies are directly or indirectly allied, either via the larger framework of NATO or bilateral treaties with the United States. Their power to act together can be seen in G7, EU, NATO and other nations’ sanctions against Russia after the further 2022 invasion of Ukraine, as well as through many partnerships such as AUKUS, DIANA, IPEF and the EU-US TTC.
Third, despite propaganda to the contrary, we’re ahead. We aren’t just ahead in terms of advanced large language models in America. The US, UK and NATO have already admitted we’re using artificial intelligence to automate cybersecurity. Given the 8 non-NATO partners included in the NATO Cooperative Cyber Defense Centre of Excellence, we can reasonably assume these AI techniques will become prevalent across the alliance and beyond.
So what do we need to do?
Quite a bit, but the first thing to realize is, as with the AIs now automating our most-advanced cyberdefenses, many critical pieces are already in motion.
First, we need to strive for AI safety. This should include security and safety measures, of course, but also AI alignment – teaching the machine to align with the core values we want it to serve, whatever it may be, instead of a single, narrow, overriding function for which it is willing to sacrifice all else.
Including us.
When possible AI alignment uses a system’s own capacity to understand or at least interpret values to make them foundational to how it executes its tasks and interacts with users and the world.
This also means weeding out actively harmful applications, securing computers against malware and malicious intrusion, and constantly, actively analyzing programs to see where they might be going wrong, intentionally or unintentionally.
AI itself can be leveraged to monitor other AIs, permitting multiple programs operating from different perspectives to parse and assess the data feed coming from other programs, to look for red flags, and to build their own analytical capabilities based on this inflow of data and the results of their reviews and interventions.
Second, we need to license and regulate in service of the above goal of AI safety. Major projects involving large language models and other powerful techniques should come under scrutiny, of course, but everyone with a significant AI should be required to have a basic license for it and regular remote monitoring of its performance.
This may sound extreme, but imagine a world in which malware and hacking were so destructive computer owners used some mythical program, let’s call it a “firewall,” to keep out threats and even used another one, perhaps under the fanciful name “antivirus software” to scan their computers for problems, often reporting issues to, and being updated in turn by, a mysterious “software provider.”
Perhaps such a thing seems unimaginable, but we know from the past it’s quite imaginable. We could use something similar for increasingly autonomous machines.
Such analysis would be focused on the AI itself, and user data and non-AI software would need to be firewalled away from such examination, for both public and private interests. But we do have experience in looking for problems without violating privacy.
Third, we need to build up an ecosystem of benign, human-protective AIs to assist us with perils from cyber, emerging technologies and other AIs. Just one protector would struggle to be everywhere, lacking unlimited reach, authority and processing power. But if we start with one or more cyber-focused AIs in each of the most-advanced democracies, we can easily build from there.
The next step would be cooperation – not only with each other, and new systems coming online in the governments of other allies, but with AIs beginning to protect state, provincial and local governments, large-to-mid-sized corporations and ultimately non-profits, small businesses, individuals, and even lone pieces of equipment with seemingly insignificant defenses and available compute.
Here we would not only be looking for cyber “antivirals,” but “antigens” – stimuli triggering immune responses wherever they emerged in the global system. Essentially, not only finding malicious or disruptive activity, but tagging it, tracking it and analyzing it on both the individual and systemic level.
The point is to turn the Internet, including the Internet of Things – to the extent it persists – into a vast monitoring web to sense disturbances in its threads and to ensnare their sources.
Previously: Emerging Futures - Because the Future Isn’t Coming Fast, It’s Already Here