The idea of artificial intelligence overthrowing humankind has been talked about for many decades, and scientists have just delivered their verdict on whether we’d be able to control a high-level computer super-intelligence. The answer? Almost definitely not.
The catch is that controlling a super-intelligence far beyond human comprehension would require a simulation of that super-intelligence which we can analyse. But if we’re unable to comprehend it, it’s impossible to create such a simulation.
Rules such as ’cause no harm to humans’ can’t be set if we don’t understand the kind of scenarios that an AI is going to come up with, suggest the authors of the new paper. Once a computer system is working on a level above the scope of our programmers, we can no longer set limits.
If we are going to push ahead with artificial intelligence, we might not even know when a super-intelligence beyond our control arrives, such is its incomprehensibility. That means we need to start asking some serious questions about the directions we’re going in.
“A super-intelligent machine that controls the world sounds like science fiction,” says computer scientist Manuel Cebrian, from the Max-Planck Institute for Human Development. “But there are already machines that perform certain important tasks independently without programmers fully understanding how they learned it.”
