Book Review: Human-Compatible – Artificial Intelligence and the Problem of Control (Viking 2019) by Stuart Russell
AI has severe limitations. Still we have reasons to worry – both because of these limitations and because they could be overcome in the future. In his new book “Human-Compatible: Artificial Intelligence and the Problem of Control” explains the principles that govern the action of autonomous AI-systems and makes proposals for how such systems should be designed to make them beneficial rather than evil.
This is a book about which principles are needed in order to create beneficial Artificial Intelligence-systems. It’s original, and it’s important.
To start with: Stuart Russell is professor of Neurological Surgery at the University of California, San Francisco and Professor of Computer Science at Berkeley. He is vice chair of the World Economic Forum’s Council on AI and Robotics. He is a fellow of the American Association for Artificial Intelligence. And so on. Reputation isn’t something that one gets for nothing. More than any arguments that the author presents, his outstanding position in those fields of science that are relevant for AI is a strong reason to listen to him.
First of all, Russell provides us with a clear estimation of where we stand. For the near future, there still will be major tasks which AI is far away from being able to tackle. The success of AI in winning over human champions in board games such as chess or Go, Russell explains, should not seduce us to think that AI has magic powers in other fields, too. The reason for this is that AI works, to a great extent, with methods of machine learning, that is autonomous learning. With games such as Go, the approach works surprisingly well, because the game is regulated by strict rules. The real world is much less convenient. One reason for this that our daily life consists of thousands little tasks which we accomplish rather effortlessly, but which are very difficult to program or to learn for an AI.
One difficulty is that very often actions and tasks that we perform intuitively are not very easy to discern and to define from an abstract point of view. “What we want is for the robot to discover for itself that [e.g.] standing up is a thing – a useful abstract action”, Russell explains. “I believe this capability is the most important step need to reach human-level AI.” So far this has not been invented.
AI cannot find by itself ways to proceed from general rules to concrete actions, if there are no human-defined rules for this. Thus, AI basically lacks the capability to plan and perform actions.
This is a major point. AI, Russell explain, cannot perform abstract reasoning. AI-machines such as IBM’s Watson, he explains, can extract simple information from clearly stated facts – “but cannot build complex knowledge structures from text; nor can they answer questions that require extensive chains of reasoning with information from multiple sources.” Or take Alpha Go – Google Deep Mind’s AI-system for playing the board game Go. “Alpha Go has no abstract plan. Trying to apply AlphaGo in the real world is like trying to write a novel by wondering whether the first letter should be an A, B, C, and so on.” This is a broad limitation. AI cannot find by itself ways to proceed from general rules to concrete actions, if there are no human-defined rules for this. Thus, AI basically lacks the capability to plan and perform actions. “At present”, Russell writes, “all existing methods for hierarchical planning rely on a human-generated hierarchy of abstract and concrete actions.” Computers that learn these hierarchies by themselves have not been invented so far. The reason: Human scientists “do not yet understand how such hierarchies can be learned [by an AI] from experience.”
From a cognitive point of view, the function of goals is that they have a focusing effect on one’s thinking. AI-machines do not have goals.
Besides abstract thinking, machine often lack something which, in cognitive science, is called smart heuristics. Smart heuristic stands for the many abbreviation and tricks that human perform to solve tasks and problems – without employing too much calculating power. It’s not just tricks, but also embedded in practical concerns. One example are goals that we are striving for. From a cognitive point of view, the function of goals is that they have a focusing effect on one’s thinking. AI-machines do not have goals. Current game-playing AI-Systems “typically consider all possible legal actions”. This is where they are superior to human players, who cannot foresee such a variety of different paths. But here lies AI’s weakness, too. Because AI cannot limit its scope, even a super-equipped AI will be overwhelmed with the variety of different paths of action in real life. Human have acquired techniques to reduce that kind of complexity. AI hasn’t – a least not in a way that we deem trustworthy.
These are severe limitations. Still we have reasons to worry – both because of these limitations and because they could be overcome in the future. Russell actually thinks that human-level AI is not impossible in principle. To the contrary: Super-intelligent machines, he warns, could actually take control of humanity. A whole chapter is devoted to this issue.
One not so-technical aside of interest to Netopia-readers refers to the inherent drive or rationality of AI-systems. It’s about maximizing clicks – getting users to visit a website in order to generate traffic. How would an intelligent system maximize click-rates? One solution: simply to present items that the user likes to click. “Wrong”, says Russell. The solution which an intelligent system would choose “is to change the user’s preferences so that they become more predictable…. Like any rational entity, the algorithm learns how to modify the state of its own environment – in this case, the user’s mind – in order to maximize its own reward.” This is thrilling – and a good example how AI can pose threats even before becoming super-intelligent.
The solution to the threat of super-intelligence and, at the same time, evil or questionable AI-systems is the same as the solution
The solution to the threat of super-intelligence and, at the same time, evil or questionable AI-systems is the same as the solution Russell sketches for the problem of coping with the limitations of current AI: “Machines (…) need to learn more about what we really want”, Russell point out, and this learning should happen “from observations of the choices we make and how we make them.”
There are two lines of reasoning underlying this proposal. The first is: If human-level AI is something that we should expect to happen, then this super-intelligence should preferably be benevolent. Being benevolent, though, is something that can neither be programmed, nor can it be learned by super-intelligent machines themselves. Even if AI could acquire the capability of abstract reasoning, they could not pursue the goal of being benevolent. The reason for this genuine philosophical: “Benevolent” cannot be defined in any unambiguous way, because there are just too many competing and not compatible values around. (To build this argument, large sections of the book are devoted to philosophical endeavors to rationally construct human preferences and perceptions of utility, both on an individual level and on a group level.)
The second line of thinking refers to the above-mentioned problems that today’s AI has with abstract thinking. It is this part of the book which is definitely of practical interest to people designing AI systems today – both on the level of software as on the level of human-machine-interaction.
A better solution, Russell thinks, would be for the AI to ask the user routinely and automatically questions
One example is the gorilla problem. Some years ago, a user of the Google Photos image-labeling service complained that the software had labelled him and his friend as gorillas. The interesting point of this incidence is that it makes clear the value-proposition build into the software. Obviously, the image-labeling service assumed that the cost of misclassifying a person as a gorilla was roughly the same as the cost of, e.g., misclassifying a Norfolk terrier as a Norwich terrier. As reaction to the incident, Google manually changed the algorithm – with the result that later, in many instances, the software just refused to do labeling in cases that were unclear. A better solution, Russell thinks, would be for the AI to ask the user routinely and automatically questions such “Which is worse, misclassifying a dog as a coat or misclassifying a person as an animal?”. Answers to questions of this kind could help to tune the labeling-service according its users’ needs.
This is what, in the end, it all boils down to. Where possible, machines need to learn about what human users really want from observation. If observation is not possible, asking is a suitable approach. Human-level AI is not about more or better computation. It’s all about the design of human-machine-interaction in order to feed human values and preferences into the system.
