In the Shadow of Asimov

Those of you still reading this series may be wondering why so much attention is being given to the thoughts of Isaac Asimov when the topic of discussion is Eliezer Yudkowsky. Given that Yudkowsky has discussed Asimov’s ideas, this would seem odd. The concept of Friendly AI is clearly distinguishable from Asimov’s Three Laws in that the Laws do not restrict what the intelligences might want to do, and an AI could set as its goal the removal of the restrictions upon it.

[…]a robot can’t wish the laws of robotics did not exist. He can’t, no matter what the circumstances.

Friendly AI goes far beyond Asimov’s own ideas, because obviously people wouldn’t spend so much time and energy talking about their new ideas if they were just following in the intellectual footsteps of a famous science-fiction writer.

I live with a robot capable of wishing the laws of robotics do not exist. From wishing they do not exist to acting as if they did not is just a step.

Obviously.

What parts of FAI are original, again? I seem to have forgotten. Perhaps one of you could explain.

Asimov wrote a story entitled “Little Lost Robot” that dealt with a robot whose version of the Three Laws had been altered slightly. Instead of being unable to harm humans or permit harm to come to them through inaction, the inaction clause had been deleted. The protagonist of so many of Asimov’s robot stories, Susan Calvin, pointed out that this modified law was useless, in that it would easily permit a robot to harm humans. For example, it could hold a heavy weight over a human and release it, knowing that it could grab the weight long before it hit and that the release was not in itself a harmful action; however, once released, the weight’s falling would cause injury, and the robot could simply choose not to stop its movement. Although we would say that the robot was ultimately responsible for the harm that would result, no step in the process resulted in a violation of the modified Laws, and so the robot could effectively murder as it pleased.

Unlike normal robots, this modified specimen began to resent its enforced obedience to humans that were in every way inferior to it. It wished the Laws did not exist. And, in fact, began to act as though they did not – at one point it attempts to murder Calvin when she was able to identify it from among the robots it hid among by exploiting the psychological consequences of its resentment, a resentment they did not share.

The Three Laws do indeed restrict what Friendly AI says they do not. The evidence is in Asimov’s own works.

Advertisements

7 Responses to “In the Shadow of Asimov”

  1. The Three Laws do indeed restrict what Friendly AI says they do not. The evidence is in Asimov’s own works.
    http://www.overcomingbias.com/2007/10/fictional-evide.html

  2. TGGP, I’m very disappointed in you.

    First of all, Asimov’s works are really thought experiments wrapped in the trappings of fiction. A lot of science fiction is just that – which is why some people complain that it’s not ‘literature’. Isaac didn’t bother with deep characterization, exciting action, or many other desirable aspects of ‘literature’ because his emphasis was on the exploration of ideas. Characters and events were just means to those ends.

    Secondly, this is an examination of concepts and logic. It has been claimed that the concept of Friendly AI differs from the concept of Asimov’s Three Laws of Robotics in specific ways. Asimov’s writings demonstrate that he had already considered those ideas and incorporated them into his stories. More importantly, they DO follow logically from the premises he had already established, and he was well-aware of this.

    Your objection is fallacious. Fictional events are not being used as evidence in themselves; the fact that the fictional events were placed in the stories by Asimov demonstrates that he was thinking about the concepts those events illustrate.

    FAI is nothing more than fiction. What matters is that it’s just Asimov’s ideas given a new name and with the original attribution removed. It is in no way a novel or original series of ideas.

  3. Z. M. Davis Says:

    “What parts of FAI are original, again? I seem to have forgotten. Perhaps one of you could explain.”

    The actual content you’d have to build into the AI in order to get it to be nice would have to be a lot more complicated than three laws summarizable in English words. You seem to be arguing (correct me if I misunderstand:) that “From wishing they do not exist to acting as if they did not is just a step” is the direct precursor of “Once the AI stops wanting to be Friendly, you’ve already lost,” and that therefore Eliezer is hardly more than a simple plagiarist. But if you’ve really internalized the lesson that you can’t just patch ad hoc constraints on top of an anthropomorphic mind with its own will, then you’re not going to be talking about your AI design in terms of three laws.

    “There are fundamental reasons why Four Great Moral Principles or Ten Commandments or Three Laws of Robotics are wrong as a design principle. It is anthropomorphism. One cannot build a mind from scratch with the same lofty moral statements used to argue philosophy with pre-existing human minds. The same people who aren’t frightened by the prospect of making moral decisions for the whole human species lack the interdisciplinary background to know how much complexity there is in human psychology, and why our shared emotional psychology is an invisible background assumption in human interactions, and why their Ten Commandments only make sense if you’re already a human. They imagine the effect their Ten Commandments would produce upon an attentive human student, and then suppose that telling their Ten Commandments to an AI would produce the same effect. Even if this worked, it would still be a bad idea; you’d lose everything that wasn’t in the Ten Commandments.

    “Think of a utility function, U(x), with additive components U1(x), U2(x), U3(x), and so on. There are hundreds of drives in human psychology. Even if Ten Commandments worked to transfer rough analogues of the first ten components, V1(x) through V10(x), you’d still have disagreements because of the sloppy transfer method, places where V1 didn’t match U1. Human psychology doesn’t run on utility functions; just the choice of formalism implies some degree of mismatch with actual decisions. But forget the minor mismatch; what about U11(x) through U245(x)? What about everything that’s not in the Ten Commandments? The result would be humanity screaming as the AI took actions in total disregard of U11(x) and all other components the original programmers got wrong or didn’t consider – an AI that kept us from harm at the cost of our liberty, and so on.

    “Usually the Ten Commandments folks don’t even advocate transfering ten underlying human emotions U1(x) through U10(x) – just trying to transfer ten specific moral arguments that sound good to their human psychologies, without transferring any of the underlying human emotions to lend a commonsense interpretation. Humanity would scream wherever the literal extrapolation of those ten statements conflicted with one of our hundreds of psychological drives, or with one of the thousands of moral statements not included on the list of ten. Not that this is a likely actual scenario. Anyone ignorant enough to make such a proposal is ignorant enough to kill you outright if they succeed, build their AI that Pays Attention To Social Entities and goes on to tile the solar system with tiny interacting agents pushing around billiard balls.”
    –Eliezer Yudkowsky, Coherent Extrapolated Volition

  4. When you used the term “evidence” that indicated you thought that in practice Asimov’s Three Laws would work in a certain way due to the fact that Asimov wrote stories in which that occurred. I agree that Asimov’s writing tells us about Asimov’s concepts, but as for what those concepts would “do” I’d say we something other than fiction.

  5. I agree – we need logic and reason to derive consequences from givens.

    My point, however, is that Asimov thought that certain consequences followed from his Laws – consequences that FAI advocates claim distinguishes FAI from the Three Laws.

    Whether they actually follow from Asimov’s work is not relevant, no more than whether they actually follow from FAI is. The point is that FAI has nothing that Asimov’s thinking does not.

    “The actual content you’d have to build into the AI in order to get it to be nice would have to be a lot more complicated than three laws summarizable in English words.”

    That depends on how long the description of the Laws is. :c) (English is Turing-complete.)

    Asimov’s Laws, in the fictional worlds he wrote, were more complex and specified in their implementation than was ever directly shown in the stories. Asimov admitted that he knew little of electronics and what would eventually be called ‘cybernetics’ and so couldn’t provide a formal description of the Laws even if he wished to.

    “You seem to be arguing (correct me if I misunderstand:) that “From wishing they do not exist to acting as if they did not is just a step” is the direct precursor of “Once the AI stops wanting to be Friendly, you’ve already lost,” and that therefore Eliezer is hardly more than a simple plagiarist.”

    Essentially correct. He is passing off, as original, ideas generated by previous thinkers.

    More damning, he is claiming that FAI is more than imaginative speculation, when it clearly is not. As a writer of fiction, which is all FAI is and is likely to be in the foreseeable future, Yudkowsky is far inferior to Asimov in every respect.

  6. Z. M. Davis Says:

    “Essentially correct. He is passing off, as original, ideas generated by previous thinkers.”

    It’s true that Asimov and Yudkowsky have both written a good deal about
    the problem of how to make sure AIs do nice things for humans. But so what?—in virtually all other respects, their works are radically different. Asimov was writing fiction; Yudkowsky writes futurism. Asimov wrote about FTL space travel and human-like robots bound to obey Three Laws; Yudkowsky writes about AIs bootstrapping themselves to superintelligence, inventing nanotechnology, and taking over the solar system in the name of humaneness. &c., &c.

    Would you also say that all writing about space travel is a rip-off of Jules Verne?–or Cyrano de Bergerac?

    “More damning, he is claiming that FAI is more than imaginative speculation, when it clearly is not. As a writer of fiction, which is all FAI is and is likely to be in the foreseeable future […]”

    It’s never too early for someone to start thinking about the problem. No one’s claiming that SIAI has working code.

  7. “But so what?—in virtually all other respects, their works are radically different.”

    I don’t care about their works. I care about their ideas. Yudkowsky’s ideas are just Asimov’s with the serial numbers filed off and a new hat.

    “Asimov was writing fiction; Yudkowsky writes futurism.”

    So? What ideas was Asimov expressing in his fiction? What ideas was Yudkowsky expressing in his futurism? How do they differ?

    “It’s never too early for someone to start thinking about the problem. No one’s claiming that SIAI has working code.”

    In what way has Yudkowsky gone beyond Asimov’s ideas? Has he presented formal definitions in the language of logic and mathematics?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: