Archive for November, 2008


Posted in Uncategorized on November, 2008 by melendwyr

Life is games that people play.
They make the rules as they wish.
Who will win? It won’t be me,
but whoever could it be?

The See-Saw of Logic

Posted in GIGO on November, 2008 by melendwyr

[Cartman]: “Kyle, all those times I called you a stupid Jew, I didn’t mean it – you’re not a Jew.”

[Kyle]: “Yes I AM, Cartman! I AM a Jew!”

[Cartman]: “No, no, don’t be so hard on yourself.”

–Exchange between Cartman and Kyle, South Park: Bigger, Longer, and Uncut

It has occurred to me that multi-variable calculus can serve as a potent metaphor for certain kinds of reasoning.

In single-variable calculus, the equations you deal with are simple: one variable is defined in terms of a series of mathematical operations applied to another. When graphed in a Cartesian system, the result is a line embedded in a plane. Direct, elegant, uncomplicated. The techniques for determining the slope of that line at any point are relatively straightforward, so that form of calculus is the one people learn first.

When you’re dealing with more than two variables, when one variable is defined in terms of operations applied to two or more, things get a little more complex. The simplest case involves three variables, and the result is often a surface embedded in a space. The methods you were taught in single-variable calculus can’t be applied to those equations.

One of the things you’re taught to do is take the original equation and rewrite it, treating all but one of the operational variables as constants each time. The standard techniques can then be applied to the modified versions of the equation. Essentially, you’re talking cross-sections through the surface in the Cartesian space along different directions, and determining the slope of the surface in a particular direction.

To measure the slope in terms of any variable, you must treat the others as constants.

There’s a certain kind of logic that operates in much the same way. We have beliefs about the world, and we compare those beliefs to incoming data from the world. We can hold those beliefs constant, or we can be uncertain as to their validity; we can accept the data as accurate and representative, or we can be uncertain as to its accuracy and representativeness. If we let the status of both the beliefs and the data vary, we cannot evaluate either.

The quoted exchange above is an excellent example of this phenomenon cast in a humorous light. For context: Cartman is a vile and bigoted child whose innate charisma barely serves to compensate for his many prejudices. He has an established set of ideas about what properties are associated with the concept of ‘Jewishness’ – various negative and unflattering ones. One of his friends, Kyle, is Jewish. Cartman frequently needles Kyle about being Jewish, much to Kyle’s annoyance, but in truth he recognizes that Kyle has a variety of laudable traits.

So: what are the variables? Kyle’s properties, and the properties of the concept.

Cartman’s idea of what Jews are like is incompatible with his observations of what Kyle is like. He must therefore hold one variable constant and set its value, while letting the other vary and determining its value in relation to the set given. The humor comes from the contrast between our expectations and what actually happens: the cliche is that the bigoted individual learns to reject his prejudices by interacting with a member of the group he prejudges, but Cartman does the opposite and instead denies that his friend is actually a member of the despised group. Instead of adapting his ideas about Jews to fit Kyle’s identification with them, he adapts his ideas about Kyle’s identification and holds fast to his prejudices.

This illustrates a very important point: The manner in which we decide which of the variables to accept as a constant given determines much about what conclusions we derive from them.

If we accept the data, we must let our evaluation of our beliefs teeter, with the data determining which way the outcome leans. If we accept the beliefs, we must let our evaluation of the data teeter, with our beliefs determining which way our thinking tips. One or the other must be held constant. How we decide which will be conserved and which will be altered depends on our evaluation of each.

If we have a well-established way of thinking, one that has served us well and endured many challenges, we are not likely to abolish it due to some contrary evidence. We are far more likely to dismiss the evidence as a fluke or error because it is incompatible with our existing theory. Logically, the conclusion could go either way: rejecting the theory or rejecting the evidence. In practice, it takes a great deal of evidence that we are unable to reject to overturn the theory, and unless a very high standard of proof is met, we reject the evidence every time.

One of the tasks of the rationalist is to always hold back from reaching definitive conclusions, to let the inclusion of this and the exclusion of that be merely provisional. Human beings aren’t good at this – when we note that something is incompatible with our understanding, we tend to reject it completely, deleting it from our awareness. It takes effort and resources to keep track of the possibilities we exclude, and it’s far easier to abolish them. But this tends to prevent us from storing up enough evidence to overturn our theories – each is evaluated individually, instead of collectively, and so a mass of data that might force us to change our minds is rejected one part at a time.

Rationalists must balance atop the see-saw, maintaining equilibrium, identifying and countering inclinations to lean one way or the other, always sensitive to changing conditions and ready to react appropriately.

Further Apologies

Posted in Uncategorized on November, 2008 by melendwyr

I’m afraid things are going to be slow for a while.

Not only does submitting a new post take up to half-an-hour with the connection I have available, but I’m not feeling too well.

Last night I took a closer look at the two-week old bottle of olive oil in my fridge and noticed that the bottom inner surface looks like a petri culture medium.  As I had recently used that oil to prepare my dinner, I am now otherwise occupied with trying to ensure that my internal organs do not become external.

I would wonder what was wrong with that particular bottle that caused the oil to spoil so rapidly, but I have other concerns at the moment.

Why Wall Street but not Detroit?

Posted in Politics and Society on November, 2008 by melendwyr

I for one cannot help but be struck by the contrast between our government’s ‘generous’ distribution of massive amounts of money to various financial institutions, much of which has gone to pay dividends to investors and supply executive compensation instead of what the politicians hoped but did not require the banks to use it for, and the relative stinginess of Washington to lend, much less give, money to the American automakers.

Perhaps part of the problem is that the American groupmind has long accepted American automakers as producing inferior products and services, even while they may have corrected this fault, while the people want to believe American financial institutions are hale and merely going through a rough patch. Detroit has a bad reputation, whether it still deserves it or not, and we desperately want Wall Street to have a good reputation whether it deserves it or not.

Are there other possible explanations?

Steve Sailer has an interesting suggestion about why.

The Cult of Victimhood

Posted in GIGO, Politics and Society on November, 2008 by melendwyr

In our society, the category of ‘victim’ deeply paradoxical.

To be accorded that status is to be held blameless; once it has been granted, no one may suggest that the victim’s own choices might have contributed to the harm they suffered, nor may anyone suggest that the victim may have had the power to prevent by some means harm done to them. Suggesting that the harm may have been rightful or fair is out of the question; victims are those who have been wronged, those treated unjustly. If they weren’t wronged, they wouldn’t be victims, and since they are victims, they must have been wronged.

Once the status is granted, alternatives contrary to that status arising are automatically dismissed. Of course they weren’t responsible – they’re victims. Of course they were helpless – they’re victims. In logic, sufficient contrary evidence overturns an assertion, but mere logic does not apply to social consensus. If someone is defined as being a victim, everything must proceed from that given; questioning that definition is an attempt to undermine the social order and is evil in itself, and probably another attempt to harm the victim.

To be perceived as powerless and harmed, according to the consensus of our society, is to become entitled. Entitled to the sympathy and aid of the powerful. Entitled to the absence of contempt. Entitled to unique treatment; entitled to private, generous rulings that do not apply to others. As a consequence, gaining the status of victim brings immense power, a sort of aristocracy of the least and worst.

This can be seen in the context of Consistency.

A sort of schizophrenic thinking takes hold in the people who want to acquire the status of victim for their favored group. For example, you can see it in a particular strain of feminists. They cannot accept that female genital mutilation is mandated and often carried out primarily by women; nor can they accept that lesser but similarly harmful fashions stem from women’s standards. Women are victims; ergo, they cannot be responsible for systematic oppression. The judgment of other women cannot possibly be involved; conformity to harmful standards cannot possibly be imposed by other women. Therefore, the odious cultural forces responsible for maintaining these atrocity must spring from men.

Not only is this sort of thinking untrue and thus harmful in itself, but it turns the assignment of responsibility into an even greater minefield that it would otherwise be.


Posted in Uncategorized on November, 2008 by melendwyr

If even Naomi Wolf voted for Obama, despite believing that his voting record is unacceptable, because he’s “the lesser evil”, no one ever’s going to refuse to take the least-worst option offered to them.

Coherent Extrapolated Volition

Posted in GIGO with tags , , on November, 2008 by melendwyr

Today we finally get around to discussing the infamous ‘Coherent Extrapolated Volition’ idea, the one that Eliezer insisted I was ignoring.

As of May 2004, my take on Friendliness is that the initial dynamic should implement the coherent extrapolated volition of humankind.

In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

Fascinating stuff. Where is this concept discussed in non-poetic terms? Where is it operationalized? There’s a whole long section in the original paper discussing the idea. Surely there must be some rigorous defining in there somewhere.

Locating it is left as an exercise for the reader, if you enjoy wild goose chases.

Let’s take a closer look at that name. ‘Coherent’ seems straightforward enough – in the context of abstract ideas, it means something along the lines of ‘logically connected; consistent’ or ‘having a natural or due arrangement of parts; harmonious’. Do humans, either individually or collectively, possess a coherent set of preferences? I have no idea how we could determine that. Judging from his writings, neither does Eliezer – but that’s not an obstacle. Eliezer doesn’t even bother to handwave the problem away – he simply ignores it.

‘Extrapolated’ is easy to grasp, as well. It has a clear definition: “to infer an unknown from something that is known”; in mathematics, it’s even clearer: “to estimate a function that is known over a range of values of its independent variable to values outside the known range”. Obviously human morality varies in a predictable way with the independent variable of time. Once, the concept of slavery was widely accepted, then the idea that it was intolerable spread itself, and now it’s abolished and we consider slavery repugnant. Clearly that’s progress because the unacceptable moralities past slowly developed into a form compatible the moralities of today. That form: the moralities of today.

Saith Eliezer:

Humanity’s moral memes have improved greatly over the last few thousand years. How much farther do we have left to go?

They have? I wasn’t aware that moral codes have improved greatly. How, precisely, do we evaluate a moral code other than by appealing to the one we’ve already accepted and internalized? Certainly there are standards of logic and self-consistency that don’t require us to make moral judgments, but my intuition tells me that’s not what Yudkowsky is talking about.

As for “how far we have left to go”, I’m not sure what to make of it. It’s as though he believes there’s some ultimate goal or teleological standard towards which humanity is collectively approaching. Peculiar, since human societies show every sign of adapting themselves to according to the conditions in which they exist, and evolutionary principles are notoriously not goal-driven.

By the standards of today, the standards of today are an improvement over the past. They’d have to be, because moral standards are necessarily applicable to standards in the abstract or general sense, and every system of standards that is self-consistent will view itself as a perfect match to its criteria for the ideal system of standards.

It is undeniable that moral conventions have changed, ‘progressed’ in the weak sense, to become what they are today. It is not clear that this change is orderly or predictable. Nor is it clear that today’s conventions are fundamentally superior to those of the past in any non-trivial sense, much less that they represent objective progress. There is a deplorable tendency of human beings to regard their present beliefs and positions, no matter how arbitrary, as being the end stage of a tremendous process of improvement. Man measures all things, and he tends to hold himself up as the standard of correctness. Unfortunately, even a little thought is sufficient to convince that we are in no sense an end stage, and that considering our prejudices and tendencies to be ultimate and defining truths is nothing more than pernicious narcissism.

You cannot extrapolate a random walk. Can we extrapolate moral change? I know of no proof that such a thing is possible, much less attainable for us. Does Eliezer offer that proof, or even a credible argument in favor of that possibility? Or does he merely take it for granted as an unquestioned assumption?

Where are the questions he claims I am bringing up as my own? Does he ask them? Or does he simply assert the answers he wishes to find as self-evident truths? To ask a question, you must not believe you already know the answer; if you have a belief, it must be set aside for the asking to occur. Does Eliezer set aside his beliefs so that he can ask honestly?

Read Yudkowsky’s writings. Take the time to look up his later writings if you wish, although I am not aware of any fundamental change in their content. You will have to judge for yourself, but I think you can guess the answer I found when I looked for one.

Another Message from Eliezer Yudkowsky

Posted in Uncategorized with tags , , on November, 2008 by melendwyr

—–Inline Attachment Follows—–

Comments which raise issues that I’ve repeatedly dealt with earlier,
as though they were your own amazing original questions, will be

Eliezer Yudkowsky
Research Fellow, Singularity Institute for Artificial Intelligence

AI in a Box

Posted in GIGO with tags , , on November, 2008 by melendwyr

(Correction: the prime number confrontation I refer to below involved Peter de Blanc, not Eliezer Yudkowsky. I was wrong in my attribution. I stand by my examination of Eliezer’s refusal to publish his own transcripts, but de Blanc’s actions cannot be used as inductive evidence for the reasons behind that refusal; thus, those arguments are invalid. The original text of this post remains below; the only alterations have been this correction and a note immediately before the prime transcript is mentioned. Thanks to Nick Tarleton for pointing out the inaccuracy.)

One proposed strategy for dealing with the possibility of a runaway AI is isolating its hardware so that it has a limited ability to access external resources, from new hardware to data. Thus, even if we don’t understand the software well, we can be confident that it will be contained. The potential weak point in this strategy is that humans retain the capacity to release the AI from its confinement, and humans are notoriously falliable.

Some time ago, Eliezer Yudkowsky announced that he had repeatedly run an experiment, with a volunteer pretending to be a human AI monitor and himself pretending to be a boxed AI which would try to persuade the human to ‘let it out’. The base rules of the experiment can easily be found at Overcoming Bias. What was surprising was that he reported the release of the AI in two cases, even with the controllers knowing that their only task was to refrain from letting it go.

How did he accomplish this extraordinary result? He refuses to reveal his method, saying that telling others how he did it might lead to people failing to take the hypothesized scenario seriously or denying that the method would work in a real case. No transcripts of the interactions have been released.

Of course! When I want people to take unusual experimental results seriously, concealing my methodology and thereby violating the rationalist standards of the scientific method just leaps to mind.

People can always deny that a ‘game’ experiment’s results offer insight into the way people behave in the ‘real world’. Keeping the method secret doesn’t make that any less likely. What it does do is make it more likely that people will reject claims based on those experimental results, suspecting that they’re somehow being scammed and that the claims aren’t really what they seem to be. It makes it easier for people to not take the whole thing seriously.

Consider also that the hypothetical method represents a vulnerability to ‘phreaking’ in potential safety systems for AI control. One of the key differences between ‘hackers’ and ‘phreakers’ is that while both can find security flaws in the systems they interact with, phreakers attempt to exploit them and cause damage, while hackers consider themselves beholden to higher ethical standards and often inform those who control the system of how they got past their security and what was accessed. If people can really be persuaded to let an AI out of the box, it is vitally important for us to know this and how the persuasion works. Keeping it secret seems quite irresponsible.

So how did Yudkowsky accomplish his persuasion? We don’t really know, but we can reasonably speculate.

(Edit: the information following this edit is wrong – Peter le Blanc was in the prime transcript, not Eliezer Yudkowsky. The arguments involving Yudkowsky’s involvement are therefore founded on an incorrect premise and are so invalidated.)

Previously, Yudkowsky has posted a transcript of a similar case: while chatting with an acquaintance, Eliezer made a ten-dollar bet regarding the possibility that he could be induced to accept an incorrect statement about prime numbers. The acquaintance repeatedly said that he was tired and attempted to steer the conversation in other directions, but Eliezer persisted and the acquaintance gave in. Eliezer then asked repeatedly whether the other accepted various prime numbers as prime – which he acceded to. Then Eliezer asked the same about a relatively large, non-prime number – and the other chatterer answered in the affirmative.

He quickly realized the error, of course. Yudkowsky told him to donate ten dollars to the Singularity Institute, the organization that employs Eliezer.

I strongly suspect there is nothing more to the AI experiment than browbeating and “wearing down” the controller. Given how successful social cons have been for Yudkowsky in the past, why would he try any other method?

Dubious Accomplishments

Posted in Uncategorized with tags , on November, 2008 by melendwyr

You may be tired of reading this series about Eliezer Yudkowsky. I know I’m tired of writing it. So let’s try to move towards conclusion.

Computer programmers know that the most important of a programming task is thoroughly understanding the problem and working through the solution. Generally, most of the work of constructing a program occurs before any code is ever written.

So the fact that Yudkowsky has no code examples or testable constructions doesn’t count against him. That’s the final stage, and it would be unreasonable to expect anyone to have reached that level of understanding at this point in AI construction.

But what has he actually accomplished? We’ve previously seen that his hypothesis of ‘Friendly AI’ covers the same ground as Isaac Asimov’s Three Laws of Robotics. The thought experiments Asimov embedded in his fiction anticipated Eliezer’s own thoughts, and Asimov wrote them long before Eliezer was born. What new territory has Eliezer covered?

In my examinations of Yudkowsky’s writings, I can find only two ideas that would seem to qualify. One is his ‘Hostile AI in a Box’ experiment, and the other is the ‘Coherent Extrapolated Volition’ hypothesis.

Hostile AI in a Box is next.