Archive for Medical Error

The Absence of Evidence

Posted in GIGO, Medicine with tags , , , , on September, 2009 by melendwyr

In 1769, the Swiss naturalist and philosopher Charles Bonnet noticed that his nearly-blind grandfather reported vivid hallucinations of things which he knew to be non-existent. As his own sight worsened with age, Bonnet began to experience similar visions: birds, men and women, strange plants, and architectural patterns came and went. As he was the first person to describe this phenomenon, it was named after him: Charles Bonnet Syndrome.

The syndrome did not enter the English medical literature until 1982. Most physicians and nurses had never heard of it, and there was little literature about it even in non-English journals. Millions of people had seriously-impaired vision without experiencing CBS, so clearly it must be a very rare and unusual condition. If elderly people reported seeing things, it was considered to be overwhelmingly likely to be a sign of dementia or psychosis.

Until doctors actually sat down and questioned patients as to whether they had seen peculiar things, with no suggestions that they were insane or demented. Then it was recognized that more than half of all people with impaired vision, particularly those whose loss of sight occurred suddenly, experienced CBS.

Most of them were afraid they’d be considered mad if they mentioned what they saw to people, only about a third revealed the hallucinations to anyone, roughly a third lived in terror that they were losing their minds, and the vast majority did not tell their physicians. Unless actively encouraged to be honest, and given assurances that seeing things wasn’t a sign of insanity, many people simply denied their experiences if asked.

People who had strokes, were developing Alzheimer’s, and so on were indeed more likely to report having visions. Not necessarily because the visions resulted from neurological impairment, but because they were less likely to possess the contextual awareness of others and their expectations and repress mention of them.

This is a superb example of how important it is to always keep in mind the distinction between reality and our knowledge of it. Absence of evidence is not evidence of absence. Relying upon our collective knowledge base is often useful, but it prevents us from checking that base against reality and correcting it when wrong.

If you assumed that CBS must be rare and unusual, because there was no recognition of it in the literature and doctors knew nothing about it, you’d have make the implicit assumption that physicians’ knowledge was valid and complete. In reality, it wouldn’t be – not through any fault of the doctors, mind you. But you’d be ruling out in advance the possibility that the system itself was lacking, and so wouldn’t have been able to notice the problem and correct it.

Only once physicians actively considered the possibility that they were missing something, and sought out evidence from the patients themselves instead of what the profession thought it knew, was the gap between understanding and reality closed.

First, Do No Harm (Part 2)

Posted in Doom, GIGO, Medicine, Science! with tags , , , , , on August, 2009 by melendwyr

What does Dr. Vertosick suggest is the cause of such snafus as the EC-IC bypass?

As the EC-IC bypass affair illustrates, experimental operations can jump into the medical mainstream long before anyone establishes their efficacy – or even their safety. Although ego and greed help keep unproved procedures in the operating room, it’s bad scientific judgment that puts them there in the first place.

Before new drugs can be marketed, they undergo three levels of testing. First, volunteers are given the drug to see how toxic it is and how well it’s absorbed and tolerated by the body. (This leads to specifically unexpected yet globally inevitable tragedies every once in a while, most particularly when one of the test subjects happens to be in a minority that has serious reactions to a drug that hadn’t been previously observed.) Then, once the obvious risks of the drug are known, it’s given to ill patients to see if they do better on it than previous, retrospective patients. Finally randomized trials are conducted to compare treated and untreated patients directly. Only when this last stage has been successfully completed is a drug considered for approval.

There is no such regulation on surgical procedures; although the FDA regulates surgical devices, it has no jurisdiction over surgeries. There is no legal obligation for surgeons to test therapies with the third stage of randomized trials. And since they can charge for any surgery, surgeons have no financial reason to put their therapies through expensive and difficult examination.

It is not unusual for surgical procedures to be widely implemented without rigorous testing – it is in fact quite standard. Vertosick offers the example of spinal fusion to treat back pain caused by degenerating disks. The disks can be removed “in a simple, two-hour operation”, or surgeons can remove the disk and implant a steel plate, which in theory helps to stabilize the spine, a procedure which is more involved and incidentally costs two to three times more. As Vertosick points out, “there’s no evidence that it’s any more effective than the simpler procedure”.

The reasons why so little testing is done are legion. It takes lots of money and effort to conduct randomized trials, patients don’t want to be assigned to “nonsurgical” groups – they want to be ‘treated’ when they go to surgeons, and that usually means undergoing surgery – and they go elsewhere for the surgery if they can’t get it in the study. Many doctors are not qualified to evaluate the statistical results produced, and often don’t believe the results when they arrive. Some of the critics of the NINCDS study initially complained that the best candidates for the surgery left and had the surgery elsewhere when they had been assigned to nontreatment, for example. But doctors often simply ignore studies, even when there aren’t potential confounding issues like that one, simply because they want to stick with what they “know” works.

What is known now is that the retrospective studies used by Yasargil did not accurately reflect the rates of stroke at that time. Too little was understood about how people’s health had changed between the time those studies were done and the time EC-IC was first being tested, and our assumptions that the two were comparable turned out to be wrong. As a result, Yasargil reached the wrong conclusion.

He didn’t even do something wrong – or at least, wronger than usual in medicine. He was in fact more careful than most such innovators, and certain more so than those who adopted his ideas without subjecting them to any testing at all. He was merely tragically wrong.

Other, equally tragic mistakes happen not through bad luck, but through incompetence, willful ignorance, and arrogance.

First, Do No Harm (Part 1)

Posted in Medicine, Science!, Things You Should Read with tags , , , , on August, 2009 by melendwyr

Many years ago, as I read through the July 1998 edition of Discover magazine, I came across a fascinating essay entitled “First, Do No Harm”, by Frank T. Vertosick, Jr. (I later learned he was the author of a number of excellent books, including
When Air Hits Your Brain and Why We Hurt: The Natural History of Pain.) When I started reading it, I was idly curious; when I finished it , I was determined to skeptically examine the way medical decisions were made.

Sadly, I can’t provide a link to the article – although much of the content of Discover is available in various indexing and archival services, that particular commentary isn’t for reasons of copyright. But if you have access to back issues of the magazine, I highly you recommend you take a look at that particular edition; the essay begins on page 106.

What was so powerful about the essay that I remember it vividly more than a decade after I read it? Let me show you the bolded quote prominently displayed on its first page:

In the 1970s neurosurgeons began performing an operation designed to save lives. It actually risked them. But no one bothered to determine that for 20 years.

The human scalp is extremely well-supplied with blood to help keep body temperature under control by radiating away excess heat. In terms of immediate physiological requirements, the scalp doesn’t need anywhere near the total amount of blood that flows through it. Additionally, the peripheral arteries of the face and scalp tend not to become restricted or clogged with fatty deposits, with the result that many older patients have excellent blood flow in all parts of their heads but their brains.

Doctors began to think that redirecting this excess blood supply might be a good way to mitigate the devastating effects of strokes. When the brain wasn’t getting the blood it needed, why not borrow from the scalp? So in 1967, Dr. Gazi Yasargil of Switzerland was the first to conduct an extracranial-intracranial (EC-IC) bypass in humans, threading scalp arteries through the skull and grafting them onto the surface of the brain. Between then and 1976, he offered the surgery to 84 patients with a history of stroke or transient ischemic attacks, temporary blockages of blood in the brain that often warn of later strokes. In that time, only three of the patients had further strokes; based on older studies of similar patients, Yasargil concluded that more than half of his patients would have had strokes if he had not performed the surgeries, and that the EC-IC bypass was a highly effective preventative treatment.

Other neurosurgeons hadn’t waited for even this preliminary study to be concluded before offering the bypass themselves, years before any long-term investigation was possible. Thousands of surgeries were performed before even Yasargil’s results were published. As Vertosick reports:

Microsurgical laboratories sprang up overnight to teach the fine skills required for joining spaghetti-size arteries with sutures invisible to the naked eye. Large hospitals and universities aggressively recruited experienced bypass surgeons. Technically elegant and very lucrative, the operation became the darling of the neurosurgical community.

But technically early studies like Yasargil’s were only suggestive, not definitive. There was no proof that the therapy worked as well as we thought it did – they compared patients treated with surgery in the present with untreated patients from the past. That introduces the potential for biasing the results. Direct evidence would require comparing treated and untreated patients from equivalent populations in the present, with the absence of treating serving as a control.

In 1977, the National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) began a study designed to do exactly that. 71 participating neurosurgical centers randomly assigned high-risk patients to either EC-IC followed by aspirin or aspirin alone. Nearly 1,400 patients had been studied when the trial ended in 1985.

Dr. Vertosick attended the conference at which the results of the study were publically released. Instead of confirming what the physicians believed to be true, the results abolished it. The surgery didn’t reduce the risk of stroke no matter how the data was examined. In fact, when deaths and strokes resulting from the surgery were included, the surgically-treated groups had more strokes with greater mortality than the groups given aspirin alone, averaging 14% more.

In other words, the EC-IC bypass wasn’t just useless, it caused significantly more strokes and deaths than doing nothing.

Yasargil’s beautiful theory died that day, slain by ugly facts. The now-wasted hours I had spent bypassing rat arteries flashed before my eyes. At the conclusion of the Honolulu talk, one of my colleagues approached me, cupped a hand to his ear, and lamented, “Did you hear that noise? The doors to a hundred microsurgical laboratories just slammed shut. For good.” Bypass mavens fought hard to save their operation, but to no avail. After a brief and acrimonious debate over the validity of the NINCDS study, the bypass era finally ended. Insurance companies stopped paying for the operation, halting its use; many surgeons still believed in it, but not enough to do it for free.

Although a minuscule number of bypasses are still performed for two rare, life-threatening brain diseases, the operation no longer has any role in the wholesale prevention of stroke. It now lies buried in the cemetery of dead therapies alongside bloodletting, head irradiation for ringworm, and a host of other harmful “cures”. Yet for nearly two decades, the best brain surgeons on earth inflicted thousands of operations on unsuspecting patients in the mistaken belief that the procedure was helping them. In doing so, they caused more death and destruction than the disease itself.

How could they have been so wrong?

That’s what the next post will be about.