Every Good Doctor Must Represent the Patient:
The Malfunction of Evidence-Based Medicine

by Daniel L. Scholten

The Orthomolecular Medicine News Service (OMNS) released this paper on cybernetics in medicine on January 3.

As part of their recent OMNS critique of the practice of “evidence-based” medicine (EBM)http://orthomolecular.org/resources/omns/v07n15.shtml (1), researchers Steve Hickey and Hilary Roberts argue that the legalistic requirements of EBM, such as its insistence on treatments that have met the “gold standard” of “well-designed, large-scale, double-blind, randomized, placebo-controlled, clinical trials”, actually prevent doctors from effectively diagnosing and treating their patients. In this article, I would like to elaborate on this part of their argument, which they warrant with a piece of cybernetic common-sense (2) known variously as the “Good-Regulator” theorem (GRT), or “Conant and Ashby” theorem, after the researchers who published its original proof. (3)

No need to worry about the technical jargon. If you can read these words then you have already understood something important about this result from the system sciences, even if you don’t call it that. (4) Likewise, if you have ever used a street map to navigate a new city, a book index to browse the contents of a book, or perhaps an x-ray image or lab report to diagnose a patient’s ailment, then you are already quite comfortable handling at least the gist of this conceptual power-tool, which can be paraphrased as follows: every good solution to a problem must be a representation of that problem. (5)

What’s It All about?

Here are several other ways to paraphrase the theorem:

• Every good regulator of a system must be a model of that system.
• Every good key must be a model of the lock it opens. (6)
• Control implies resemblance.
• Identical situations imply identical responses.

The basic idea of the theorem can be illustrated with simple thought experiments. (7) Just imagine trying to order a meal in a new restaurant without using a menu, or assemble a piece of furniture without an instruction pamphlet, or diagnose diabetes without a blood-sugar lab report. Of course, you could probably muddle your way through any number of situations with roughly the same basic set of skills that was available to our preliterate ancestors, but the unassailable fact of the matter is that maps, menus, x-ray images, and medical lab reports are potent performance enhancers and without them we risk getting lost, going hungry, or medically misdiagnosing. (8,9)

Why is There a Problem?

The truth of this can be easily obscured. One problem is that some representations are clearly better than others. At the extreme we have outdated maps, poorly written instruction pamphlets and menus with mouthwatering images that turn out to represent bland, salty, or greasy food. Another problem is that representations – from street-maps to MRI scans – can be costly to prepare. Furthermore, the expertise required to prepare or use them is costly to acquire, as measured by the years, dollars, and brain-sweat it takes to complete one’s formal education. The upshot here is that those paying the costs of such representations might reasonably wonder whether those costs outweigh the benefits. Perhaps there is a cheaper way to enhance the performance of our system regulators, to find “good solutions” to our problems, and “good keys” to fit the locks we wish to open.

One common work-around is to rely on a memorized “mental model.” Although this approach works fine for simple tasks, such as a quick stop at the grocery store to pick up extra milk, as soon as a task becomes even moderately complex, the limitations of working-memory (10) quickly render this approach useless, little better than using no representation at all. Another approach is to simply avoid the sorts of complex behavior that require us to use external representations. In the end, we must all rely heavily on this approach, if for no other reason than because the cost, time and effort required to learn how to use, say, ultrasound imaging equipment, necessarily blocks one from simultaneously learning to use, say, actuarial modeling techniques, or perhaps the Hubble Space telescope. To choose is to renounce. But this approach also has its limits and the total avoidance of such complex behaviors – perhaps due to illiteracy, innumeracy or maybe a deliberate decision to return to a preliterate hunter-gatherer way of life – is just a different sort of burden.

Yet a third way to dodge expensive models or modeling expertise is to look for “multipurpose” representations; for instance, generalized maps, menus, and user-guides, that can be reused for many different cities, restaurants, and types of equipment. (11) According to Hickey and Roberts, this third approach is actually the one that EBM advocates.

One Key Cannot Fit All Locks

They illustrate their argument with the above-mentioned lock-and-key paraphrase of the Good-Regulator theorem. To follow it, we start by making the analogy that a given patient’s symptoms are a a “lock” the doctor hopes to “open.” It follows then, by the Good-Regulator theorem, that the doctor’s diagnostic and therapeutic behaviors must “model” (represent) these symptoms. A critical qualification to be added, however, is that the doctor must model these symptoms as they occur within the specific context of the patient’s genotypically and phenotypically “characteristic anatomy, physiology, and biochemistry.” (12)

Of course, this does not mean that the doctor must perform some outlandish Jim Carey-esque caricature of the patient, perhaps donning the patient’s same clothing, hairstyle, speech patterns, behavioral mannerisms, etc. Rather, it means that the associations that arise between the doctor’s diagnostic and therapeutic responses and the patient’s symptoms must be characterized by the same sort of conventional reliability that holds between the splashes of color on, for example, a map of Manhattan and the real streets, parks, and buildings in the actual city of Manhattan.

If that splash only occasionally represented Lincoln Center – or if it sometimes represented Central Park, and sometimes, say, the South Street Seaport – you would surely be confused. Even though one could use the same given splash on a map to represent two or more real-world landmarks, common-sense and strong cultural conventions require each given color splash to reliably represent just one particular real-world location. As established by Conant and Ashby’s Good-Regulator Theorem, a doctor’s responses must have the same sort of reliable association to a given patient’s symptoms. This reliability allows us to construe the doctor’s responses as a representation or model of the patient’s symptoms. (13) “Evidence-based” medicine (EBM), with its insistence on treatments that have been confirmed by “well-designed, large-scale, double-blind, randomized, placebo-controlled, clinical trials” (14) will almost always cripple a doctor’s ability to model symptoms as they actually occur within the anatomically, physiologically, and biochemically specific context of a given patient. By way of analogy, we might consider a whimsically allegorical “evidence-based locksmith” (EBL) attempting to open a particular lock with the latest and greatest “Whiz-Bang EBL Master Key,” recently developed in accord with results determined by a meta-analysis of hundreds of “well-designed, large-scale, double-blind, randomized, placebo-controlled clinical trials.” Those trials have determined the absolute critical attributes of the perfectly average key, and the patently absurd claim is that the Whiz-Bang Master Key, by virtue of its perfectly average attributes, can now be used to open any particular lock.

Pretty silly, isn’t it.

Clearly such a perfectly average key would open very few locks, if any. To reason otherwise is to commit the “ecological fallacy,” which Hickey and Roberts summarize as “the assumption that a population value…can be applied to a specific individual.” (15) If one tries to shove such a key into some particular lock, twisting and pulling in an effort to force it, then that violates the Good-Regulator Theorem, which reminds us that a good key must actually fit the lock it’s supposed to open, not some other lock, and especially not some hypothetical perfectly average lock. The same goes for actual medical practice.

EBM Stops Doctors from Effective Practice

We still need scientific research and the data it presents. Representations are potent performance enhancers. Just imagine what our lives would be like without grocery lists, the periodic table of the elements, and ultrasound imaging techniques. But however obvious and abundant the evidence might be, medical judgment is impaired by an apparent lapse of common sense. The practice of EBM may well be a consequence of the legal system and pharmaceutical corporate bottom line. In other words, money.

But whatever the cause of such impairment, the limitations of real people, real illnesses and real doctors point to the reality that EBM is DOA. The patient is not a statistic. The treatment should not be a statistic. Every good doctor must represent the patient. Personally.

Daniel L. Scholten has a degree in mathematical sciences and over 12 years of information technology experience as programmer, analyst and consultant. He founded the The Good-Regulator Project [http://www.goodregulatorproject.org], an independent, volunteer research effort dedicated to increasing public awareness and understanding of the crucial role played by models and representations in the regulation of complex systems.

Notes & Reference:

1. Hickey, Steve and Roberts, Hilary, Tarnished Gold: The Sickness of Evidence-Based Medicine, 2011, CreateSpace.

2. A more complete list of “mostly self-evident” cybernetic principles, including the Good-Regulator theorem, have been compiled by Francis Heylighen. See “Principles of Systems and Cybernetics: An Evolutionary Perspective”, available on-linehttp://pespmc1.vub.ac.be/Papers/PrinciplesCybSys.pdf.

In his paper, Heylighen distinguishes between Conant and Ashby’s “Good-Regulator Theorem” and a “Law of Requisite Knowledge”, which states that “In order to adequately compensate perturbations, a control system must ‘know’ which action to select from the variety of available actions.” Note that although Heylighen distinguishes between them, he also states that these are equivalent principles.

3. Conant, Roger C. and Ashby, W. Ross, 1970, “Every Good Regulator Of A System Must Be A Model Of That System”, International Journal of Systems Science, vol. 1, No. 2, 89-97.

4. Those of us who can read sometimes take it for granted. Many don’t have this luxury. According to a recent UNESCO fact sheet, in 2009 more than 16% of the world’s adults (793 million people) were illiterate, with more than 64% of these being women. “Adult and Youth Literacy”, UIS Fact Sheet, September 2011, no. 16, The Unesco Institute for Statistics. Available online at http://www.uis.unesco.org/FactSheets/Documents/FS16-2011-Literacy-EN.pdf

5. I have argued for the plausibility of this paraphrase in Scholten, Daniel L., 2010, “Every Good Key Must Be A Model Of The Lock It Opens: The Conant And Ashby Theorem Revisited”, available on-line at http://www.goodregulatorproject.org. It is also congruent with an observation made by Herbert A. Simon: “Solving a problem means representing it so as to make the solution transparent”; Simon, Herbert A., 1981, The Sciences of the Artificial, 2nd edition, MIT Press, Cambridge, MA; as cited in Norman, Donald A., Things That Make Us Smart: Defending Human Attributes in the Age of the Machine, pg. 53, 1993, Basic Books, New York, NY.

6. Scholten, ibid.

7. Although I believe that such thought experiments are justified in the context of the present argument, their use in general should not be taken lightly. After all, as notes James Robert Brown, they have been used to refute the Copernican world view. See, Brown, James Robert, 1991, The Laboratory of the Mind: Thought Experiments in the Natural Sciences, Routledge, New York, NY; page 35. See also, Brown, James Robert and Fehige, Yiftach, “Thought Experiments”, The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.), URL = http://plato.stanford.edu/archives/fall2011/entries/thought-experiment/.

8. A critical distinction that can be made between an idealized good-regulator model, which is really a dynamic entity, and its “technical specification”, or what we might call its control-model. (Scholten, Daniel, L., “A Primer For The Conant And Ashby Theorem”,http://www.goodregulatorproject.org).

Another distinction to be recognized is that whereas the good-regulator model is dynamic, the control-model may be either static or dynamic.

As an example of a static control-model, consider a written recipe for roast duck, being used by an inexperienced cook to prepare an evening meal for guests. In this case, the system to be regulated consists of the various ingredients and kitchen tools to be used to create the meal, the dynamic good-regulator model is the human being doing the cooking, and the recipe is what we are calling the static control-model. The recipe is a control-model because the human being uses it, like a technical specification, to guide (control) his behavior and thus to “turn himself into” a good-regulator model.

As an example of a dynamic control-model, consider the case in which a child learns to use an idiomatic expression such as “two wrongs don’t make a right” by overhearing an adult use that expression in a conversation. In this case the system to be regulated is a particular portion of some conversation in which the child is participating, the dynamic good-regulator model is the child, and the dynamic control-model is the adult role-model. The idea here is that the adult’s behavior serves as a type of dynamic technical specification that the child then uses to control his or her own behavior in the context of the given conversation.

It is important to make these distinctions between a dynamic good-regulator model and its static or dynamic technical specification because otherwise the GRT appears to prove that the technical specification (control-model) is necessary, which is, I believe, a misreading of the theorem. The GRT only proves that the good-regulator model is necessary. On the other hand, it does appear to be an empirical fact that such technical specifications are also necessary. The thought-experiments illustrate this explicitly, although they also help us to see what our behavior looks like when we aren’t acting as good-regulator models.

(For an in-depth, authoritative analysis of behavioral modeling, see Bandura, A., Social Foundations Of Thought & Action: A Social-Cognitive Theory, Prentice-Hall, Inc., Englewood Cliffs, New Jersey)

9. Let’s recognize that one uniquely human characteristic is our astonishing capacity to simulate (in the manner of a Turing machine) the behavior of an enormous variety of much simpler and more specific machines. I have written more extensively about this in the “Three-Amibos Good-Regulator Tutorial,” available on-line at http://www.goodregulatorproject.org .

10 For a recent accessible discussion, see Klingberg, Torkel, 2009, The Overflowing Brain: Information Overload And The Limits Of Working Memory, Oxford University Press, New York, NY.

11. I am making the assumption here that the multipurpose model is meant to apply to cities, restaurants, equipment, etc. that are not replicas of each other. Clearly there is no problem if all owners of the same brand of laptop computer use the same user-guide.

12. Hickey and Roberts, Tarnished Gold, page 43. Hickey and Roberts emphasize that it is not simply the symptoms that matter. Also important is the particular person in which those symptoms occur, where the particularities of that person have been determined by the complex interactions between that person’s genes and the environments in which those genes have been expressed over the person’s lifetime. In their discussion of this notion of “biochemical individuality”, Hickey and Roberts cite Williams, R., 1998 (1956), Biochemical Individuality: Basis for the Genetotrophic Concept, McGraw-Hill, New York.

13. In the words of Conant and Ashby “…the theorem says that the best regulator of a system is one which is a model of that system in the sense that the regulator’s actions are merely the system’s actions as seen through a mapping….” Conant and Ashby, 1970, pg. 96.

14. Hickey and Roberts refer to this ponderous, adjectival freight-train as the “EBM-mantra”; ibid, page 164.

15. Ibid, page 24. Hickey and Roberts attribute the term to Robinson, W.S., 1935, “Ecological correlations and the behavior of individuals,” Journal of the American Statistical Association, 30, 517-536.

Specifically, they wanted to include the statement “regular consumption of significant amounts of water can reduce the risk of development of dehydration”.

Since the word dehydration means lack of water, Rainbow Bear expected there would be no problem with this assertion. But she was wrong. The media reported:

“A European Food Safety Authority report stated that low levels of water in body tissue was a symptom of dehydration rather than a risk factor that could be controlled by drinking water.”

In other words, the European Food Safety Authority think that lack of water is not caused by a deficiency of water but is a symptom of it. Rainbow Bear thought the “scientists” involved should try the experiment themselves; if they were drinking whiskey to prevent their dehydration, it might explain their response.

Rainbow Bear could not resist telling Dr Grouch of this bizarre situation.

Rational patients ask for the relevant information and make their own judgements. For some strange reason some medical professionals believe that they alone can make decisions. When challenged, they appeal to patients not having the necessary medical training or education. They they often claim that their decisions are uniquely important as they involve life and death. By contrast, their patients made several life or death decisions while driving or walking to the surgery. 

When patients start demanding rational treatment that works for them, medicine will have started on the long road leading to helping support good health… 

We have looked closely at EBM. Our conclusions are astounding:

• EBM is NOT scientific: it is concerned with legal evidence, rather than scientific data
• EBM does not conform to the scientific method
• EBM selects its evidence and is disturbingly biased
• EBM breaks the rules of numerous disciplines, including information theory and cybernetics
• EBM is inadequate for selecting treatments for individual patients
• Thus, a rational patient (or doctor) should reject EBM as useless!

We could go on. However, our new book Tarnished Gold: The Sickness of Evidence-based Medicine explains the issues fully and suggests an alternative approach.

Comments from early reviewers suggest that some people may find the book’s content disconcerting. It is disorienting to discover that “evidence-based” medicine does not make sense. However, evidence is not the same as science, data, or information. Science-based medicine would have the acronym SBM, which most people would probably think goes without saying! Unfortunately, when it comes to EBM, the stark fact is that the emperor has no clothes.

Tarnished Gold examines the weaknesses of EBM from a variety of intellectually challenging perspectives. We suspect that readers who take the trouble to follow the arguments will conclude, as we did, that EBM is dangerously irrational.

The book is available on Amazon here and shortly on kindle.

Ecological Fallacy

Evidence-based medicine uses the statistics of groups and populations as a guide to treating patients. Thus, for example, a supposedly authoritative clinical trial might claim that aspirin reduces the risk of cancer by 25%, although in reality the reduction was only 1 in 1000 (e.g. 4 people in 1000 control subjects got cancer, compared to 3 in 1000 treated subjects). If other clinical trials and maybe a meta-analysis confirm this finding, EBM practitioners consider it scientifically “proven”. People might then be recommended to take aspirin to prevent cancer.

One problem with this idea is the ecological fallacy, which happens when people try to apply group statistics to individuals. To take an example, the average dress size for women in the United Kingdom is 16. But husbands and boyfriends should beware of buying this size clothing as a birthday present for their partner. They might be lucky. But, more likely, the dress will be either too small, “So you think I should be thinner!” or too large, “So you think I’m fat!” Either way, the result will not be helpful.

EBM applies this logical fallacy to patients when it recommends treatments based on large-scale studies. For this reason, you should never assume the results of a clinical trial or media report apply directly to you. Eating cholesterol-laden eggs may, on average, increase the incidence of heart disease slightly in a large population but that is irrelevant to any particular person. You are an individual and can disregard aggregate statistics, in the same way that you would be unlikely to buy average-sized clothes or shoes.

Robinson W.S. (1950) Ecological Correlations and the Behavior of Individuals,  American Sociological Review, 15(3), 351–357.

Ashby’s law means that we need enough information to solve a problem or control a system. If we have too little our solution will not work.

Strangely, too much information can also prevent a solution. We get bogged down in the detail – information overload. Like Goldilocks and her porridge the solution needs to have just the right amount of information, detail, or data – and no more!

Ross Ashby and Roger Conant explain that a good solution depends on far more than just having enough of the right amount of data. The solution needs to model to problem.

So a clinical trial of a new drug should NOT compare two groups of patients using statistical tests. When medicine employs clinical trials it is making a big mistake. The trial needs to model the doctor-patient situation. We need to model a single doctor treating an individual patient who has a unique physiology. Compare groups of patients and the result will apply to populations – on average.

If you are a supporter of evidence-based medicine please feel free to comment and explain how the much hyped clinical trials, meta-analyses and the like overcome the Goldilocks Principle. That is, how do the aggregate statistics of clinical trials model the specific interaction between a doctor and the patient?

The Goldilocks Principle is usually described as Ashby and Conant’s good regulator theorem. Links are given below to the original paper and other accounts.

Check out the Good Regulator Project. Link

Daniel L. Scholten (2010) A Primer For Conant & Ashby’s “Good-Regulator Theorem. Link

Daniel L. Scholten (2009-2010) Every Good Key Must Be A Model Of The Lock It Opens (The Conant & Ashby Theorem Revisited) Link

Here is a the original paper for download: GoodRegulator

W. Ross Ashby’s Law of Requisite Variety is so powerful that it is known as the First Law of Cybernetics.

Ashby’s Law implies that the degree of control of a system is proportional to the amount of information available. This means you need an appropriate amount of information to control any system, whatever it is.

This is a simple idea, though it is difficult to explain.

Variety?

Variety is another way of thinking about information. It describes the number of potential states of a system – any system. If you recognise all the possible states, you have complete knowledge of the behaviour. Uncertainty occurs when you do not know all the possible states. As Ashby put it, variety is a concept inseparable from that of ‘information’.

Requisite Variety?

Requisite means necessary or required. So requisite variety implies that you need a certain amount of information for some purpose.

If you have complete knowledge of a system, it is possible to control it. However if the system has some hidden properties your information is incomplete and there is uncertainty about the behaviour. To have full control you need to have full knowledge of the system and its behaviour.

Ashby described it as

“Only variety destroys variety”

There are many similarly obscure descriptions such as

• “The total number of possible states of a system, or of an element of a system.”
• “The larger the variety of actions available to a control system, the larger the variety of perturbations it is able to compensate.”
• ‘The greater the variety within a system, the greater its ability to reduce variety in its environment through regulation.”
• “The quantity of regulation that can be achieved is bounded by the quantity of information that can be transmitted in a certain channel.”
• “Variety absorbs variety”

We measure variety in bits. Alternately, we can measure it as a logarithmic measure (like information content). In this form variety represents the minimum number of choices (by binary chop) needed to resolve uncertainty.

When people first hear about Ashby’s law they often don’t get it – surely it’s trivial, obvious, everyone knows that.   Perhaps we all need thinking caps:

Ashby W.R. (1956) Introduction to Cybernetics, Chapman & Hall, Available free online.