CLASS BLENDING: Simpson's Paradox

For the past two days, we've been posting on Class Blending. Simpson's paradox is a special case that demonstrates what may happen when classes of information are blended.

---

Simpson's paradox is a well-known problem for statisticians. The paradox is based on the observation that findings that apply to each of two data sets may be reversed when the two data sets are combined.

One of the most famous examples of Simpson's paradox was demonstrated in the 1973 Berkeley gender bias study (1). A preliminary review of admissions data indicated that women had a lower admissions rate than men:

Men    Number of applicants.. 8,442   Percent applicants admitted.. 44%
Women  Number of applicants.. 4,321   Percent applicants admitted.. 35%

A nearly 10% difference is highly significant, but what does it mean? Was the admissions office guilty of gender bias?

A closer look at admissions department-by-department showed a very different story. Women were being admitted at higher rates than men, in almost every department. The department-by-department data seemed incompatible with the combined data.

The explanation was simple. Women tended to apply to the most popular and oversubscribed departments, such as English and History, that had a high rate of admission denials. Men tended to apply to departments that the women of 1973 avoided, such as mathematics, engineering and physics. Men tended not to apply to the high occupancy departments that women preferred. Though women had an equal footing with men in departmental admissions, the high rate of women rejections in the large, high-rejection departments, accounted for an overall lower acceptance rate for women at Berkeley.

Simpson's paradox demonstrates that data is not additive. It also shows us that data is not transitive; you cannot make inferences based on subset comparisons. For example in randomized drug trials, you cannot assume that if drug A tests better than drug B, and drug B tests better than drug C, then drug A will test better than drug C (2). When drugs are tested, even in well-designed trials, the test populations are drawn from a general population specific for the trial. When you compare results from different trials, you can never be sure whether the different sets of subjects are comparable. Each set may contain individuals whose responses to a third drug are unpredictable. Transitive inferences (i.e., if A is better than B, and B is better than C, then A is better than C), are unreliable.

- Jules Berman (copyrighted material)

key words: data science, irreproducible results, complexity, classification, ontology, ontologies, classifications, data simplification, jules j berman

Reference:

1. Bickel PJ, Hammel EA, O'Connell JW. Sex Bias in Graduate Admissions: Data from Berkeley. Science 187:398-404, 1975.

2. Baker SG, Kramer BS. The transitive fallacy for randomized trials: If A bests B and B bests C in separate trials, is A better than C? BMC Medical Research Methodology 2:13, 2002

Expunging a Blended Class: The Fall of Kingdom Protozoa

In yesterday's blog, we introduced and defined the term "Class blending". Today's blog extends this discussion by describing the most significant and most enduring class blending error to impact the natural sciences: the artifactual blending of all single cell organisms into the blended class, Protozoa.

For well over a century, biologists had a very simple way of organizing the eukaryotes (i.e., the organisms that were not bacteria, whose cells contained a nucleus) (1). Basically, the one-celled organisms were all lumped into one biological class, the protozoans (also called protists). With the exception of animals and plants, and some of the fungi (e.g., mushrooms), life on earth is unicellular. The idea of lumping every type of unicellular organism into one class, having shared properties, shared ancestry, and shared descendants, made no sense. What's more, the leading taxonomists of the nineteenth century, such as Ernst Haeckel (1834 - 1919), understood the class Protozoa was at best, a temporary grab-bag holding unrelated organisms that would eventually be split into their own classes. Well, a century passed, and complacent taxonomists preserved the Protozoan class. In the 1950s, Robert Whittaker elevated Class Protozoa as a kingdom in his broad new "Five Kingdom" classification of living organisms (2). This classification (more accurately, misclassification) persisted through the last five decades of the twentieth century.

Modern classifications, based on genetics, metabolic pathways, shared morphologic features, and evolutionary lineage, have dispensed with Class Protozoa, assigning each individual class of eukaryotes to its own hierarchical position. A simple schema demonstrates the modern classification of eukaryotes (3). Many modern taxonomists are busy improving this fluid list (vida infra), but, most significantly, Class Protozoa is nowhere to be found.

Eukaryota (organisms that have nucleated cells)
  Bikonta (2-flagella)
    Excavata
      Metamonada
      Discoba
        Euglenozoa
        Percolozoa
    Archaeplastida, from which Kingdom Plantae derives
    Chromalveolata
      Alveolata
        Apicomplexa
        Ciliophora
      Heterokontophyta
  Unikonta
    Amoebozoa
    Opisthokonta
      Choanozoa
      Animalia
      Fungi

Why is it important to expunge Class Protozoa from modern classifications of living organisms? Every class of living organism contains members that are pathogenic to other classes of organisms. To the point, most classes of organisms contain members that are pathogenic to humans, or to the organisms that humans depend on for their existence (e.g., other animals, food plants, beneficial organisms). There are way too many species of pathogens for us to develop specific drugs and techniques to control the growth of each disease-causing organism. Our only hope is to develop general treatments for classes of organisms, that share the same properties; hence the same weaknesses. For example, in theory, it's much easier to develop drugs that work on Apicomplexans that it is to develop separate drugs that work on each pathogenic species of Apicomplexan (3).

By lumping every single-celled organisms into one blended class, we have missed the opportunity to develop true class-based remedies for the most elusive disease-causing organisms on our planet. The past two decades have seen enormous progress in reclassifying the former protozoans. Unfortunately, the errors of the past are repeated in textbooks and dictionaries.

Here are three definitions of protozoa that I found on the web. Notice that these definitions don't even agree with one another. Notice that the first definition includes single celled organisms that may be free-living or parasitic. The second definition indicates that protozoans are obligate intracellular organisms. The third definition indicates that some protozoans are pathogenic in animals but omits mention of pathogenicity for other types of organisms. None of the definitions tell us that modern taxonomists have abandoned "protozoa" as a bona fide class of organisms.

from: http://www.dictionary.com/browse/protozoan
Protozoan: Any of a large group of one-celled organisms (called protists) that live in water or as parasites. Many protozoans move about by means of appendages known as cilia or flagella. Protozoans include the amoebas, flagellates, foraminiferans, and ciliates.

from: www.medicinenet.com/script/main/art.asp?articlekey=5091
Protozoa: A parasitic single-celled organism that can divide only within a host organism. For example, malaria is caused by the protozoa Plasmodium.

from: http://www.merriam-webster.com/dictionary/protozoan
Protozoan: any of a phylum or subkingdom (Protozoa) of chiefly motile and heterotrophic unicellular protists (as amoebas, trypanosomes, sporozoans, and paramecia) that are represented in almost every kind of habitat and include some pathogenic parasites of humans and domestic animals.

References:

[1] Scamardella JM. Not plants or animals: a brief history of the origin of Kingdoms Protozoa, Protista and Protoctista. Internatl Microbiol 2:207-216, 1999.

[2] Hagen JB. Five kingdoms, more or less: Robert Whittaker and the broad classification of organisms. BioScience 62:67-74, 2012.

[3] Berman JJ. Taxonomic Guide to Infectious Diseases: Understanding the Biologic Classes of Pathogenic Organisms. Academic Press, Waltham, 2012.


- Jules Berman (copyrighted material)

key words: data science, irreproducible results, complexity, classification, ontology, ontologies, protozoa, Apicomplexa, protists, protoctista,jules j berman

Intro to Class Blending

I thought I'd devote the next few blogs to a concept that has gotten much less attention than it deserves: blended classes. Class blending lurks behind much of the irreproducibility in "Big Science" research, including clinical trials. It also is responsible for impeding progress in various disciplines of science, particularly the natural sciences, where classification is of utmost importance. We'll see that the scientific literature is rife with research of dubious quality, based on poorly designed classifications and blended classes.

For today, let's start with a definition and one example. We'll discuss many more specific examples in future blogs.

Blended class - Also known as class noise, subsumes the more familiar, but less precise term, "Labeling error." Blended class refers to inaccuracies (e.g., misleading results) introduced in the analysis of data due to errors in class assignments (i.e., assigning a data object to class A when the object should have been assigned to class B). If you are testing the effectiveness of an antibiotic on a class of people with bacterial pneumonia, the accuracy of your results will be forfeit when your study population includes subjects with viral pneumonia, or smoking-related lung damage. Errors induced by blending classes are often overlooked by data analysts who incorrectly assume that the experiment was designed to ensure that each data group is composed of a uniform and representative population. A common source of class blending occurs when the classification upon which the experiment is designed is itself blended. For example, imagine that you are a cancer researcher and you want to perform a study of patients with malignant fibrous histiocytomas (MFH), comparing the clinical course of these patients with the clinical course of patients who have other types of tumors. Let's imagine that the class of tumors known as MFH does not actually exist; that it is a grab-bag term erroneously assigned to a variety of other tumors that happened to look similar to one another. This being the case, it would be impossible to produce any valid results based on a study of patients diagnosed as MFH. The results would be a biased and irreproducible cacaphony of data collected across different, and undetermined, species of tumors. This specific example, of the blended MFH class of tumors, is selected from the real-life annals of tumor biology (1), (2).

References:

[1] Al-Agha OM, Igbokwe AA. Malignant fibrous histiocytoma: between the past and the present. Arch Pathol Lab Med 132:1030-1035, 2008.

[2] Nakayama R, Nemoto T, Takahashi H, Ohta T, Kawai A, Seki K, et al. Gene expression analysis of soft tissue sarcomas: characterization and reclassification of malignant fibrous histiocytoma. Modern Pathology 20:749-759, 2007.


- Jules Berman (copyrighted material)

key words: data science, irreproducible results, complexity, classification, ontology, ontologies, jules j berman