Precision Medicine meets the Data Complexity Barrier

A funny thing happened on the way to "Precision Medicine". It seems that many of the fundamental studies in the field of pharmacogenomics and personalized medicine are yielding irreproducible results. We find that we can not depend on the data that we depend on. If you don't believe me, consider these shocking headlines:

1. "Unreliable research: Trouble at the lab." (1) The Economist, in 2013 ran an article examining flawed biomedical research. The magazine article referred to an NIH official who indicated that "researchers would find it hard to reproduce at least three-quarters of all published biomedical findings." The article also described a study conducted at the pharmaceutical company Amgen, wherein 53 landmark studies were repeated. The Amgen scientists were successful at reproducing the results of only 6 of the 53 studies. Another group, at Bayer HealthCare, repeated 63 studies. The Bayer group succeeded in reproducing the results of only one-fourth of the original studies.

2. "A decade of reversal: an analysis of 146 contradicted medical practices." (2) The authors reviewed 363 journal articles, reexamining established standards of medical care. Among these articles were 146 manuscripts (40.2%) claiming that an existing standard of care had no clinical value.

3."Cancer fight: unclear tests for new drug." (3). This New York Times article examined whether a common test performed on breast cancer tissue (Her2) was repeatable. It was shown that for patients who tested positive for Her2, a repeat test indicated that 20% of the original positive assays were actually negative (i.e., falsely positive on the initial test). (3).

4. "Why most published research findings are false." (4). Modern scientists often search for small effect sizes, using a wide range of available analytic techniques, and a flexible interpretation of outcome results. Under such conditions, the manuscript author found that research conclusions are more likely to be false than true (4).

5. "Reproducibility crisis: Blame it on the antibodies" (5). Biomarker developers are finding that they cannot rely on different batches of a reagent to react in a consistent manner, from test to test. Hence, laboratory analytic methods, developed using a controlled set of reagents, may not have any diagnostic value when applied by other laboratories, using different sets of the same analytes (5).

Anyone who tries to stay current in biomedical research understands that much of the published literature is irreproducible (6); and that almost anything published today might be retracted tomorrow. This appalling truth applies to some of the most respected laboratories in the world (7), (8), (9), (10), (11), (12), (13). Those of us who have been involved in assessing the rate of progress in disease research are painfully aware of the numerous reports indicating a general slowdown in medical progress (14), (15), (16), (17), (18), (19), (20), (21).

For the optimists, it is tempting to assume that any problems that we may be experiencing today are par for the course, and temporary. It is the nature of science to stall for a while and lurch forwards in fits. Errors and retractions will always be with us so long as humans are involved in the scientific process.

For the pessimists, such as myself, there seems to be something going on that is really new and different; a game changer. This game changer is the "complexity barrier", a term credited to Boris Beizer, who used it to describe the impossibility of managing increasingly complex software products (22). The "complexity barrier" applies equally well to biomedical research. Recent studies have shown that inherited behavior is not fully determined by the genetic sequence of DNA. There are many different elements that modify and control genetic expression, and these elements do not have simple functionality (23). When complex cells are perturbed from their normal, steady-state activities, the rules that define cellular behavior become complex, and impossible to predict (24).

Modern biomedical data is high-volume (e.g., gigabytes and larger), heterogeneous (i.e., derived from diverse sources), private (i.e., measured on human subjects), and multi-dimensional (e.g., containing thousands of different measurements for each data record). The complexities of handling such data are daunting. When we closely examine the kinds of systemic flaws that cropping up in the recent biomedical literature, complexity always seems to play a supporting role. Here are a few examples:

1. Errors in sample selection, labeling, and measurement (25), (26), (27)

3. Misinterpretation of the data (28), (4), (29), (19), (30), (31), (32)

4. Data hiding and data obfuscation (33), (34)

5. Unverified and unvalidated data (35), (36), (37), (38), (30), (39)

6. Outright fraud (34), (40)

As biomedical data becomes increasingly complex, the complexity barrier will become impenetrable. It is ironic that just as we are coming to accept the limits of analyzing complex data, we are given President Obama's gift of a new funding initiative in support of precision medicine (41). How can we use this money effectively when we know that human diseases reside on the far side of the complexity barrier?

As it happens, not all diseases are genetically complex. The rare genetic diseases of humans, with very few exceptions, involve a single mutation in a single gene. In the past decade, we have made remarkable advances in understanding the rare diseases. For example, FDA approved a total of 44 drugs in 2014 (42). Of those 44 drugs, 21 (47%) were approved for the treatment of rare diseases, including: non-24-hour sleep-wake disorder, Morquio A syndrome, neurogenic orthostatic hypotension, generalized lipodystrophy, psoriatic arthritis, hemophilia B, multicentric Castleman's disease, hemophilia A (two drugs), Non-Hodgkin lymphoma, hereditary angioedema, leukemia (two drugs), multiple sclerosis, Gaucher disease, idiopathic pulmonary fibrosis (two drugs), gastric cancer, melanoma (two drugs), and ovarian cancer. In fact, most of the medical advances in the past two decades have occurred in the rare diseases; not the common diseases (43).

The rare diseases, being genetically simple, can be successfully treated with drugs targeted to a specific pathway. As it happens all of the cancers that we can cure in an advanced stage of growth (i.e., with metastases) are rare cancers (43), (44): choriocarcinoma, acute lymphocytic leukemia of childhood, Burkitt lymphoma, Hodgkin lymphoma, acute promyelocytic leukemia, large follicular center cell (diffuse histiocytic)lymphoma, embryonal carcinoma of testis, hairy cell leukemia, and seminoma.

Too often, the rare diseases are casually dismissed as outliers, not representative of metabolic pathways that drive the common diseases. Bad decision. Historically, the most important advances in common diseases have come from studying the rare diseases (43). Pathways that are studied, understood, and treated in the rare cancers will likely apply to common (i.e., genetically complex) cancers that share some of the same pathogenic pathways. Hence, the rare diseases are not the exceptions to the general rules that apply to common diseases; the rare diseases are the exceptions upon which the general rules of common diseases are based (43).

In this new, highly-funded era of precision medicine research, funding should flow to the rare diseases. If the common diseases are the genetic puzzles that modern medical researchers are mandated to solve, then the rare diseases are the pieces of the puzzles (43).

- © 2015 Jules J. Berman

References

[1] Unreliable research: Trouble at the lab. The Economist October 19, 2013.

[2] Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc 88:790-8, 2013.

[3] Kolata G. Cancer fight: unclear tests for new drug. The New York Times April 19, 2010.

[4] Ioannidis JP. Why most published research findings are false. PLoS Med 2:e124, 2005.

[5] Baker M. Reproducibility crisis: Blame it on the antibodies. Nature 521:274-276, 2015.

[6] Naik G. Scientists' Elusive Goal: Reproducing Study Results. Wall Street Journal December 2, 2011.

[7] Zimmer C. A sharp rise in retractions prompts calls for reform. The New York Times April 16, 2012.

[8] Altman LK. Falsified data found in gene studies. The New York Times October 30, 1996.

[9] Weaver D, Albanese C, Costantini F, Baltimore D. Retraction: altered repertoire of endogenous immunoglobulin gene expression in transgenic mice containing a rearranged mu heavy chain gene. Cell 65:536 (inclusive), 1991.

[10] Chang K. Nobel winner in physiology retracts two papers. The New York Times September 23, 1010.

[11] Fourth paper retracted at Potti's request. The Chronicle March 3, 2011.

[12] Whoriskey P. Doubts about Johns Hopkins research have gone unanswered, scientist says. The Washington Post March 11, 2013.

[13] Lin YY, Kiihl S, Suhail Y, Liu SY, Chou YH, Kuang Z, et al. Retraction: Functional dissection of lysine deacetylases reveals that HDAC1 and p300 regulate AMPK. Nature 482:251-255, retracted November, 2013.

[14] Innovation or Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products. U.S. Department of Health and Human Services, Food and Drug Administration, 2004.

[15] Hurley D. Why Are So Few Blockbuster Drugs Invented Today? The New York Times November 13, 2014.

[16] Angell M. The Truth About the Drug Companies. The New York Review of Books Vol 51, July 15, 2004.

[17] Crossing the Quality Chasm: A New Health System for the 21st Century. Quality of Health Care in America Committee, editors. Institute of Medicine, Washington, DC., 2001.

[18] Wurtman RJ, Bettiker RL. The slowing of treatment discovery, 1965-1995. Nat Med 2:5-6, 1996.

[19] Ioannidis JP. Microarrays and molecular research: noise discovery? The Lancet 365:454-455, 2005.

[20] Weigelt B, Reis-Filho JS. Molecular profiling currently offers no more than tumour morphology and basic immunohistochemistry. Breast Cancer Research 12:S5, 2010.

[21] Personalised medicines: hopes and realities. The Royal Society, London, 2005.Available from: https://royalsociety.org/~/media/Royal_Society_Content/policy/publications/2005/9631.pdf, viewed Jan 1, 2015.

[22] Beizer B. Software Testing Techniques. Van Nostrand Reinhold; Hoboken, NJ 2 edition, 1990.

[23] Ecker JR, Bickmore WA, Barroso I, Pritchard JK, Gilad Y, Segal E. Genomics: ENCODE explained. Nature 489:52-55, 2012.

[24] Rosen JM, Jordan CT. The increasing complexity of the cancer stem cell paradigm. Science 324:1670-1673, 2009.

[25] Bandelt H, Salas A. Contamination and sample mix-up can best explain some patterns of mtDNA instabilities in buccal cells and oral squamous cell carcinoma. BMC Cancer 9:113, 2009.

[26] Knight, J. Agony for researchers as mix-up forces retraction of ecstasy study. Nature 425:109, September 11, 2003.

[27] Gerlinger M, Rowan AJ, Horswell S, Larkin J, Endesfelder D, Gronroos E, et al. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med 366:883-892, 2012.

[28] Ioannidis JP. Is molecular profiling ready for use in clinical decision making? The Oncologist 12:301-311, 2007.

[29] Ioannidis JP. Some main problems eroding the credibility and relevance of randomized trials. Bull NYU Hosp Jt Dis 66:135-139, 2008.

[30] Ioannidis JP, Panagiotou OA. Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. JAMA 305:2200-2210, 2011.

[31] Ioannidis JPA, Panagiotou OA. "Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. JAMA 305:2200-2210, 2011.

[32] Ioannidis JP: Excess significance bias in the literature on brain volume abnormalities. Arch Gen Psychiatry 68:773-780, 2011.

[33] Harris G. Diabetes drug maker hid test data, files indicate. The New York Times July 12, 2010.

[34] Berman JJ. Machiavelli's Laboratory. Amazon Digital Services, Inc., 2010.

[35] Misconduct in science: an array of errors. The Economist. September 10, 2011.

[36] Begley S. In cancer science, many 'discoveries' don't hold up. Reuters Mar 28, 2012,

[37] Abu-Asab MS, Chaouchi M, Alesci S, Galli S, Laassri M, Cheema AK, et al. Biomarkers in the age of omics: time for a systems biology approach. OMICS 15:105-112, 2011.

[38] Moyer VA; on behalf of the U.S. Preventive Services Task Force. Screening for prostate cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med May 21, 2011

[39] How science goes wrong. The Economist Oct 19, 2013.

[40] Shafer SL. Letter: To our readers. Anesthesia and Analgesia. February 20, 2009.

[41] Pear R. Obama to Request Research Funding for Treatments Tailored to Patients’ DNA. The New York Times January 24, 2015.

[42] Munos B. 2014 New Drug Approvals Hit 18-Year High. Forbes Jan 2, 2015. Available at: http://www.forbes.com/sites/bernardmunos/2015/01/02/the-fda-approvals-of-2014, viewed June 20, 2015.

[43] Berman JJ. Rare diseases and orphan drugs: Keys to Understanding and Treating Common Diseases. Academic Press, in press, 2014.

[44] Holland Frei Cancer Medicine. Kufe D, Pollock R, Weichselbaum R, Bast R, Gansler T, Holland J, Frei E, eds. BC Decker, Ontario, Canada, 2003.

key words: precision medicine, personal medicine, pharmacogenomics, pharmacogenetics, personalized medicine, genomics, bioinformatics, funding, President Obama's initiative, common diseases, rare diseases, complexity barrier, complexity ceiling, scientific error

The Genome Complexity Barrier in Precision Medicine

Elsevier has just published an editorial of mine entitled, The Data Complexity Barrier and the Surprising Importance of Rare Diseases. The premise of the article is that the common diseases of humans are way too complex to understand. Money spent on President Obama's Precision Medicine initiative will be wasted if funds are primarily directed to the common diseases. As it happens, the rare diseases are genetically simple, compared with the common diseases. Money spent on rare disease research is money well spent. In most instances, breakthroughs in understanding the rare diseases have led to new treatments for the common diseases.

I urge you to read the full editorial. Unlike much of the newsfeed hype on the subject of Precision Medicine, my article is well-researched and includes 44 references to the supporting literature.

- Jules J. Berman, Ph.D., M.D.

key words: precision medicine, pharmacogenomics, pharmacogenetics, personalized medicine, genomics, bioinformatics, funding, President Obama's initiative, common diseases, rare diseases, complexity barrier

President Obama's Precision Medicine Initiative: A Suggestion

Yesterday, Reuter's published the numbers

Total budget: $215 million in his 2016 budget for the initiative.

$130 million would go to the NIH to fund the research cohort

$70 million to NIH's National Cancer Institute

$10 million would go to the Food and Drug Administration to develop databases

$5 million would go to the Office of the National Coordinator for Health Information Technology to develop privacy standards and ensure the secure exchange of data.

I couldn't understand, from reading the Reuter's article, whether this budget covers expenses to be incurred in 2016, or whether the budget is spread over multiple years. My guess is that it's the former, and that the initiative will draw additional funds for each year that it's active. It appears that the initiative will not be confined to genomics. Other high-throughput 'omics data will also be captured (e.g., epigenomics, metabolomics).

If we've learned anything in the post-genomics era, it's that biological systems are incredibly complex. Assembling terabytes of 'omics data will probably teach us this same, frustrating, lesson AGAIN; but it's not obvious how this effort will help us to substantially prevent, diagnose and treat the common diseases.

Believe it or not, most of the clinical advances in molecular medicine have come from studying rare diseases and rare variants of common disease. Unlike the common diseases, the rare diseases are often simple (one gene -> one protein -> one pathway -> one disease). It has been a surprise to some, but drugs developed for rare diseases often have applicability to common conditions. This happens because common diseases employ disease pathways that are also found in rare diseases. In my opinion, the best way to conquer the [complex] common diseases is to start by understanding the pathways operative in the [simple] rare diseases. So, if we want to invest $250 million in 'omics research, please let's focus on disease pathways discovered in rare diseases and shared by the common diseases.

- Ⓒ 2015 Jules J. Berman

tags: rare disease, orphan disease, personalized medicine, individualized medicine, genetic testing, gene testing, molecular diagnostics, biomarkers, pharmacogenetics, pharmacogenomics, future of medicine, state of the union address, funding initiative, NIH, FDA, big pharma, lobbyists, lobbying, scientific politics, scientific ethics, disease genetics, common diseases, complex diseases, rare variants, gene variants, whole genome sequencing

President Obama's Precision Medicine Initiative: Counter-arguments

In his 2015 State of the Union Address, President Obama launched his new Precision Medicine Initiative to “bring us closer to curing diseases like cancer and diabetes -- and to give all of us access to the personalized information we need to keep ourselves and our families healthier.” The initiative will be described in greater detail in the administration's budget proposal, but the following are widely assumed to be true: 1) requested funding will be in the hundreds of millions of dollars; 2) budgeted items will received considerable bipartisan support in congress; 3) the initiative will be championed by scientific and regulatory agencies of government (e.g., NIH and FDA) and by the pharmacy industry; and 4) the goals of the initiative will be similar to the goals of earlier legislation (The Genomics and Personalized Medicine Act) co-sponspored by President Obama when he was a Senator.

This blog lists six reasons why I am very skeptical of this initiative. Before I start, I'd like to clarify that I am a huge fan of President Obama. I believe he is the greatest U.S. president since FDR, and I will be eternally grateful for all that he has done for this country and for the world. Also, I am a great believer in the importance of genomics research. Hence, when I criticize this initiative, so early in the game, I do so with a great deal of ambivalence.

Nonetheless, here are my counter-arguments to the Precision Medicine Initiative.

1. Genomics research, this past decade, has taught us that genetics is a lot more complex than we had imagined. Human traits (e.g., height, weight), common diseases (e.g., diabetes, obesity, heart disease, and cancer), may involve hundreds of gene variants, non-coding regulatory sequences, competing epigenomic influences, and so on. Extrapolating from what we know about the complexity of gene expression, it seems that we are a very long way from understanding how disease genes are controlled. If we don't know how disease genes are controlled, then, with a few exceptions [see point 3], we can't practice genomic-based medicine.

2. It has long been promised that gene sequencing will lead, rapidly, to important advances in personalized medicine (also known as pharmacogenomics, and most recently renamed “precision medicine”). As it happens, simply knowing a sequence of nucleotides has not proven to be particularly helpful for understanding diseases that involve hundreds of genes, and an unspecified number of poorly characterized regulatory modifiers. Hence, personalized medicine, has not appreciably reduced the burden of morbidity and mortality produced by common diseases [see point 3].

3. Most of genomics-based medical progress has come in the realm of rare diseases and rare subsets of common diseases. As a generalization, rare diseases tend to be caused by single gene errors, while common diseases are caused by environmental agents (e.g., infections, toxins, carcinogens) or by multi-gene errors, or by some combination of the environment and multiple genes. The reason for this genetic dichotomy between rare diseases and common diseases is discussed, in depth, in my recently published book Rare Diseases and Orphan Drugs: Keys to Understanding and Treating the Common Diseases. So yes, there are examples wherein abnormalities of one gene can account for a specific disease, but this phenomenon applies exclusively to rare diseases, to rare variants of common diseases, and to a few common genetic disorders associated with a relatively benign clinical course. In aggregate, single gene disorders account for a miniscule portion of the life-threatening disease burden in the U.S. and the world.

As an aside, I am an advocate for funding research into the genetics of rare diseases. It is the premise of my book that we have been astoundingly successful in rare disease research, and that treatments developed for the rare diseases are applicable to the common diseases [because rare and common diseases use the same cellular pathways]. Hence, a good way to conquer the common diseases is to steer the NIH research budget towards funding rare disease research.

4. It is conceivable that genomic tests will prove useful, notwithstanding the aforementioned complexities in gene controls. How so? It could be that a common disease is driven by a particular biological pathway that is dominated by one particular protein. If that protein were blocked [or stimulated in the case of an inhibitor protein], then it would be great to have a test for the pathway, or its dominant protein, or for gene variants that influence the activity of the expressed protein. Hence, precision medicine would apply in these instances. Nonetheless, the approach to precision medicine has been focused on looking for sequence variations, and I don't believe that sequence variations are bringing us much closer to finding the key proteins that drive the cellular pathways that account for common diseases. Again, the best way of finding key pathways operative in common diseases is to study the rare diseases [further explained in my book].

5. Clinical practice, based on candidate molecular tests requires lots of clinical trials and huge outcomes databases (i.e., validation data). Clinical trials are hugely expensive, and can require more than a decade to accrue patients, collect and analyze the data, and draw conclusions. Experience suggests that in most cases, the final conclusions are disappointing. Population databases, also hugely expensive, pose problems related to patient confidentiality and privacy, accuracy of data, analytic methodology, and validity of the conclusions drawn from the data. At present, we are nowhere near having the kinds of databases that we will need to confirm the value of new molecular tests.

Historically, the molecular biomarker field has been an embarrassment for the clinical research community. There have been a few successful exceptions (e.g., herceptin), but for the most part, biomarkers have been a bust. Should we really have much faith in the future of precision biomarkers, when history suggests that the odds of success are low?

6. Do we have any trustworthy authorities who can objectively claim that precision medicine is a good idea? Let's look at who's backing funding for the precision medicine initiative.

First, there's congress. Many of the members of congress, perhaps the majority, do not believe in global warming, or evolution, or the benefit of vaccinations. An appreciable number of congressmen believe that the earth is 5,000 years old, that pollution is not harmful, and that there's good evidence indicating that the end times are close at hand. Does anyone seriously believe that Congress can distinguish good science from bad science?

Of course, big pharma loves precision medicine because it generates expensive new tests and treatments that will be paid by insurers, no matter how great the cost or how small the benefit.

Then there is the matter of the federal agencies that fund or conduct science. It is the responsibility of the leaders of federal agencies to be responsive to Congress. If legislators need to justify, to their constituents, a new research initiative, then legislators will simply ask their agency heads to invent a credible scientific justification.

The question that never seems to be discussed is, “Is this the best way to improve the health of the nation, or are there alternative research initiatives that would give us a better outcome?”

- Ⓒ 2015 Jules J. Berman

tags: rare disease, orphan disease, personalized medicine, individualized medicine, genetic testing, gene testing, molecular diagnostics, biomarkers, pharmacogenetics, pharmacogenomics, future of medicine, state of the union address, funding initiative, NIH, FDA, big pharma, lobbyists, lobbying, scientific politics, scientific ethics, disease genetics, common diseases, complex diseases, rare variants, gene variants, whole genome sequencing