Medical record de-identifier (using the doublet method)

Earlier today, I wrote a blog describing an identifier-free doublet list. This blog describes a scrubber use-case for the doublet list.

The de-identification of medical text is a good use of the doublet list. To share medical records (usually for the purposes of research) it is often imporant to remove all of the identifiers (that could link a patient to the record).

There are now available a variety of medical text scrubbers for this purpose. Most require the users to develop identifier lists for their site (list of patient names, doctor names, etc), run very slowly (typically, about one record per second), and do not remove anywhere close to all of the identifiers.

Not so for the doublet method. A doublet scrubber parses through any text, matching doublets from the text against an external identifier-free doublet list, preserving all matching doublets from the text, and blocking all non-matching words with an asterisk. If your list of doublets contains no identifiers, the scrubbed output should be perfectly de-identified. Though perfection can never be guaranteed, I have never encountered any "missed" identifiers in a text that was parsed under these conditions. A public domain list of doublets is available , but I cannot guarantee that the list is identifier-free or that it is the best list for your purposes. Feel free to modify the list, add to the list, or create your own list of identifier-free doublets.

The doublet method is described in Ruby Programming for Medicine and Biology.

For each citation, the list of authors is put on a line, and is immediately followed by its scrubbed version on the next line. Then the title of the article is put on the next line, followed by the scrubbed version of the title of the article. This pattern is repeated for the 1500+ citation.

The doublet scrubber is small (just a few dozen lines of code) and fast. It took approximately 2 seconds to parse the 15000 citations using a Perl script with access to a list of about 200,000 identifier-free doublets. I used my home computer (2.8 GHz, 512 MByte RAM). This is a scrubbing rate of 1 MegaByte per second. At this speed, a 1 GByte file could be parsed in about 15 minutes. It can parse a 1 Terabyte file in about a week. Large hospitals produce about 1 Terabyte of data each week, so this scrubber can, for now, "keep up" with the vast load of data produced by many hospitals (using a modest desktop computer).

The only limitation that I have found with the doublet scrubber is that it scrubs too much, blocking all doublets not found in the external doublet list. You can be the judge by reviewing the provided output file. The output attached here can be used to assess the effectiveness of the doublet method of text scrubbing.

-Jules Berman tags: common rule, data scrubbing, de-identification, deidentification, hipaa, medical records

---

Science is not a collection of facts. Science is what facts teach us; what we can learn about our universe, and ourselves, by deductive thinking. From observations of the night sky, made without the aid of telescopes, we can deduce that the universe is expanding, that the universe is not infinitely old, and why black holes exist. Without resorting to experimentation or mathematical analysis, we can deduce that gravity is a curvature in space-time, that the particles that compose light have no mass, that there is a theoretical limit to the number of different elements in the universe, and that the earth is billions of years old. Likewise, simple observations on animals tell us much about the migration of continents, the evolutionary relationships among classes of animals, why the nuclei of cells contain our genetic material, why certain animals are long-lived, why the gestation period of humans is 9 months, and why some diseases are rare and other diseases are common. In “Armchair Science”, the reader is confronted with 129 scientific mysteries, in cosmology, particle physics, chemistry, biology, and medicine. Beginning with simple observations, step-by-step analyses guide the reader toward solutions that are sometimes startling, and always entertaining. “Armchair Science” is written for general readers who are curious about science, and who want to sharpen their deductive skills.

The difference between "de-identification" and "anonymization"

In medical records, "de-identified" means that the personal identifiers in a record have been extracted and that it would be very difficult to re-establish any of the people mentioned in the original record.

"Anonymized" means that all of the links between a person and the person's record have been irreversibly broken so that it would be virtually impossible to re-establish any of the people in the original record.

The biggest difference between de-identification and anonymization is that in the former you can get permission to re-identify the patient from the de-identified record, so long as you have IRB (Institutional Review Board) approval. The re-identification method may be as simple as having a confidential list assigning de-identified records back to the original human subjects. There are many possible protocols that might be approved by an IRB that would permit re-identification of de-identified records.

There's no re-identification of anonymized records, because the links back to the subjects are irreversibly broken (by any one of a variety of approved anonymization methods).

What is the legal significance of this difference between "de-identification" and "anonymization"

In the U.S., two federal regulations control the use of medical records and human tissues in biomedical research: The Common Rule (Title 45 Code of Federal Regulations, Part 46, Protection of Human Subjects) and the Standards for Privacy of Individually Identifiable Health Information, Final Rule (usually referred to under the broader act, the Health Insurance Portability and Accountability Act, HIPAA)

The Common Rule sets the basic principles for protecting patients from research risks, mandating the activities of Institutional Review Boards, and using human tissues in support of medical research. It is essential reading for anyone involved in human subject research. The Common Rule also regulates the use of tissues and related records in biomedical research.

Department of Health and Human Services.45 CFR (Code of Federal Regulations), 46. Protection of Human Subjects (Common Rule). Federal Register, Volume 56, p. 28003-28032, June 18, 1991.

You can think of HIPAA as the regulation that pertains to electronic documents. HIPAA provides a list of 18 types of so-called "safe harbor" identifers that,if removed from records, would constitute de-identificaiton.

Department of Health and Human Services. 45 CFR (Code of Federal Regulations), Parts 160 through 164. Standards for Privacy of Individually Identifiable Health Information (Final Rule). Federal Register, Volume 65, Number 250, Pages 82461-82510, December 28, 2000.

If you're banking tissues, you need to follow both HIPAA and the Common Rule (in the U.S.).

Well, the Common Rule doesn't work in the realm of de-identification. The Common Rule works in the realm of anonymization. To get paragraph E4 exemption (from IRB approval) for tissue work, you need to be using anonymized records and tissues. De-identified records won't do the trick.

This means that if you're exempted from HIPAA because you've de-identified your records, you may still need to get IRB approval for tissue-related activities that fall under the Common Rule because de-identification falls short of the anonymization (so-called e4 exemption) needed for IRB exemption under the Common Rule.

---

In June, 2014, my book, entitled Rare Diseases and Orphan Drugs: Keys to Understanding and Treating the Common Diseases was published by Elsevier. The book builds the argument that our best chance of curing the common diseases will come from studying and curing the rare diseases.

I urge you to read more about my book. There's a generous preview of the book at the Google Books site. If you like the book, please request your librarian to purchase a copy of this book for your library or reading room.

- Jules J. Berman, Ph.D., M.D.