DATA SIMPLIFICATION: Specifications to the Rescue!

Over the next few weeks, I will be writing on topics related to my latest book, Data Simplification: Taming Information With Open Source Tools (release date March 23, 2016). I hope I can convince you that this is a book worth reading.

Blog readers can use the discount code: COMP315 for a 30% discount, at checkout.

---

Today's blog continues yesterday's discussion of Standards and Specifications

Despite the problems inherent in standards, government committees cling to standards as the best way to share data. The perception is that in the absence of standards, the necessary activities of data sharing, data verification, data analysis, and any meaningful validation of the conclusions will be impossible to achieve (1). This long-held perception may not be true. Data standards, intended to simplify our ability to understand and share data, may have increased the complexity of data science. As each new standard is born, our ability to understand our data seems to diminish. Luckily, many of the problems produced by the proliferation of data standards can be avoided by switching to a data annotation technique broadly known as "specification." Although the terms "specification" and "standard" are used interchangeably, by the incognoscenti, the two terms are quite different from one another. A specification is a formal way of describing data. A standard is a set of requirements, created by an standards development organization, that comprise a pre-determined content and format for a set of data.

A specification is an accepted method for describing objects (physical objects such as nuts and bolts; or symbolic objects, such as numbers; or concepts expresed as text). In general, specifications do not require explicit items of information (i.e. they do not impose restrictions on the content that is included in or excluded from documents), and specifications do not impose any order of appearance of the data contained in the document (i.e., you can mix up and rearrange the data records in a specification if you like). Specifications are not generally certified by a standards organization. Examples of specifications are RDF (Resource Description Framework) produced by the W3C (WorldWide Web Consortium), and TCP/IP (Transfer Control Protocol/Internet Protocol), maintained by the Internet Engineering Task Force. The most widely implemented specifications are simple; thus, easily adopted.

Specifications proved a simple and uniform way of representing the information you choose to include in your reports, messages, and files. Some of the most useful and popular specifications are XML, RDF, Notation 3, and Turtle. In general, specifications do not require explicit items of information (i.e. they do not impose restrictions on the content that is included in or excluded from documents), and specifications do not impose any order of appearance of the data contained in the document (i.e., you can mix up and rearrange the data records in a specification if you like). Specifications are not typically certified by a standards organization, but they are developed by special interest groups. Their legitimacy depends on their popularity.

Files that comply with a specification can be parsed and manipulated by generalized software designed to parse the markup language of the specification (e.g., XML, RDF) and to organize the data into data structures defined within the file.

Specifications serve most of the purposes of a standard, plus providing many important functions that standards typically lack (e.g., full data description, data exchange across diverse types of data sets, data merging, and semantic logic). Data specifications spare us most of the heavy baggage that comes with a standard, which includes: limited flexibility to include changing data objects, locked-in data descriptors, licensing and other intellectual property issues, competition among standards that compete within the same data domain, and bureaucratic overhead (2).

Most importantly, specifications make standards fungible. A good specification can be ported into a data standard, and a reasonably good data standard can be ported into a specification. For example, there are dozens of image formats (e.g., jpeg, png, gif, tiff). Although these formats have not gone through a standards development process, they are used by billions of individuals and have achieved the status as de facto standards. For most of us, the selection of any particular image format is inconsequential. Data scientists have access to robust image software that will convert images from one format to another.

A common mistake committed by data scientists is to convert all their data (legacy data and newly acquired data) into a contemporary standard, and then relying on analytic software that is designed to operate exclusively upon the chosen standard. Doing so only serves to perpetuate their frustrations. You can be certain that your data standard and your software application will be unsuitable for the next generation of data scientists. It makes much more sense to port data into a general specification, from which data can be ported to any current or future data standard.

References:

[1] National Committee on Vital and Health Statistics. Report to the Secretary of the U.S. Department of Health and Human Services on Uniform Data Standards for Patient Medical Record Information. July 6, 2000. Available from: http://www.ncvhs.hhs.gov/hipaa000706.pdf

[2] Berman JJ. Repurposing Legacy Data: Innovative Case Studies. Morgan Kaufmann, Waltham, MA, 2015.

- Jules Berman

key words: computer science, data analysis, data repurposing, data simplification, data wrangling, information science, simplifying data, taming data, complexity, standards, specifications, semantic web, jules j berman

DATA SIMPLIFICATION: Substandard Standards

Over the next few weeks, I will be writing on topics related to my latest book, Data Simplification: Taming Information With Open Source Tools (release date March 23, 2016). I hope I can convince you that this is a book worth reading.

Blog readers can use the discount code: COMP315 for a 30% discount, at checkout.

---

"The nice thing about standards is that you have so many to choose from." -Andrew S. Tanenbaum

Data standards are the false gods of informatics. They promise miracles, but they can't deliver. The biggest drawback of standards is that they change all the time. If you take the time to read some of the computer literature from the 1970s or 1980s, you will come across the names of standards that have long-since fallen into well-deserved obscurity. You may find that the literature from the 1970s is nearly impossible to read with any level of comprehension, due to the large number of now-obsolete standards-related acronyms scattered through every page. Today's eternal standard is tomorrow's indecipherable gibberish (1).

The Open Systems Interconnection (OSI) was an internet protocol created in 1977 with approval from the International Organization for Standardization. It has been supplanted by TCP/IP, the protocol that everyone uses today. A hand full of programming languages have been recognized as standards by the American National Standards Institute. These include Basic, C,, Ada and Mumps. Basic and C are still popular languages. Ada, recommended by the Federal Government, back in 1995, as the recommended language for all high performance software applications, is virtually forgotten (2). Mumps is still in use, particularly in hospital information systems, but it changed its name to M, lost its allure to a new generation of programmers, and now comes in various implementations that may not strictly conform to the original standard.

In many cases, as a standard matures, it becomes hopelessly complex. As the complexity becomes unmanageable, those who profess to use the standard may develop their own idiosyncratic implementations. Organizations that produce standards seldom provide a mechanism to ensure that the standard is implemented correctly. Standards have long been plagued by non-compliance or (more frequently) under-compliance. Over time, so-called standard-compliant systems tend to become incompatible with one another. The net result is that legacy data, purported to conform to a standard format, is no longer understandable.

Regarding versioning, it is a very good rule of thumb that when you encounter a standard whose name includes a version number (e.g. International Classification of Diseases-10, Diagnostic and Statistical Manual of Mental Disorders-5), you can be certain that the standard is unstable, and must be continually revised. Some continuously revised standards cling tenaciously to life, when they really deserve to die. In some cases, a poor standard is kept alive indefinitely by influential leaders in their fields, or by entities who have an economic stake in perpetuating the standard.

Raymond Kammer, then Director of the U.S. National Institute of Standards and Technology, understood the down-side of standards. In a year 2000 government report, he wrote that "the consequences of standards can be negative. For example, companies-and nations-can use standards to disadvantage competitors. Embodied in national regulations, standards can be crafted to impede export access, sometimes necessitating excessive testing and even redesigns of products. A 1999 survey by the National Association of Manufacturers reported that about half of U.S. small manufacturers find international standards or product certification requirements to be barriers to trade. And according to the Transatlantic Business Dialogue, differing requirements add more than 10% to the cost of car design and development." (3)

As it happens, data standards are seldom, if ever, implemented properly. In some cases, the standards are simply too complex to comprehend. Try as they might, every implementation of a complex standard is somewhat idiosyncratic. Consequently, no two implementations of a complex data standard are equivalent to one another. In many cases, corporations and government agencies will purposefully veer from the standard to accommodate some local exigency. In some cases, a corporation may find it prudent to include non-standard embellishments to a standard to create products or functionalities that cannot be easily reproduced by their competitors. In such cases, customers accustomed to a particular manufacturer's rendition of a standard may find it impossible to switch providers).

The process of developing new standards is costly. Interested parties must send representatives to many meetings. In the case of international standards, meetings occur in locations throughout the planet. Someone must pay for the expertise required to develop the standard, improve drafts, and vet the final version. Standards development agencies become involved in the process, and the end-product must be shepherded through one of the agencies that confer final approval. After a standard is approved, it must be accepted by its intended community of users. Educating a community in the use of a standard is another expense. In some cases, an approved standard never gains traction. Because standards cost a great deal of money to develop, it is only natural that corporate sponsors play a major role in the development and deployment of new standards. Software vendors are clever and have learned to benefit from the standards-making process. In some cases, members of a standards committee may knowingly insert a fragment of their own patented property into the standard. After the standard is released and implemented, in many different vendor systems, the patent holder rises to assert the hidden patent. In this case, all those who implemented the standard may find themselves required to pay a royalty for the use of intellectual property sequestered within the standard (4).

Corporations can profit from standards by obtaining patents on the uses of the standard; not on the patent itself. For example, an open standard may have been created that can be obtained at no cost, and that is popular among its intended users, and that contains no hidden intellectual property. An interested corporation or individual may discover a use for the standard that is non-obvious, novel,and useful; these are the three criteria for awarding patents. The corporation or individual can patent the use of the standard, without needing to patent the standard itself. The patent holder will have the legal right to assert the patent over anyone who uses the standard for the purpose claimed by the patent. This patent protection will apply even when the standard is free and open (4).

Despite the problems inherent in standards, government committees cling to standards as the best way to share data. The perception is that in the absence of standards, the necessary activities of data sharing, data verification, data analysis, and any meaningful validation of the conclusions will be impossible to achieve (5). This long-held perception may not be true. Data standards, intended to simplify our ability to understand and share data, may have increased the complexity of data science. As each new standard is born, our ability to understand our data seems to diminish. Luckily, many of the problems produced by the proliferation of data standards can be avoided by switching to a data annotation technique broadly known as "specification." Although the terms "specification" and "standard" are used interchangeably, by the incognoscenti, the two terms are quite different from one another. A specification is a formal way of describing data. A standard is a set of requirements, created by an standards development organization, that comprise a pre-determined content and format for a set of data.

More on specifications in tomorrow's blog.

References:

[1] Berman JJ. Repurposing Legacy Data: Innovative Case Studies. Morgan Kaufmann, Waltham, MA, 2015.

[2] FIPS PUB 119-1. Supersedes FIPS PUB 119. 1985 November 8. Federal Information Processing Standards Publication 119-1 1995 March 13. Announcing the Standard for ADA. Available from: http://www.itl.nist.gov/fipspubs/fip119-1.htm, viewed August 26, 2012.

[3] Kammer RG. The Role of Standards in Today's Society and in the Future. Statement of Raymond G. Kammer, Director, National Institute of Standards and Technology, Technology Administration, Department of Commerce, Before the House Committee on Science Subcommittee on Technology, September 13, 2000.

[4] Berman JJ. Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information. Morgan Kaufmann, Waltham, MA, 2013.

[5] National Committee on Vital and Health Statistics. Report to the Secretary of the U.S. Department of Health and Human Services on Uniform Data Standards for Patient Medical Record Information. July 6, 2000. Available from: http://www.ncvhs.hhs.gov/hipaa000706.pdf


- Jules Berman

key words: computer science, data analysis, data repurposing, data simplification, data wrangling, information science, simplifying data, taming data, complexity, standards, specificationsjules j berman

More Introduction to RDF

As discussed in an earlier post, RDF (Resource Description Framework) is a formal method for describing specified data objects with paired metadata and data.

It is important to understand that in informatics, assertions only have meaning when a pair of metadata and data (the descriptor for the data and the data itself) is assigned to a specific subject.

The "triples" that form the basis of RDF specifications are: Specified subject then metadata then data

Examples of triples that might be found in a medical dataset:

“Jules Berman” “blood glucose level” “85”
“Mary Smith” “blood glucose level” “90”
“Samuel Rice” “blood glucose level" "200"
“Jules Berman” “eye color” “brown”
“Mary Smith” “eye color” “blue”
“Samuel Rice” “eye color" "green"


Some triples found in a haberdasher's dataset

“Juan Valdez” “hat size” “8”
“Jules Berman” “hat size” “9”
“Homer Simpson” “hat size” “9”
“Homer Simpson” “hat_type” “bowler”

Triples collected from both datasets whose subject is "Jules Berman"

“Jules Berman” “blood glucose level” “85”
“Jules Berman” “eye color” “brown”
“Jules Berman” “hat size” “9”

This is a simple example of data integration over heterogeneous datasets!

Triples can port their meaning between different databases because they bind described data to a specified subject. This supports data integration of heterogeneous data and facilitates the design of software agents. A software agent, as used here, is a program that can interrogate multiple RDF documents on the web, initiating its own actions based on inferences yielded from retrieved triples.

RDF (Resource Description Framework) is a syntax for writing computer-parsable triples. For RDF to serve as a general method for describing data objects, we need to answer the following four questions:.

1. How does the triple convey the unique identity of its subject? In the triple, “Jules Berman” “blood glucose level” “85”, The name "Jules Berman" is not unique and may apply to several different people.

2. How do we convey the meaning of metadata terms? Perhaps one person's definition of a metadata term is different from another person's. For example, is "hat size" the diameter of the hat, or the distance from ear to ear on the person who is intended to wear the hat, or a digit selected from a pre-defined scale?

3. How can we constrain the values described by metadata to a specific datatype? Can a person have an eye color of 8? Can a person have an eye color of "chartreuse"?

4. How can we indicate that a unique object is a member of a class and can be described by metadata shared by all the members of a class?

In subsequent blog posts, we'll examine how RDF provides answers to these four questions.

-Jules Berman tags: data integration, meaning, rdf, specifications, standards, triples, science

---

Science is not a collection of facts. Science is what facts teach us; what we can learn about our universe, and ourselves, by deductive thinking. From observations of the night sky, made without the aid of telescopes, we can deduce that the universe is expanding, that the universe is not infinitely old, and why black holes exist. Without resorting to experimentation or mathematical analysis, we can deduce that gravity is a curvature in space-time, that the particles that compose light have no mass, that there is a theoretical limit to the number of different elements in the universe, and that the earth is billions of years old. Likewise, simple observations on animals tell us much about the migration of continents, the evolutionary relationships among classes of animals, why the nuclei of cells contain our genetic material, why certain animals are long-lived, why the gestation period of humans is 9 months, and why some diseases are rare and other diseases are common. In “Armchair Science”, the reader is confronted with 129 scientific mysteries, in cosmology, particle physics, chemistry, biology, and medicine. Beginning with simple observations, step-by-step analyses guide the reader toward solutions that are sometimes startling, and always entertaining. “Armchair Science” is written for general readers who are curious about science, and who want to sharpen their deductive skills.

Ex Parte Reexamination of Patents

The USPTO (U.S. Patent and Trademark Organization) offers a method for the ex parte review of a previously issued patent.

This means that a single party (ex parte means from one party) can request the USPTO to review a patent to determine whether it was issued in error. Such a procedure would not involve the entity holding the patent.

The USPTO has specific criteria for determining when an ex parte review can be permitted, hinging on "a substantial new question of patentability".

This process could occasionally be an option for SDOs (standards development organizations) that are not directly involved in a patent infringement lawsuit involving the use of their standard. If the SDO wishes to defend the free use of their standard in cases for which an encumbering patent seems trivial or non-original, this might be something to consider.

This post (and any post on this blog site that touches on a legal subject) is not legal advice. This is just a discussion of a published USPTO document.

-Jules Berman

---

My book, Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information was published in 2013 by Morgan Kaufmann.

I urge you to explore my book. Google books has prepared a generous preview of the book contents.

tags: big data, metadata, data preparation, data analytics, data repurposing, datamining, data mining, patent infringement, sdo, specifications, standards, uspto

Patenting the uses of the DICOM medical image standard

In a prior post, I described how the uses of standards can be patented, using DICOM as an example. DICOM (Digital Imaging and Communications in Medicine) is being actively promoted as a universal and exclusive standard for all medical images.

This blog examines one instance, in some detail, of a patent related to a common use of the DICOM standard.

U.S. Patent 6,725,231 is entitled "DICOM XML DTS/schema generator". This patent was filed March 27, 2001 by Jingkun H and Kwok Pun Lee and assigned to Philips Electronics. The patent was awarded on April 20, 2004.

The methods covered by a patent are enumerated in a list of patent claims. Claim 1 (of 15 claims) for this patent is:

"1. A method for mapping a DICOM specification into an XML document, comprising: mapping each entry of a DICOM table of the DICOM specification into a corresponding XML element of a plurality of XML elements, outputting each XML element of the plurality of XML elements to the XML document, in an output format that conforms to at least one of: an XML document-type-definition and an XML Schema. "

The reach of a patent is extended if it is awarded in both U.S. and International Patent Offices. The same application, has been awarded by the World Intellectual Property Organization, (WIPO), WO/2002/077896 DICOM XML DTS/schema generator.

Even after a patent has been awarded, it can sometimes be successfully argued, in court, that the claims are obvious or non-original and cannot be asserted against a user.

How do scientists demonstrate that an idea is original and non-obvious? They publish their work in a respected journal in their field of work. Journals are expected to reject submissions that are obvious or for which prior art (earlier publications) exist. For centuries, scientists have used publications as evidence of the validity, originality and scientific value of their work.

Jingkun H and Kwok Pun Lee published three original papers in the Journal of the American Medical Informatics Association (JAMIA), that are contemporary with their patent applications, and that describe methods related to their patents.

Zhao L, Lee KP, Hu J.
Generating XML schemas for DICOM structured reporting templates.
J Am Med Inform Assoc. 2005 Jan-Feb;12(1):72-83.

Lee KP, Hu J.
XML Schema Representation of DICOM Structured Reporting.
J Am Med Inform Assoc. 2003 Mar-Apr;10(2):213-23.

Tirado-Ramos A, Hu J, Lee KP.
Information object definition-based unified modeling language
representation of DICOM structured reporting: a case study of
transcoding DICOM to XML.
J Am Med Inform Assoc. 2002 Jan-Feb;9(1):63-71.

These described steps exemplify the way that the uses of a standard were patented. As shown previously, many uses of the DICOM standard have been included in current patent applications.

Having multiple image standards provides users an "out" when one standard becomes excessively encumbered. As discussed in a prior post, there are times when using a specification, rather than a standard, facilitates the the free, unencumbered exchange of annotations and image binary data.

-Jules Berman

tags: DICOM, intellectual property, medical images, patents, specifications, standards

---

My book, Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information was published in 2013 by Morgan Kaufmann.

I urge you to explore my book. Google books has prepared a generous preview of the book contents.

biomedical standards

Bruce Friedman and I have informally discussed a disturbing problem in health informatics: the insinuation of intellectual property (via licenses and patents) into healthcare data standards. This issue also arose during my digital imaging
presentation, February 6, at CaBIG.

Industry and government agree that we need to have standard ways of describing, organizing, and exchanging data. In the U.S., there are thousands of standards development organizations (SDOs) busily creating new standards and revising existing standards. In the healthcare industry, it is likely that a small number of standards will emerge as the "big winners" in this evolutionary struggle. A great deal of money will be invested in the development of software and other implementing technologies for those standards that are recommended or required by the U.S. Government.

Unfortunately, the implementation of standards in commercial systems (even standards that have been recommended or required by the government) may put commercial entities at some legal risk.

--The standard may contain methods that are covered by existing
patents.

--The standard may itself be "owned" by entities that exert a license.

--The uses of the standard (not the standard itself) may be patented.

I will be writing a series of blogs over the next few weeks that discuss problematic IP issues related to biomedical standards.

Next blog in the series

---

My book, Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information was published in 2013 by Morgan Kaufmann.

I urge you to explore my book. Google books has prepared a generous preview of the book contents.

Jules J. Berman, Ph.D., M.D.
tags: big data, metadata, data preparation, data analytics, data repurposing, datamining, data mining, intellectual property, patents, standards