Thursday, March 15, 2007

Data standards should not be monopolistic

In a prior blog, I listed 16 good practice suggestions for SDOs (Standards Development Organizations).

One suggestion was:

"Make optional standards, not required standards, so that the user community is not locked into one implementation."

This suggestion seems to defy common sense. The purpose of a standard is to provide a common process for a user community. Wouldn't a standard lose its significance if it were designed to be one of many?

First off, remember that I'm only addressing data standards (not physical standards). Data standards are special because, in many cases, you can interconvert data from one standard to another quite easily. Data standards are usually developed to facilitate data exchange and interoperability in a defined data domain. It is seldom the case that a given data standard will have universal appeal. We have dozens (if not hundreds) of image format standards. The multiplicity of standards can be useful. There are times when a GIF format is superior to a JPEG and other times when a PNG format is appropriate. Most people who work with images have robust file conversion applications that make it easy to exchange many different image formats.

Yet somehow, when a committee gets together to write a data standard, they often develop a very self-centered culture that tries to eliminate the "competing" standards.

If a data domain has one standard, then patents that encumber the uses of the standard will impact negatively on everyone. If a data domain has multiple standards, then the user community can simply switch between available standards to avoid patent prosecution. They might use one standard to accomplish a task that is exempt from patent infringement (typically the task for which the standard was designed and for which no patents apply). If/when a newly patented use of the one standard emerges, the user can avoid legal headaches by switching to another data standard not covered by the patent. It's really quite simple.

Members of data standards committees should understand that the purpose of any standards effort is to serve the user community with improved methods for exchanging data, for software interoperability and for enhanced opportunities to use data. A data standard is just an arbitrary document. It hardly even rates as a "thing" since it has no physical existence. SDOs should try to make new standards that fill a particular utility "niche" not covered by other standards in the same domain. If users gravitate to the standard in preference to other standards, that's OK. But crushing the "competition" should not be a goal for any SDO.

-Jules Berman
My book, Principles of Big Data: Preparing, Sharing, and Analyzing Complex Information was published in 2013 by Morgan Kaufmann.

I urge you to explore my book. Google books has prepared a generous preview of the book contents. If you like the book, please request your librarian to purchase a copy of this book for your library or reading room.

tags: big data, metadata, data preparation, data analytics, data repurposing, datamining, data mining