A 439Kb PDF of this article as it appeared in the magazine—complete with images—is available by clicking HERE
Technology-induced changes continue to occur in the surveying and mapping profession at a dizzying rate. Advances in computer software are particularly demanding of our attention due to the frequency with which changes occur and because of the impact software has on the ways our work is accomplished.
The fast evolution of computer technology has not been without significant growing pains. By way of example, consider two of the more prominent problems that many of us currently experience with our server computers and desktop workstations:
The Undesirables During the past few years, there has been a large increase in malicious software (e.g., viruses, Trojans, spyware, etc.) which can wreak havoc on our systems and ata. There are several reasons for the seriousness of this problem, but one very important factor is that some software on our computers allows the execution of undesired and unauthorized programs. Such software can be exploited, and the time and money spent installing and updating protective software (anti-virus programs, privacy programs, anti-spyware software, etc.), repairing corrupted systems, and restoring (or recreating) lost data is excessive.
File Sharing Sharing data files between applications, between users, and over time often requires too much effort, and with too much software there are simply no assurances that information contained in the data files it creates will be accessible in years to come. "Vendor lock-in" has become one of the key issues for those wishing to maintain full access to the information they’ve labored to create.
This article is the first in a series of three. The ultimate purpose of the series is to describe the Linux operating system and the use of open source software, because using such software provides a way to help alleviate these (and other) problems. However, due to the fundamental importance that I personally attribute to the issue of data formats (and my belief that the other problem exemplified is probably more self-evident to most computer users by now), I will devote the remainder of this installment to the issue of open data standards.
A "data format" is the method of encoding data and storing it in a readable and writable medium for access at a later time. Anyone designing a data format for use in a program must resolve technical issues (such as whether to store in ASCII, "binary," or some other scheme) and determine how to arrange the data within a file (e.g., NE or EN coordinate order in a file of station positions).
I’m sure that most readers have experienced the headache associated with sharing digital information with someone whose software employs a data format different from that used by your own program. You may have even experienced problems with data format incompatibilities between different versions of the same program. Sometimes there is an easy way to convert among different file formats, but sometimes conversion is either not possible or just imperfect enough to be not be viable. One of the customary ways of dealing with this problem is to make sure all users have the same version of the same program. However, this strategy can create other problems for computer users. If one or more of the group sharing information updates their software to a newer version, data compatibility issues may force other users into doing expensive upgrades, even if no other compelling reasons for upgrading exist for them.
There is also the problem known as "vendor lock-in." If the authors of a program do not disclose the file format used by their software, it is frequently very difficult or impossible to use alternative software to access the data files created. Therefore, if you have data stored in a proprietary and secret data format, it is likely that you will have only one viable option to succeed in maintaining access: You have to continue to buy and use the software which created the data files in question. Further, you have to hope that the vendor which created that data format both continues to exist and continues to support that format for as long as you expect to need access to your data. The true result of this situation is that you have lost a significant amount of control over your own information.
I suspect that by now most surveyors have been clobbered by having a "legacy application" which is no longer supported by the vendor or which cannot run on the latest generation of computer hardware and operating system. The user’s choices are probably very limited: upgrade or lose the ability to access your data. We simply need more or better options.
It’s all too easy to postpone worrying about this potential problem. Nonetheless, this state of affairs has now become intolerable for many individuals and organizations that have recognized that their data must not be "held hostage" by secret data formats. Data formats must be "open formats." The European Union and the Commonwealth of Massachusetts are pursuing initiatives to promote the use of "open standards," something they consider paramount in fulfilling their public duties.
What’s the Difference?
An open data format is simply a data format whose description is published and readily available for others to inspect. In principle, if a data format is open, a programmer can create software which is able to read and write data files in that format. This is extremely beneficial to users because it promotes more choices for suitable software and provides assurances that access to data stored in open formats can continue for long into the future. An example of an open data format would be the PDF file format.
However, an open data standard is an open format which has two additional highly desirable traits: the data format is usually designed by a not-for-profit formal standards organization whose primary objective is to promote longterm, free data interchange, and the use of the format is royalty free and free of licensing restrictions (such as those resulting from patents or patent applications.) The difference between mere "open formats" and "open standards" is monumental, for if the goals of specifying openness are to guarantee the free interchange of data among users and to have access to one’s data indefinitely, we must not only have the technical knowledge to do this, but we need to retain the legal right to do so. Adobe Systems Incorporated holds both patents and copyrights on the PDF file format, suggesting that it might not be an open standard, but Adobe licenses the use of the format on a royalty-free basis to all. It’s usually regarded as, very nearly, an open standard as well. LandXML is an example of an open standard data format relevant which is certainly relevant to our profession.
A very long article could be written ab out "intellectual property" and the debate about the impact that patents on software and data standards have upon the creation of software, data formats, and ultimately upon the users of software. It is not within the scope of this series of articles to examine that debate, nor will I present a thorough survey of all the possible data formats currently used by available surveying software to assess the openness of each. My immediate goal is simply to remind readers of the importance of maintaining control of your own data, and to advocate movement toward the use of open data standards. If you wish to read more about file format issues from those who advocate open standards, there is a magazine in PDF format called Free Software Magazine, which is free for the download. The first
issue contains two relevant pieces: "Format Wars," by Marco Fioretti, and "Free File Formats and the Future of Intellectual Freedom," by Terry Hancock. You can find them at www.freesoftwaremagazine.com.
The next two articles of this series in The American Surveyor will deal with the Linux operating system and software which is known as "free open source software." In my opinion, part of the solution to some of the computational problems that we experience today (such as data file obstacles, the plague of viruses and "malware," and others) is to adopt the use of Linux and other open source software. These new software trends will be offering us welcome new choices which deserve your consideration.
Loren Gibson is a Project Surveyor at Keith and Schnars, P.A., Fort Lauderdale, Florida, and is licensed in the states of Florida and New York. He received his MS degree in Computational Science from the State University of New York College at Brockport.
A 439Kb PDF of this article as it appeared in the magazine—complete with images—is available by clicking HERE