Access and Conversion of Large Data Sets
Senior Project: 1991-1992
Stephan Latta and Mark Warshal
The National Center for Atmospheric Research (NCAR) houses a mass storage
system with 1014 bits of data. This includes two distinct classes of
data: numerical representations of floating values generated from model
simulations or observing platforms and data derived from graphics objects, such
as metacode, polygons and image files.
There were two problems associated with this data storage. First, access to
data on the mass storage system is through a complex command line interface
which has many required parameters that are difficult to get correct and
tedious to specify. Second, the NCAR computing environment is heterogeneous, so
that the data must be in a form that is easily manipulated on a variety of
hardware.
This project provided two components to solve these problems. First was a
simple, interactive interface to the storage and retrieval of data from the
mass storage system. It prompts the user for the required parameters, providing
help as to the meaning and specification of parameters, and allows the user to
maintain default parameter specifications. The second aspect of the project was
to provide conversion utilities, which allow the data to be converted to a
netCDF format, a standard format in which the data is stored in a machine
independent manner. The software was written in C and FORTRAN.
|
