NCDF_DATA__DEFINE
This program is designed to make it easier to browse and read the data and metadata in netCDF and HDF files. The user can browse files, and read the data and metadata into main-level IDL variables. New netCDF and HDF files can be opened at any time. The user interacts with the program via a browser window (GUI) or directly through the methods of the object. The program implements an IDL object. Note that only HDF files with scientific datasets (SD) can be read currently. There is no support for VDATA objects or other objects sometimes found in HDF files. Also note that when variables are returned from HDF files, they are returned in a calibrated form, if calibration information about the variable is present in the file. Calibration information is presented as an extra variable attribute in the browser. calibratedData = calData.cal * (uncalibratedData - calData.offset)
FANNING SOFTWARE CONSULTING David Fanning, Ph.D. 1645 Sheely Drive Fort Collins, CO 80526 USA Phone: 970-221-0438 E-mail: david@idlcoyote.com Coyote's Guide to IDL Programming: http://www.idlcoyote.com
File I/O
IDL> nCDFObject = Obj_New('NCDF_DATA', filename)
filename: The name of a netCDF or HDF file to open and browse.
BROWSE: If this keyword is set, the Browse Window is invoked as soon as the object is initiated. DESTROY_FROM_BROWSER: As with all objects, this object is persistent until it is destroyed. However, with this keyword set, the object will be destroyed when the user closes the Browse Window. EXTENSION: In general, netCDF and HDF files use *.nc, *.ncf, *.ncdf of *.hdf file extensions to identify themselves as netCDF or HDF files. Some users have their own file extensions. You can use this keyword to identify the file extension you wish to use. If set here, it will be used as the file filter in place of the normal file extensions in DIALOG_PICKFILE. obj = ('NCDF_DATA', file, EXTENSION='*.bin') NO_READ_ON_PARSE: Normally, when a file is opened it is parsed for information. One piece of information is the minimum and maximum values of the variables. This requires actually reading the variables. This can slow things down considerably is the variable is large. Setting this keyword will suppress the reading of the variables during the parsing of the data file, with the result that no minimum or maximum values will be reported.
This program is designed to be flexible in how it is used, so it can be used in both interactive and non-interactive (called directly) ways. A less flexible way of interacting with the program is via the NCDF_BROWSER program, which is a front-end to this object. The netCDF and HDF file formats are thought to be "standards". And to a large extent, they are. But files are not always created to standards, and both netCDF and HDF files can be quirky. If you look carefully at the code you will see places where I work around quirks in the files I typically use on a daily basis. If you find you can't read a particular file, let me know about it. I may be able to improve the program in such as way that it can be read. This program is not meant to be the be-all and end-all of programs. Rather, it is a tool I use, and improve upon whenever necessary, in my own work with netCDF and HDF files. It will get better for all of us if you report problems to me directly.
The following methods can be used directly. ncdfObject -> Browse ; Use GUI to browse file data and metadata. ncdfObject -> OpenFile, filename ; Opens a new netCDF or HDF file. globalAttr = ncdfObject -> ReadGlobalAttr() ; Return a structure containing global attributes. attribute = ncdfObject -> ReadAttribute(attrname); Return an attribute, identified by name. dim = ncdfObject -> ReadDimension(dimName) ; Return a dimension, identified by name. variable = ncdfObject -> ReadVariable(varname) ; Return a variable, identified by name. varstruct = ncdfObject -> ReadVariableWithAttr(varname) ; Return a variable, identified by ; name, along with its attributes. allData = ncdfObject -> ReadFile(filename) ; Read all the data in the file, into structures.
IDL> filename = 'example.nc' IDL> ncdfObj = Obj_New('NCDF_DATA', filename) IDL> ncdfObj -> Browse IDL> Obj_Destroy, ncdfObj
Written by: David W. Fanning, 03 Feb 2008. Used ideas from many people, including Chris Torrence, Ken Bowman, Liam Gumely, Andrew Slater, and Paul van Delst. Added EXTENSION keyword, resizeable TLB, and ability to download individual global attibutes. DWF. 04 Feb 2008. Added ReadDimension and ReadVariableWithAttr methods. DWF. 05 Feb 2008. Ill-formed attribute names giving me fits. Now doing checks with IDL_VALIDNAME before creating structures. 06 February 2008. DWF. Same problem. Wide use of IDL_VALIDNAME everywhere it seems wise. 06 Feb 2008. DWF. Added functionality to read a variable with its attributes from the browser interface, and fixed a problem with reading CHAR values. 2 March 2008. DWF. Fixed a problem with changing variable name when reading variable plus attributes. 6 March 2008. DWF. Fixed a problem with not setting GLOBAL keyword when inquiring about global attribute. 6 March 2008. DWF. Made sure file was parsed before attempting to read variables and attributes to avoid errors. 7 March 2008. DWF. Small bug with variable attributes fixed. 18 Dec 2008. DWF. Added ability to read HDF files containing Scientific Datasets (SD). 21 February 2009. DWF. Added error handling and protection for NCDF variables that have a dimension of length zero. 22 April 2009. DWF. Added NO_READ_ON_PARSE keyword. 22 April 2009. DWF. Now convert NCDF CHAR type variables to strings on output. 22 April 2009. DWF Fixed a problem with the directory being correct when file name passed in. 11 May 2009. DWF. Added COUNT, OFFSET, and STRIDE keywords to ReadVariable method. 25 June 2009. DWF. When reading a netCDF variable by itself (without it's attributes), the program now looks for a SCALE_FACTOR and ADD_OFFSET attribute, and if found will apply this to the variable before it is returned to the user. 24 August 2009. DWF. Added the methods GetAttrNames, GetVarNames, GetVarAttrNames, and ReadVarAttr to retrieve specfic information from the data files. 16 November 2009. DWF. Modified the ReadVariableWithAttr method to include the number of dimensions (in the NDIMS field, and the dimensions (in the DIMS field) in the return structure. For HDF files, the DIMS field is a vector of the dimensions of the variable. For netCDF files, the DIMS field is a vector of dimension IDs for the dimensions of the variable. 27 Nov 2009. DWF. Andy Meigs alerted me to a problem creating a structure when the ncdf variable name is ill-formed according to IDL structure tag name rules. Fixed in the ReadFile method. 30 November 2009. DWF. Added NO_NEW_FILE keyword to the BROWSE method. This keyword will suppress the OPEN FILE button on the browse interface. 3 Feb 2010. DWF. Made the default browser size a bit larger to accomodate longer variable names. 3 Feb 2010. DWF. Add a check for HDF/netCDF file type in the INIT method to better accommodate reading data from the file without first parsing the file. 16 March 2010. DWF. Changed the ReadVariable for netCDF files to now check for missing data, using either the depreciated missing_value attribute or the compliant _FillValue attribute. Missing data is now identified via new output keywords MISSINGINDICES and FILLVALUE, and missing data is not scaled or offset, if these operations are applied to the data prior to return. 21 March 2010. DWF. Problem with these changes, fixed 23 March 2010. DWF. Fixed a problem with memory leakage when the input file cannot be read. 1 May 2010. DWF. Fixed a problem with memory leakage from created structures. 1 May 2010. DWF. Have done some work on parsing HDF-EOS swath files, but currently unused in code. 15 May 2010. DWF. Modified the ReadVariable method to check for 0 length dimensions when reading variables from HDF files. 21 July 2010. DWF. Modified the global attribute structure so that the "filename" field, which holds the name of the netCDF of HDF file is now named "ncdf_filename" or "hdf_filename". This will avoid conflicts with global attributes with "filename". 20 January 2011. DWF. Typo in the section reading calibration data fixed. 12 March 2013. DWF. Problem reading a variable containing no attributes fixed. 20 June 2014. DWF.