Chapter 10
Properties and Property Lists in HDF5

10.1. Introduction

HDF5 properties and property lists make it possible to shape or modify an HDF5 file, group, or object, or even an I/O stream, in a number of ways. For example, you can do any of the following:

Some properties enable an HDF5 application to take advantage of the capabilities of a specific computing environment while others make a file more compact; some speed the reading or writing of data while others enable more record-keeping at a per-object level. HDF5 offers nearly a hundred specific properties that can be used in literally thousands of combinations to maximize the usability of HDF5-stored data.
 

More prosaically,  

Graphic illustration of property list ecosystem

Figure 1.  The HDF5 property list ecosystem
a property list is a collection of name/value pairs that can be passed to various HDF5 functions, usually modifying default settings. Certain kinds of property lists can be created by one application and used by an application that later operates on the same file. A property list may contain separate values for each of several properties, or characteristics, of an object; without property lists, each of these values would have to be passed to each function as a separate parameter.

For instance, a file creation operation needs to know several things about a file, such as the size of the user-block at the beginning of the file, or the sizes of various file data structures. Bundling this information in a property list simplifies the interface by reducing the number of parameters to the function H5Fcreate.

One can think of the basic HDF5 property list classes as templates for building the property lists that you will use in an application.

HDF5 property lists and the property list interface (H5P) provide a mechanism for storing characteristics of objects in an HDF5 file and economically passing them around in an HDF5 application. In this capacity, property lists significantly reduce the burden of additional function parameters in the rest of the HDF5 application programming interface (the HDF5 API). Another advantage of property lists is that features can often be added to HDF5 by adding only property list functions to the API; all other requirements of the feature can be accomplished internally to the library.
 

The remaining sections in this chapter discuss the following topics:

The discussions and function listings in this chapter focus on general property operations, object and link properties, generic properties, and related functions.

Property lists and property list functions that apply only to specific types of HDF5 objects are generally listed and discussed in the chapter discussing those objects, where that information will be most convenient to users. For example, the Datasets chapter discusses dataset creation property lists and functions, dataset access property lists and functions, and dataset transfer property lists and functions. This chapter does not duplicate those discussions.

This chapter assumes an understanding of the following chapters of this HDF5 User’s Guide:

10.2. Properties, Property Lists, and Property List Classes

As illustrated in Figure 1 above, the property list environment in a three-level hierarchy:

A property list class defines the roles that property lists of that class can play. This is accomplished by means of a property list template that includes all properties that are valid in that class. In that template, each property is set to its default value.

A property list is a collection of related properties that are used together in specific circumstances. A new property list created from a property list class inherits the properties of the property list class and each property’s default value. A fresh dataset creation property list, for example, includes HDF5 properties relevant to the creation of a new dataset.

The preset default property values are intended to be reasonable for general use cases, so an application can often use the property list without further modification. On the other hand, adjusting these settings is a routine action as there are many reasons for an application to do so.

A property is the basic element of the property list hierarchy (the atomic element, if you will). Properties control things like access rights to a file, the storage layout of a dataset, or whether missing intervening groups will be automatically created when a new object is created.

10.2.1. Property List Classes

HDF5 offers several classes of property lists, as listed in the table below.  

  Table 1: Property list classes in HDF5
 
  Property List Class
       Notes
  File creation (FCPL) H5P_FILE_CREATE    Discussed in “The HDF5 File”.
  File access (FAPL) H5P_FILE_ACCESS    Discussed in “The HDF5 File”.
  File mount (FMPL) H5P_FILE_MOUNT    Used only as H5P_DEFAULT (see footnote 1).
  Object creation (OCPL) H5P_OBJECT_CREATE     
  Object copy (OCPYPL) H5P_OBJECT_COPY     
  Group creation (GCPL) H5P_GROUP_CREATE    Discussed in “HDF5 Groups”.
  Group access (GAPL) H5P_GROUP_ACCESS    Discussed in “HDF5 Groups”.
  Link creation (LCPL) H5P_LINK_CREATE     
  Link access (LAPL) H5P_LINK_ACCESS     
  Dataset creation (DCPL) H5P_DATASET_CREATE    Discussed in “HDF5 Datasets”.
  Dataset access (DAPL) H5P_DATASET_ACCESS    Discussed in “HDF5 Datasets”.
  Dataset transfer (DXPL) H5P_DATASET_XFER    Discussed in “HDF5 Datasets”.
  Datatype creation (TCPL) H5P_DATATYPE_CREATE    Discussed in “HDF5 Datatypes”.
  Datatype access (TAPL) H5P_DATATYPE_ACCESS    Discussed in “HDF5 Datatypes”.
  String creation (STRCPL) H5P_STRING_CREATE    Discussed in “HDF5 Datasets” and “HDF5 Datatypes”.
  Attribute creation (ACPL)   H5P_ATTRIBUTE_CREATE    Discussed in “HDF5 Attributes”.
 
  The abbreviations to the right of each property list class name in this table are widely used in both HDF5 programmer documentation and HDF5 source code. For example, FCPL is file creation property list, OCPL is object creation property list, OCPYPL is object copy property list, and STRCPL is string creation property list. These abbreviations may appear in either uppercase or lowercase.

The following figure graphically presents the inheritance hierarchy of HDF5’s property list classes.


Graphic illustration of property list class 
                 inheritance hierarchy

Figure 2. HDF5 property list class inheritance hierarchy
Entries above in black are property list classes that are directly accessible through the programming interface; entries in gray are HDF5 internal entities and are not user-accessible.
The red empty set symbol () below FMPL indicates that the file mount property list class is an empty class; that is, it has no settable properties (see footnote 1).

To illustrate the inheritance diagrammed above, consider the following prose example (coding examples appear in later sections of this chapter): Before creating an HDF5 dataset, you will need to determine the creation properties the dataset must have. That is, you must establish the dataset’s DCPL. This DCPL will be derived from the dataset creation property list class and will inherit all of the appropriate properties, each set to a default value. If necessary, you may then change any of those default values in the new DCPL with the appropriate H5P calls, such as H5Pset_layout and H5Pset_chunk. You can now use the newly-created and modified DCPL to create any dataset that has the same creation property requirements.

10.2.2. Property Lists

A property list is a container holding a collection of name/value pairs specifying a set of properties that are always used together. A property list contains information for one to many properties.

A new property list is derived from a property list class. When a property list is created, it contains all of the properties that are relevant to the parent property list class with each property set to its default value.

An HDF5 property list is a mechanism for passing values into HDF5 Library calls. Property lists were created to allow HDF5 calls to be extended with the equivalent of new parameters without changing the actual API or breaking existing code. The call by value nature of property lists means that property lists can be copied, reused, and discarded with ease.

Property lists can also be freely reused to create consistency. For example, a single set of file, group, and dataset creation property lists might be created at the beginning of a project and used to create hundreds, thousands, even millions, of consistent files, file structures, and datasets over the project’s life. When such consistency is important to a project, this is an economical means of providing it.

Note that the HDF5 property list functions generate fully populated property lists and that additional properties are rarely added by an application. If customized properties and property lists are required by an application or project, they can be created through the use HDF5’s generic property functionality.

10.2.3. Properties

HDF5 properties each describe a specific characteristic of an object or aspect of its access or creation. Examples include the following:

     Purpose    Examples    Property List
  Specify the driver to be used to open to a file   A POSIX driver or an MPI IO driver   FAPL
  Specify filters to be applied to a dataset   Gzip compression or checksum evaluation   DCPL
  Specify whether to record key times associated with an object   Creation time and last-modified time   OCPL
  Specify the access mode for a file opened via an external link   Read-only or read-write   LAPL

A property can be used in many property lists while a property list must belong to exactly one property list class.

As stated above, each property is initialized with a default value. For each property, there are one or more dedicated H5Pset_* calls that can be used to change that value.

Creation and access properties:
Properties fall into one of several major categories: creation properties, access properties, and transfer properties.

Creation properties control permanent object characteristics. These characteristics must be established when an object is created and are immutable; they cannot change through the life of the object.

Examples of creation properties include:

Access properties control transient object characteristics. These characteristics may change with the circumstances under which an object is accessed.

Examples of access properties include:

10.3. Programming Model

The programming model for HDF5 property lists is actually quite simple:

  1. Create a property list.
  2. Modify the property list, if required.
  3. Use the property list.
  4. Close the property list.
There are nuances, of course, but that is the basic process.

In some cases, you will not have to define property lists at all. If the default property settings are sufficient for your application, you can tell HDF5 to use the default property list.

The following sections first discuss the use of default property lists, then each step of the programming model, and finally a few less frequently used property list operations.

10.3.1. Using Default Property Lists

Default property lists can simplify many routine HDF5 tasks because you do not always have to create every property list you use.

An application that would be well-served by HDF5’s default property settings can use the default property lists simply by substituting the value H5P_DEFAULT for a property list identifier. HDF5 will then apply the default property list for the appropriate property list class.

For example, the function H5Dcreate2 calls for a link creation property list, a dataset creation property list, and a dataset access property list. If the default properties are suitable for a dataset, this call can be made as

dset_id = H5Dcreate2( loc_id, name, dtype_id, space_id;
       H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT );
HDF5 will then apply the default link creation, dataset creation, and dataset access property lists correctly.

Of course, you would not want to do this without considering where it is appropriate, as there may be unforeseen consequences. Consider, for example, the use of chunked datasets. Optimal chunking is quite dependent on the makeup of the dataset and the most common access patterns, both of which must be taken into account in setting up the size and shape of chunks.

10.3.2. Basic Steps of the Programming Model

The steps of the property list programming model are described in the sub-sections below.
10.3.2.1. Create a Property List
A new property list can be created either as an instance of a property list class or by copying an existing property list. Consider the following examples. A new dataset creation property list is first created “from scratch” with H5Pcreate. A second dataset creation property list is then created by copying the first one with H5Pcopy.
dcplA_id = H5Pcreate (H5P_DATASET_CREATE);
 
The new dataset creation property list is created as an instance of the property list class H5P_DATASET_CREATE.

The new dataset creation property list’s identifier is returned in dcplA_id and the property list is initialized with default dataset creation property values.

A list of valid classes appears above in “Table 1: Property List Classes in HDF5”.

dcplB_id = H5Pcopy (dcplA_id);
 
A new dataset creation property list, dcplB_id, is created as a copy of dcplA_id and is initialized with dataset creation property values currently in dcplA_id.
At this point, dcplA_id and dcplB_id are identical; they will both contain any modified property values that were changed in dcplA_id before dcplB_id was created. They may, however, diverge as additional property values are reset in each.

While we are creating property lists, let’s create a link creation property list; we will need this property list when the new dataset is linked into the file below:

lcplAB_id = H5Pcreate (H5P_LINK_CREATE);

10.3.2.2. Change Property Values
This section describes how to set property values.

Later in this section, the dataset creation property lists dcplA_id and dcplB_id created in the section above will be used respectively to create chunked and contiguous datasets. To set this up, we must set the layout property in each property list. The following example sets dcplA_id for chunked datasets and dcplB_id for contiguous datasets:

error = H5Pset_layout (dcplA_id, H5D_CHUNKED);
error = H5Pset_layout (dcplB_id, H5D_CONTIGUOUS);

Since dcplA_id specifies a chunked layout, we must also set the number of dimensions and the size of the chunks. Th example below specifies that datasets created with dcplA_id will be 3-dimensional and that the chunk size will be 100 in each dimension:

error = H5Pset_chunk (dcplA_id, 3, [100,100,100]);

These datasets will be created with UTF-8 encoded names. To accomplish that, the following example sets the character encoding property in the link creation property list to create link names with UTF-8 encoding:

error = H5Pset_char_encoding (lcplAB_id, H5T_CSET_UTF8);
dcplA_id can now be used to create chunked datasets and dcplB_id to create contiguous datasets. And with the use of lcplAB_id, they will be created with UTF-8 encoded names.
10.3.2.3. Use the Property List
Once the required property lists have been created, they can be used to control various HDF5 processes. For illustration, consider dataset creation.

Assume that the datatype dtypeAB and the dataspaces dspaceA and dspaceB have been defined and that the location identifier locAB_id specifies the group AB in the current HDF5 file. We have already created the required link creation and dataset creation property lists. For the sake of illustration, we assume that the default dataset access property list meets our application requirements. The following calls would create the datasets dsetA and dsetB in the group AB. The raw data in dsetA will be contiguous while dsetB raw data will be chunked; both datasets will have UTF-8 encoded link names:

    dsetA_id = H5Dcreate2( locAB_id, dsetA, dtypeAB, dspaceA_id, 
                        lcplAB_id, dcplA_id, H5P_DEFAULT );
    dsetB_id = H5Dcreate2( locAB_id, dsetB, dtypeAB, dspaceB_id, 
                        lcplAB_id, dcplB_id, H5P_DEFAULT ); 
10.3.2.4. Close the Property List
Generally, creating or opening anything in an HDF5 file results in an HDF5 identifier. These identifiers are of HDF5 type hid_t and include things like file identifiers, often expressed as file_id; dataset identifiers, dset_id; and property list identifiers, plist_id. To reduce the risk of memory leaks, all of these identifiers must be closed once they are no longer needed.

Property list identifiers are no exception to this rule, and H5Pclose is used for this purpose. The calls immediately following would close the property lists created and used in the examples above.

error = H5Pclose (dcplA_id);
error = H5Pclose (dcplB_id);
error = H5Pclose (lcplAB_id);

10.3.3. Additional Property List Operations

A few property list operations fall outside of the programming model described above. This section describes those operations.
10.3.3.1. Query the Class of an Existing Property List
Occasionally an application will have a property list but not know the corresponding property list class. A call such as in the following example will retrieve the unknown class of a known property list:
PList_Class = H5Pget_class (dcplA_id);
Upon this function’s return, PList_Class will contain the value H5P_DATASET_CREATE indicating that dcplA_id is a dataset creation property list.
10.3.3.2. Determine Current Creation Property List Settings in an Existing Object
After a file has been created, another application may work on the file without knowing how the creation properties for the file were set up. Retrieving these property values is often unnecessary; HDF5 can read the data and knows how to deal with any properties it encounters.

But sometimes an application must do something that requires knowing the creation property settings. HDF5 makes the acquisition of this information fairly straight-forward; for each property setting call, H5Pset_*, there is a corresponding H5Pget_* call to retrieve the property’s current setting.

Consider the following examples which illustrate the determination of dataset layout and chunking settings:

The application must first identify the creation property list with the appropriate get creation property list call. There is one such call for each kind of object.

H5Dget_create_plist will return a property list identifier for the creation property list that was used to create the dataset. Call it DCPL1_id.

H5Pset_layout sets a dataset’s layout to be compact, contiguous, or chunked.

H5Pget_layout called with DCPL1_id will return the dataset’s layout, either H5D_COMPACT, H5D_CONTIGUOUS, or H5D_CHUNKED.

H5Pset_chunk sets the rank of a dataset, that is the number of dimensions it will have, and the maximum size of each dimension.

H5Pget_chunk, also called with DCPL1_id, will return the rank of the dataset and the maximum size of each dimension.

If a creation property value has not been explicitly set, these H5Pget_ calls will return the property’s default value.

10.3.3.3. Determine Access Property Settings
Access property settings are quite different from creation properties. Since access property settings are not retained in an HDF5 file or object, there is normally no knowledge of the settings that were used in the past. On the other hand, since access properties do not affect characteristics of the file or object, this is not normally an issue. (See footnote 2 for exceptions.)

One circumstance under which an application might need to determine access property settings might be when a file or object is already open but the application does not know the property list settings. In that case, the application can use the appropriate get access property list call to retrieve a property list identifier. For example, if the dataset dsetA from the earlier examples is still open, the following call would return an identifier for the dataset access property list in use:

    dsetA_dacpl_id = H5Dget_access_plist( dsetA_id ); 
The application could then use the returned property list identifier to analyze the property settings.

10.4. Generic Properties Interface and User-defined Properties

HDF5’s generic property functions provide tools for managing the entire property hierarchy and for the creation and management of user-defined property lists and properties. The generic property functions also enable an application or a driver to create, modify, and manage custom properties, property lists, and property list classes. A comprehensive list of generic property functions appears below in the generic property list functions table.

Property names beginning with ‘H5’ are reserved for HDF5 Library use and should not be used for user-defined properties.

Beyond these comments and the function listing below, discussions of HDF5’s generic property interface and user-defined properties and property lists are beyond the current scope of this document.

10.5. Property List Function Summaries

General property functions, generic property functions and macros, property functions that are used with multiple types of objects, and object and link property functions are listed below.

Property list functions that apply to a specific type of object are listed in the chapter that discusses that object. For example, the Datasets chapter has two property list function listings: one for dataset creation property list functions and one for dataset access property list functions. As has been stated, this chapter is not intended to describe every property list function.

Function Listing 1. General property list functions (H5P)

C Function
Fortran Subroutine    
  Purpose

H5Pcreate
h5pcreate_f   
  Creates a new property list as an instance of a specified parent property list class.

H5copy
h5copy_f
  Creates a new property list by copying the specified existing property list.

H5Pget_class
h5pget_class_f
  Retrieves the parent property list class of the specified property list.

H5Pclose
h5pclose_f
  Closes the specified property list.

Object property functions can be used with several kinds of objects.  

Function Listing 2. Object property functions (H5P)

C Function
Fortran Subroutine    
  Purpose

Object Creation Properties

H5Pget_attr_creation_order
h5pget_attr_creation_order_f
  Retrieves tracking and indexing settings for attribute creation order.

H5Pget_attr_phase_change
h5pget_attr_phase_change_f
  Retrieves attribute storage phase change thresholds.

H5Pget_obj_track_times
h5pget_obj_track_times_f
  Determines whether times associated with an object are being recorded.

H5Pset_attr_creation_order
h5pset_attr_creation_order_f
  Sets tracking and indexing of attribute creation order.

H5Pset_attr_phase_change
h5pset_attr_phase_change_f
  Sets attribute storage phase change thresholds.

H5Pset_obj_track_times
h5pset_obj_track_times_f
  Sets the recording of times associated with an object.

Object Copy Properties

H5Padd_merge_committed_dtype_path
(none)
  Adds a path to the list of paths that will be searched in the destination file for a matching committed datatype.

H5Pfree_merge_committed_dtype_paths   
(none)
  Clears the list of paths stored in an object copy property list.

H5Pget_copy_object
h5pget_copy_object_f
  Retrieves the properties to be used when an object is copied.

H5Pget_mcdt_search_cb
(none)
  Retrieves the callback function from the specified object copy property list.

H5Pset_copy_object
h5pset_copy_object_f
  Sets the properties to be used when an object is copied.

H5Pset_mcdt_search_cb
(none)
  Sets the callback function that H5Ocopy will invoke before searching the entire destination file for a matching committed datatype.

The following table lists link creation properties. Since the creation of a link is almost always a step in the creation of an object, these properties may also be set in group creation property lists, dataset creation property lists, datatype creation property lists, and the more generic object creation property lists. Some are also applicable to the attribute creation property lists.    

Function Listing 3. Link creation property functions (H5P)
These properties can be used with any of the indicated property lists.

C Function
Fortran Subroutine    
  Purpose

H5Pget_char_encoding
h5pget_char_encoding_f   
  Queries the character encoding used to encode link or attribute names.
Any link, object, dataset, datatype, group, or attribute creation property list

H5Pset_char_encoding
h5pset_char_encoding_f   
  Sets the character encoding used to encode link and attribute names.
Any link, object, dataset, datatype, group, or attribute creation property list

H5Pget_create_intermediate_group
h5pget_create_intermediate_group_f   
  Queries setting for creation of intermediate groups.
Link creation property list, which in turn can be used in the create call for any dataset, datatype, or group

H5Pset_create_intermediate_group
h5pset_create_intermediate_group_f   
  Specifies whether to create intermediate groups when they do not already exist.
Link creation property list, which in turn can be used in the create call for any dataset, datatype, or group

Generic property functions allow an application to create properties, property lists, and property list classes beyond those provided by HDF5.   Beyond this function listing and the generic and user-defined properties section above, discussions of HDF5’s generic property interface and user-defined properties and property lists are beyond the current scope of this document.  

Function Listing 4. Generic property functions (H5P)

C Function or Macro
Fortran Subroutine    
  Purpose

H5Pclose_class
h5pclose_class_f
  Closes an existing property list class.

H5Pcopy_prop
h5pcopy_prop_f
  Copies a property from one property list or property list class to another.

H5Pcreate_class
h5pcreate_class_f
  Creates a new property list class.

H5Pequal
h5pequal_f
  Compares two property lists or property list classes for equality.

H5Pexist
h5pexist_f
  Queries whether a property name exists in a property list or property list class.

H5Pget
h5pget_f
  Queries the value of a property.

H5Pget_class_name
h5pget_class_name_f
  Retrieves the name of a property list class.

H5Pget_class_parent
h5pget_class_parent_f   
  Retrieves the parent class of a property list class.

H5Pget_nprops
h5pget_nprops_f
  Queries the number of properties in a property list or property list class.

H5Pget_size
h5pget_size_f
  Queries the size of a property value in bytes.

H5Pinsert
h5pinsert_f
  Registers a temporary property with a property list.

H5Pisa_class
h5pisa_class_f
  Determines whether a property list is a member of a class.

H5Piterate
h5piterate_f
  Iterates over the properties in a property list or property list class.

H5Pregister
h5pregister_f
  Registers a permanent property with a property list class.

H5Premove
h5premove_f
  Removes a property from a property list.

H5Pset
h5pset_f
  Sets a property list value.

H5Punregister
h5punregister_f
  Removes a property from a property list class.

10.6. Additional Property List Resources

Property lists are ubiquitous in an HDF5 environment and are therefore discussed in many places in HDF5 documentation. The following sections and listings in the HDF5 User’s Guide are of particular interest:
In the “HDF5 Data Model and File Structure” chapter, see the “Property List” section.
 
In the “HDF5 File” chapter, see the following sections and listings:
File Creation and File Access Properties
File Property Lists and its sub-sections
Example with the File Creation Property List
Example with File Access Property List
File creation property list functions
File access property list functions
File driver functions
 
In the “HDF5 Attributes” chapter, see the “Attribute creation property list functions” listing.
 
In the “HDF5 Groups” chapter, see the “Group creation property list functions” listing.
 
Property lists are discussed throughout the “HDF5 Datasets” chapter.

All property list functions are described in the “H5P: Property List Interface” section of the HDF5 Reference Manual. The function index at the top of the page provides a categorized listing grouped by property list class. Those classes are listed below:

The in-memory file image feature of HDF5 uses property lists in a manner that differs substantially from their use elsewhere in HDF5. Whose who plan to use in-memory file images must study “File Image Operations” (PDF) in the Advanced Topics in HDF5 collection.


Footnotes

1. File mount properties
While the file mount property list class H5P_FILE_MOUNT is a valid HDF5 property list class, no file mount properties are defined by the HDF5 Library. References to a file mount property list should always be expressed as H5P_DEFAULT, meaning the default file mount property list.

2. Access and creation property exceptions
There are a small number of exceptions to the rule that creation properties are always retained in a file or object and access properties are never retained.

The following properties are file access properties but they are not transient; they have permanent and different effects on a file. They could be validly classified as file creation properties as they must be set at creation time to properly create the file. But they are access properties because they must also be set when a file is reopened to properly access the file.
     Property    Related function
     Family file driver   H5Pset_fapl_family
     Split file driver   H5Pset_fapl_split
     Core file driver  

H5Pset_fapl_core
The following is a link creation property, but it is not relevant after an object has been created and is not retained in the file or object.
     Property   Related function
     Create missing intermediate groups   H5Pset_create_intermediate_groups