How to map pieces of information to NeXus

Conceptually, mapping between representations of concepts and instance data is a key tasks in information science. The plugin pynxtools-apm implements this specifically for the file and serialization formats used within the research field of atom probe tomography. The plugin offloads this code from eventually other software where NeXus is used to avoid that every user and software developer has to make individual changes in their own tools.

Technically, instance data representing concepts within the realm of one ontology, or a semantic file format like NeXus, are considered the source (src). Source data are mapped on instance data representing a concept in NeXus as the target (trg). This mapping is only performed by pynxtools-apm, if src and trg can be transformed such that src and trg are connected via a relationship such as is_equivalent or same_as relationship. A transformation can be as simple as that an instance src has an elementary data type (e.g. a string or a single-precision floating point value) that is copied into an instance trg using the same data type. In practice, though the situation is more complex such that the mapping demands further normalization or processing. One example is when src represents a tilt angle with unit radiants used at the src side but trg requires that this value should be stored in degrees and maybe even promoted from single precision to double precision. In this case, the transformation is composed of a copy, a multiplication, and value promoting of the value that was read from src, convert from radiants to degrees, and is finally returned as value behind trg.

Such transformations are configured via the respective files in the config directory of pynxtools-apm. Upon parsing, it is the set of functions in mapping_functor.py which implements the actual transformations by reading the configuration and returning the properly formatted and transformed target to successively fill the template dictionary variable. The name functor is used because mapping may demand to use not only a copy or data type conversion as it was explained. It is this template variable from which core functions like convert.py of the pynxtools write the actual NeXus/HDF5 file. The latter tool is also referred to as the dataconverter of pynxtools.

The actual reading of content from file formats is implemented in the ifes_apt_tc_data_modeling Python library. This project is backed by the International Field Emission Society (IFES). IFES acknowledges the project's aim to provide a single implementation of high quality and tested readers for file formats used in atom probe tomography to enable members of the community to work together instead of having to maintain a zoo of parsers. A similar activity that we here would like to acknowledge is rosettasciio.