Skip to content

Getting started

This is the entry point for anybody that is new to the NeXus data format and to FAIRmat/NOMAD. It serves as a guide to getting started with the pynxtools software.

What should you should know before reading this guide?

Nothing!

However, to get started, it does not hurt to read the following explanations:

What you will know at the end of this guide?

You will have

  • a basic understanding of what this software is about and its capabilities
  • how the different software tools are interconnected

What is NeXus?

NeXus is a data format intended for describing and standardizing experimental data. NeXus provides a specific grammar and syntactic rules via NXDL (NeXus Definition Language) for organizing data within files in addition to a dictionary of well-defined domain-specific field concepts. Since each individual concept is properly documented, it allows communities to agree on terms describing their data. Thus, NeXus acts as a contract on which concepts have to be present and how to name them in a given dataset.

For a more detailed description on the general principles of NeXus we recommend:

What is FAIRmat?

FAIRmat is one of the consortia of the German National Research Data Infrastructure (NFDI). It is tasked with building a FAIR data infrastructure for condensed-matter physics and the chemical physics of solids.

What is NOMAD?

Within FAIRmat, we develop NOMAD: an open source research data management system for making materials science data searchable and publishable. NOMAD hosts a wide variety of datasets from different domains of materials science - including, but not limited to, NeXus data.

What is pynxtools?

pynxtools is our main software tool for end-to-end handling of data from experiments using NeXus. It contains a parser for combining various instrument output formats and electronic lab notebook (ELN) formats into an HDF5 file according to NeXus application definitions. It provides validation against these NeXus definitions and can be used to annotate existing NeXus files with semantic meaning.

pynxtools acts as a Python framework with a built-in API for writing reader plugins that provide specialized reading and conversion functionality for different domains of materials science.

pynxtools can be used for standalone NeXus conversion, but it can also be used as a plugin to NOMAD, extending NOMAD schemas and parsing capabilities with NeXus-specific capabilities.

How can I install pynxtools? How can I contribute?

Does pynxtools require NOMAD or NOMAD OASIS?

No. With the available plugins or community-developed plugins, you can use pynxtools as a standalone tool for converting raw data from experiments to NeXus-compliant files. Therefore, this tool acts as the framework to design instances of data within the NeXus universe. The software can, however, be used as a NOMAD plugin to parse NeXus files, please see the section below for details.

How to use pynxtools with NOMAD

NeXus is supported by the research data management platform NOMAD (as a NOMAD plugin). Experimental data files that are compatible with a NeXus application definition can easily be uploaded to NOMAD and translated into the NOMAD Metainfo data model using pynxtools. Therefore, the data file can be recognized by NOMAD's search system. If you want to learn more about uploading NeXus data to NOMAD, please refer to the NeXus to NOMAD tutorial of this documentation.

To use the pynxtools Python package with NOMAD, simply install it in the same Python environment as the nomad-lab package. NOMAD will recognize pynxtools as a plugin automatically and offer automatic parsing of .nxs files. In addition, NOMAD will automatically transform the NeXus definitions shipped with pynxtools into its own data model called Metainfo. By default, pynxtools is already included in the NOMAD [production]https://nomad-lab.eu/prod/v1/gui/ and staging deployments..

A note on FAIR data

FAIRmat's contribution to the existing NeXus standard, together with the tools provided through pynxtools, enable scientists and research groups working with data, as well as helping communities implement standardized FAIR research data.

You can think of NeXus fulfilling the interoperability and reproducibility part and a research data management platform like NOMAD the findable and accessible part.

We consider pynxtools particularly useful for meeting the following FAIR principle as defined in FAIR Principles: Interpretations and Implementation Considerations: F2-4, I2-I3, and R1.

Where to go next?

We suggest you have a look at one of our tutorials: