Skip to content

Development guide

This tutorial will guide you through on how to set up a working environment for developing pynxtools.

What should you should know before this tutorial?

What you will know at the end of this tutorial?

You will know

  • how to setup your environment for developing pynxtools
  • how to make changes to the software
  • how to test the software
  • how to contribute on GitHub
  • how to use pynxtools as a NOMAD plugin

Contributing

Structure of the pynxtools repository

The software tools are located inside src/pynxtools. They are shipped with unit tests located in tests. Some examples from the scientific community are provided in examples. They guide you through the process of converting instrument data into the NeXus standard and visualizing the files' content.

Setup

It is recommended to use python 3.11 with a dedicated virtual environment for this package. Learn how to manage python versions and virtual environments. We recommend using uv, an extremely fast manager Python package and project manager. In this tutorial, you will find paralleled descriptions, using either uv or a more classical approach using venv and pip.

Start by creating a virtual environment:

uv is capable of creating a virtual environment and install the required Python version at the same time.

uv venv --python 3.11

Note that you will need to install the Python version manually beforehand.

python -m venv .venv

That command creates a new virtual environment in a directory called .venv.

Development installation

We start by cloning the repository:

git clone https://github.com/FAIRmat-NFDI/pynxtools.git \\
    --branch master \\
    --recursive pynxtools
cd pynxtools
git submodule sync --recursive
git submodule update --init --recursive --jobs=4

Note that we are using the NeXus definitions as a Git submodule. The last two lines initiate the submodule and upgrade it to match the first used in pynxtools.

Next, we install the package in editable mode (together with its dependencies):

uv pip install -e ".[dev]"

Note that you will need to install the Python version manually beforehand.

pip install --upgrade pip
pip install -e ".[dev]"

Linting and formatting

We are using ruff and mypy for linting, formatting, and type checking. It is recommended to use the pre-commit hook available for ruff which formats the code and checks the linting before actually making an actual Git commit.

Install the precommit by running

pre-commit install

from the root of this repository.

Testing

There exist unit tests for the software written in pytest which can be used as follows:

pytest -sv tests

Editing the documentation

We are using `mkdocs for the documentation. If you edit the documentation, you can build it locally. For this, you need to install an additional set of dependencies:

uv pip install -e ".[docs]"

pip install -e ".[docs]"

You can then serve the documentation by running

mkdocs serve

Running the examples

A number of examples exist which document how the tools can be used. For a standalone usage convenient jupyter notebooks are available for some of the tools. To use these notebooks, jupyter and related tools have to be installed in the development environment as follows:

uv pip install jupyter
uv pip install jupyterlab
uv pip install jupyterlab_h5web

pip install jupyter
pip install jupyterlab
pip install jupyterlab_h5web

Contributing to the package on Github

Once you are happy with the changes, please commit them on a separate branch and create a pull request on GitHub. We run a number of GitHub actions that check the correct linting, run the tests in an isolated environment, and build the documentation. Once these pass and a peer review of the code has occurred, your code will be accepted

Developing pynxtools as a NOMAD plugin

If you plan to contribute to the NOMAD plugin functionality of pynxtools, it often makes sense to use the NOMAD development environment called nomad-distro-dev. You can learn more in the NOMAD documentation.

Troubleshooting

If you face any issues with the tool or when setting up the development environment, please create a new Github Issue.