2.1.2.3.1. Rules for Structuring Information in NeXus Files¶
All NeXus files contain one or many groups of type NXentry
at root level.
Many files contain only one
NXentry
group, then the name is entry
.
The NXentry level of hierarchy is there to support the storage of multiple related experiments in one file.
Or to allow the NeXus file to serve as a container for storing a whole scientific workflow from data acquisition to
publication ready data.
Also, NXentry
class
groups can contain raw data or processed data.
For files with more than one NXentry
group, since HDF requires
that no two items at the same level in an HDF file may have the same name,
the NeXus fashion is to
assign names with an incrementing index appended, such as
entry1
, entry2
, entry3
, etc.
In order to illustrate what is written in the text, example hierarchies like the one in figure Raw Data are provided.
Content of a Raw Data NXentry
Group¶
An example raw data hierarchy is
shown in figure Raw Data
(only showing the relevant parts of the data hierarchy).
In the example shown, the data
field in the NXdata
group
is linked to the 2-D detector data (a 512x512 array of 32-bit integers).
The attribute signal = data
on the NXdata group marks this
field as the default plottable data of the data:NXdata
group.
The NXdata group attribute axes = . .
declares that both dimensions of
the data
field do not have associated dimension scales (plotting
routines should use integer scaling for each axis).
Note that [,]
represents a 2D array.
NeXus Raw Data Hierarchy
1 entry:NXentry 2 @default = data 3 instrument:NXinstrument 4 source:NXsource 5 .... 6 detector:NXdetector 7 data:NX_INT32[512,512] 8 sample:NXsample 9 control:NXmonitor 10 data:NXdata 11 @signal = data 12 @axes = [".", "."] 13 data --> /entry/instrument/detector/data
An NXentry
describing raw data contains at least a NXsample
,
one NXmonitor
,
one NXdata
and a NXinstrument
group.
It is good practice to use the names
sample
for the NXsample
group,
control
for the NXmonitor
group holding the
experiment controlling monitor and
instrument
for the NXinstrument
group.
The NXinstrument
group contains further groups describing the individual
components of the instrument as appropriate.
The NXdata
group contains links to all those data items in the NXentry
hierarchy
which are required to put up a default plot of the data.
As an example consider a SAXS instrument with a 2D detector.
The NXdata
will then hold a link to the detector image.
If there is only one NXdata
group,
it is good practice to name it data
.
Otherwise, the name of the detector bank represented is a good selection.
Content of a processed data NXentry
group¶
Processed data, see figure Processed Data,
in this context means the results of a data reduction or
data analysis program. Note that []
represents a 1D array.
NeXus Processed Data Hierarchy
1 entry:NXentry 2 @default = data 3 reduction:NXprocess 4 program_name = "pyDataProc2010" 5 version = "1.0a" 6 input:NXparameters 7 filename = "sn2013287.nxs" 8 sample:NXsample 9 data:NXdata 10 @signal = data 11 @axes = "." 12 data
NeXus stores such data in a simplified
NXentry
structure. A processed data NXentry
has at minimum a NXsample
,
a NXdata
and a NXprocess
group.
Again the preferred name for the NXsample
group is sample
.
In the case of processed data, the NXdata
group holds the
result of the processing together with the associated axis data.
The NXprocess
group holds the name and version of the program used for this processing
step and further NXparameters
groups. These groups ought to contain the
parameters used for this data processing step in suitable detail so that
the processing step can be reproduced.
Optionally a processed data NXentry
can hold a NXinstrument
group with
further groups holding relevant information about the instrument. The
preferred name is again instrument
. Whereas for a raw data file, NeXus
strives to capture as much data as possible, a NXinstrument
group for
processed data may contain a much-reduced subset.
NXsubentry
or Multi-Method Data¶
Especially at synchrotron facilities, there are experiments which perform several different methods
on the sample at the same time. For example, combine a powder diffraction experiment with XAS.
This may happen in the same scan, so the data needs to be grouped together. A suitable NXentry
would need to adhere to two different application definitions. This leads to name clashes which cannot be
resolved easily. In order to solve this issue, the following scheme was implemented in NeXus:
The complete beamline (all data) is stored in an appropriate hierarchy in an
NXentry
.The
NXentry
group contains furtherNXsubentry
groups, one for each method.Each NXsubentry group is constructed like a
NXentry
group. It contains links to all those data items required to fulfill the application definition for the particular. method it represents.The name of the application definition is stored in the
definition
field of the NXsubentry groupEach
NXsubentry
group contains aNXdata
group describing the default plottable data for that experimental method. To satisfy the NeXus requirement of finding the default plottable data from aNXentry
group, theNXdata
group from one of theseNXsubentry
groups (the fluoresence data) was linked.
See figure NeXus Multi Method Hierarchy for an example hierarchy.
Note that [,]
represents a 2D array.
NeXus Multi Method Hierarchy
1 entry:NXentry 2 @default = data 3 user:NXuser 4 sample:NXsample 5 instrument:NXinstument 6 SASdet:NXdetector 7 data:[,] 8 fluordet:NXdetector 9 data:[,] 10 large_area:NXdetector 11 data:[,] 12 SAS:NXsubentry 13 definition = "NXsas" 14 instrument:NXinstrument 15 detector:NXdetector 16 data --> /entry/instrument/SASdet/data 17 data:NXdata 18 data --> /entry/instrument/SASdet/data 19 Fluo:NXsubentry 20 definition = "NXfluo" 21 instrument:NXinstrument 22 detector --> /entry/instrument/fluordet/data 23 detector2 --> /entry/instrument/large_area/data 24 data:NXdata 25 @signal = detector 26 @axes = [".", "."] 27 detector --> /entry/instrument/fluordet/data 28 data:NXdata --> /entry/Fluo/data
Rules for Special Cases¶
Scans¶
Scans are difficult to capture because they have great variety. Basically,
any variable can be scanned. Such behaviour cannot be captured in application definitions.
Therefore NeXus solves this difficulty with a set of
rules. In this section, NP
is used as a symbol for the number of scan
points.
The scan dimension
NP
is always the first dimension of any multi-dimensional dataset. The reason for this is that HDF allows the first dimension of a dataset to be unlimited. Which means, that data can be appended to the dataset during the scan.All data is stored as arrays of dimensions
NP
, original dimensions of the data at the appropriate position in theNXentry
hierarchy.The
NXdata
group has to contain links to all variables varied during the scan and the detector data. Thus theNXdata
group mimics the usual tabular representation of a scan.The
NXdata
group has attributes to enable the default plotting, as described in the section titled NXdata Facilitates Automatic Plotting.
Simple scan¶
Examples may be in order here. Let us start with a simple case, the sample is rotated around its rotation axis and data is collected in a single point detector. See figure Simple Scan for an overview. Then we have:
A dataset at
NXentry/NXinstrument/NXdetector/data
of lengthNP
containing the count data.A dataset at
NXentry/NXsample/rotation_angle
of lengthNP
containing the positions ofrotation_angle
at the various steps of the scan.
NXdata
contains links to:
NXentry/NXinstrument/NXdetector/data
NXentry/NXsample/rotation_angle
All other fields have their normal dimensions.
NeXus Simple Scan Example
1 entry:NXentry 2 @default = data 3 instrument:NXinstrument 4 detector:NXdetector 5 data[NP] 6 sample:NXsample 7 rotation_angle[NP] 8 control:NXmonitor 9 data[NP] 10 data:NXdata 11 @signal = "data" 12 @axes = "rotation_angle" 13 @rotation_angle_indices = 0 14 data --> /entry/instrument/detector/data 15 rotation_angle --> /entry/sample/rotation_angle
Simple scan with area detector¶
The next example is the same scan but with an area detector with xsize
times ysize
pixels. The only thing which changes is that
/NXentry/NXinstrument/NXdetector/data
will have the dimensions
NP, xsize, ysize
. See figure Simple Scan with Area Detector for an overview.
NeXus Simple Scan Example with Area Detector
1 entry:NXentry 2 instrument:NXinstrument 3 detector:NXdetector 4 data:[NP,xsize,ysize] 5 sample:NXsample 6 rotation_angle[NP] 7 control:NXmonitor 8 data[NP] 9 data:NXdata 10 @signal = "data" 11 @axes = ["rotation_angle", ".", "."] 12 @rotation_angle_indices = 0 13 data --> /entry/instrument/detector/data 14 rotation_angle --> /entry/sample/rotation_angle
The NXdata
group attribute axes = rotation_angle . .
declares that only the first
dimension of the plottable data
has a dimension scale (by name, rotation_angle
).
The other two dimensions have no associated dimension scales and should be plotted against
integer bin numbers.
Complex hkl scan¶
The next example involves a complex movement along the \(h\) axis in reciprocal space which requires mutiple motors of a four-circle diffractometer to be varied during the scan. We then have:
A dataset at
NXentry/NXinstrument/NXdetector/data
of lengthNP
containing the count data.A dataset at
NXentry/NXinstrument/NXdetector/polar_angle
of lengthNP
containing the positions of the detector’s polar_angle at the various steps of the scan.A dataset at
NXentry/NXsample/rotation_angle
of lengthNP
containing the positions ofrotation_angle
at the various steps of the scan.A dataset at
NXentry/NXsample/chi
of lengthNP
containing the positions of chi at the various steps of the scan.A dataset at
NXentry/NXsample/phi
of lengthNP
containing the positions of phi at the various steps of the scan.A dataset at
NXentry/NXsample/h
of lengthNP
containing the positions of the reciprocal coordinate \(h\) at the various steps of the scan.A dataset at
NXentry/NXsample/k
of lengthNP
containing the positions of the reciprocal coordinate \(k\) at the various steps of the scan.A dataset at
NXentry/NXsample/l
of lengthNP
containing the positions of the reciprocal coordinate \(l\) at the various steps of the scan.NXdata
contains links to:NXentry/NXinstrument/NXdetector/data
NXentry/NXinstrument/NXdetector/polar_angle
NXentry/NXsample/rotation_angle
NXentry/NXsample/chi
NXentry/NXsample/phi
NXentry/NXsample/h
NXentry/NXsample/k
NXentry/NXsample/l
The
NXdata
also contains appropriate attributes as described in Associating plottable data using attributes applied to the NXdata group.All other fields have their normal dimensions.
NeXus Complex hkl Scan
1 entry:NXentry
2 @default = data
3 instrument:NXinstrument
4 detector:NXdetector
5 data[NP]
6 polar_angle[NP]
7 name
8 sample:NXsample
9 name
10 rotation_angle[NP]
11 chi[NP]
12 phi[NP]
13 h[NP]
14 k[NP]
15 l[NP]
16 control:NXmonitor
17 data[NP]
18 data:NXdata
19 @signal = data
20 @axes = "h"
21 @h_indices = 0
22 @k_indices = 0
23 @l_indices = 0
24 @chi_indices = 0
25 @phi_indices = 0
26 @polar_angle_indices = 0
27 @rotation_angle_indices = 0
28 data --> /entry/instrument/detector/data
29 rotation_angle --> /entry/sample/rotation_angle
30 chi --> /entry/sample/chi
31 phi --> /entry/sample/phi
32 polar_angle --> /entry/instrument/detector/polar_angle
33 h --> /entry/sample/h
34 k --> /entry/sample/k
35 l --> /entry/sample/l
Multi-parameter scan: XAS¶
Data can be stored almost anywhere in the NeXus tree. While the previous examples
showed data arrays in either NXdetector
or NXsample
,
this example demonstrates that data can be stored in other places. Links are used
to reference the data.
The example is for X-ray Absorption Spectroscopy (XAS) data where the monochromator
energy is step-scanned and counts are read back from detectors before (I0
)
and after (I
) the sample. These energy scans are repeated at a sequence
of sample temperatures to map out, for example, a phase transition. While it is customary
in XAS to plot log(I0/I), we show them separately here
in two different NXdata
groups to demonstrate that such things are possible.
Note that the length of the 1-D energy array is NE
while
the length of the 1-D temperature array is NT
NeXus Multi-parameter scan: XAS
1 entry:NXentry 2 @default = "I_data" 3 instrument:NXinstrument 4 I:NXdetector 5 data:NX_NUMBER[NE,NT] 6 energy --> /entry/monochromator/energy 7 temperature --> /entry/sample/temperature 8 I0:NXdetector 9 data:NX_NUMBER[NE,NT] 10 energy --> /entry/monochromator/energy 11 temperature --> /entry/sample/temperature 12 sample:NXsample 13 temperature:NX_NUMBER[NT] 14 monochromator:NXmonochromator 15 energy:NX_NUMBER[NE] 16 I_data:NXdata 17 @signal = "data" 18 @axes = ["energy", "temperature"] 19 @energy_indices = 0 20 @temperature_indices = 0 21 data --> /entry/instrument/I/data 22 energy --> /entry/monochromator/energy 23 temperature --> /entry/sample/temperature 24 I0_data:NXdata 25 @signal = data 26 @axes = ["energy", "temperature"] 27 @energy_indices = 0 28 @temperature_indices = 0 29 data --> /entry/instrument/I00/data 30 energy --> /entry/monochromator/energy 31 temperature --> /entry/sample/temperature
Rastering¶
Rastering is the process of making experiments at various locations in the
sample volume. Again, rasterisation experiments can be variable. Some people
even raster on spirals! Rasterisation experiments are treated the same way as
described above for scans. Just replace NP
with
P
, the number of raster points.
Special rules apply if a rasterisation happens on a regular grid of size
xraster, yraster
. Then the variables varied in the rasterisation will be
of dimensions xraster, yraster
and the detector data of dimensions
xraster, yraster, (orginal dimensions)
of the detector. For example, an area detector of
size xsize, ysize
then it is stored with
dimensions xraster, yraster, xsize, ysize
.
Warning
Be warned: if you use the 2D rasterisation method with xraster, yraster
you may end up with
invalid data if the scan is aborted prematurely. This cannot happen if the first method is used.
Streaming Data Acquisition And Logging¶
More and more data is collected in streaming mode. This means that time stamped data is logged for one or more inputs, possibly together with detector data. Another use case is the logging of parameters, for example temperature, while a long running data collection is in progress. NeXus covers this case too. There is one simple rule for structuring such files:
Just use the standard NeXus raw data file structure, but replace the corresponding data object with an NXlog or NXevent_data structure of the same name.
For example, consider your instrument is streaming detector images against a magnetic_field on the sample. In this case both NXsample/magnetic_field and NXdetector/data would become NXlog structures instead of simple arrays i.e. the NXlog structure will have the same name as the NeXus field involved.
NXcollection¶
On demand from the community, NeXus introduced a more informal method of
storing information in a NeXus file. This is the NXcollection
class which can appear anywhere underneath NXentry
.
NXcollection
is a container for holding other data.
The foreseen use is to document collections of similar data which do not
otherwise fit easily into the NXinstrument
or NXsample
hierarchy, such as the intent to record
all motor positions on a synchrotron beamline.
Thus, NXcollection
serves as a quick point of access
to data for an instrument scientist or another expert. NXcollection is
also a feature for those who are too lazy to build up the complete NeXus
hierarchy. An example usage case is documented in figure
NXcollection example.
NXcollection
Example1 entry:NXentry 2 positioners:NXcollection 3 mxx:NXpositioner 4 mzz:NXpositioner 5 sgu:NXpositioner 6 ttv:NXpositioner 7 hugo:NXpositioner 8 .... 9 scalars:NXcollection 10 title (dataset) 11 lieselotte (dataset) 12 ... 13 detectors:NXcollection 14 Pilatus:NXdata 15 MXX-45:NXdata 16 ....