Unittests vs. global tests

Although some testing can be done globally across all the codes, like checking that the program_name is given, there are many things that cannot be properly tested with global tests. Each code supports very different calculation modes (single point, DOS, MD, optimization, vibration, etc.), different input and output formats, can output different things, and so on. This is why each parser should be tested individually to see that the parser can handle the unique functionality provided by the software.

Local backend for easier testing

To have a convenient and fast way for parser developers to test the parsers, it makes sense to capture the metainfo into a custom native python backend, instead of writing the results into say a JSON dictionary with JsonParseEventsWriterBackend (nomadcore/parser_backend.py).

This kind of a native python backed has been implemented, and can be found as the class LocalBackend in the file (nomadcore/local_backend.py). It is a drop-in replacement for the JsonParseEventsWriterBackend that is used by default, but also supports some custom features that make testing easier. To start using this backend, you must provide it as the argument super_backend to the mainFunction that can be found in nomadcore/simple_parser.py. The other alternative is to derive your parser directly from the abstract base class ParserInterface, which can be found in nomadcore/baseclasses. Using this class, you can specify the used backend directly in the object constructor.

In the constructor of LocalBackend-object you can specify if checking the input values against their metainfo-definition are turned on. These tests specifically check the input types, and shapes so that they match the metainfo definition. This option is controlled by the boolean argument debug.

The results in LocalBackend are gathered to the attribute results, which is a dictionary-like object that provides an easy access to the metainfos. Once the parsing is done, you can query these results from this object. Here is an example of this:

xc = results["XC_functional"]  # Provides a flattened list of the values of one metainfo. If only one value is present, the only item in the list of provided by default.
sccs = results["section_single_configuration_calculation"]  # When querying for a section, all the found sections are returned, and they contain all the subsections and values that can be accessed with dictionary-like syntax
for scc in sccs:
    print(scc["energy_total"])

The interface of this Results-class is very similar to the Archive-class found in (nomadcore/archive.py). There are however some differences, that can be easily resolved if necessary. This would be nice, because then one could write tests that work both on the LocalBackend object, that is directly stored by a parser, and the Archive-objects that can be created from a HDF5 or JSON file already stored in the NOMAD Archive.

Examples of using LocalBackend to implement unittesting can be found from multiple parser, e.g. CP2K, CPMD, VASP, NWChem, BigDFT. The tests can be found in the file regtests/regtests.py that is present in the main repository of each of these parsers.