nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2024-03-08T09:07:45Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1629Draft: Pynxtools as dependency2024-03-08T09:07:45ZFlorian DobenerDraft: Pynxtools as dependencyThis PR remoevs pynxtools (aka nexus parser) as a submodule and adds it as a python dependency to `pyproject.toml`.This PR remoevs pynxtools (aka nexus parser) as a submodule and adds it as a python dependency to `pyproject.toml`.Florian DobenerFlorian Dobenerhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1704Draft: Full refactoring of `run` into `data`2024-03-11T12:30:43ZJose PizarroDraft: Full refactoring of `run` into `data`We need to keep track of the changes in files somewhere.
Closes #1728We need to keep track of the changes in files somewhere.
Closes #1728https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1719Draft: bump dependencies for pandas and scipy2024-03-11T14:04:41ZAhmed IlyasDraft: bump dependencies for pandas and scipyAhmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1663Draft: Resolve "rework elabftw parser"2024-03-15T10:08:24ZAmir GolparvarDraft: Resolve "rework elabftw parser"Closes #1882Closes #1882Amir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1734Resolve "plotly config dictionary can't be fed in from normalize function"2024-03-15T10:59:52ZDinga WonankeResolve "plotly config dictionary can't be fed in from normalize function"We created a new function create_topology_porosity, which will enable users to easily populate the topology.
In addition, we moved most of the global functions from mof_deconstructor into atomutils, which will facilitate the implementat...We created a new function create_topology_porosity, which will enable users to easily populate the topology.
In addition, we moved most of the global functions from mof_deconstructor into atomutils, which will facilitate the implementation of the wrapping function in the future.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1740Draft: Resolve "pyiron integration"2024-03-18T08:51:38ZAmir GolparvarDraft: Resolve "pyiron integration"Closes #1872Closes #1872Amir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1745Draft: Ruff 2024 style guide2024-03-20T12:44:47ZAhmed IlyasDraft: Ruff 2024 style guideAhmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1746Draft: Resolve "Metainfo improvements"2024-03-20T16:05:28ZMarkus ScheidgenDraft: Resolve "Metainfo improvements"Closes #1947Closes #1947https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1726Integrity check2024-03-20T23:12:44ZTheodore ChangIntegrity checkTheodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1706Draft: Resolve "automatic creation and resolution of hdf5 references for larg...2024-03-22T09:48:10ZAlvin Noe LadinesDraft: Resolve "automatic creation and resolution of hdf5 references for large arrays"Closes #1903 #1789Closes #1903 #1789Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1752Draft: Fixing issue in nomad elastic search and ensure creation of the nexus ...2024-03-26T08:51:30ZRubel MozumderDraft: Fixing issue in nomad elastic search and ensure creation of the nexus subsection in nexus metainfo.Closes #1934Closes #1934https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1753Resolve "Booleans are not displayed on overview page"2024-03-27T08:42:06ZAhmed IlyasResolve "Booleans are not displayed on overview page"Closes #1956
![image](/uploads/5f6fe6f89425ac2e6484c3f17121e93c/image.png)Closes #1956
![image](/uploads/5f6fe6f89425ac2e6484c3f17121e93c/image.png)Ahmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1731Draft: Resolve "Parsing nexus files generates a lot of warnings and errors"2024-03-28T07:45:32ZMarkus ScheidgenDraft: Resolve "Parsing nexus files generates a lot of warnings and errors"Fixed some problems of #1934
- I fixed a bug in the attribute generation. Basically some attributes were missing. But this only solved a small portion of the problems.
- I fixed a bug in the MetainforNormalizer which tried to call thin...Fixed some problems of #1934
- I fixed a bug in the attribute generation. Basically some attributes were missing. But this only solved a small portion of the problems.
- I fixed a bug in the MetainforNormalizer which tried to call things called "normalize" in the schema, but where not actually normalize functions.
- I fixed the error handling a bit to better see what is actually causing the problems.
### The first error type mentioned in #1934
This is fixed with the MetainforNormalizer fix.
### The second error type mentioned in #1934
These are a few separate issues repeating. They are reported while setting an attribute, but they are caused because the parser could not find the quantity or section to put the attribute on. Therefore, they do not seem to be attribute problems. They are indicating a few mapping problems that I cannot resolve!
- string `unit` in `nxdl_path`, where the parser only expects instances of `ET.Element`. This maybe is a problem in `pynxtools.nexus.HandleNexus` which produces all the `nxdl_path`.
```
WARNING nomad.client 2024-03-14T13:46:17 Error while setting attribute.
- exception: Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/parsing/nexus/nexus.py", line 167, in _populate_data
assert isinstance(
AssertionError: nx_path is not a list of ET.Element, one segment is "units:<class 'str'>"
```
- The parsre tries to set `version__attribute` to `nexus.NXentry.COLLECTION:SubSection`, but the nexus mapping does not define attributes on sub sections, I guess this is supposed to go to the section that the sub-section points to?
```
WARNING nomad.client 2024-03-14T13:46:17 Error while setting attribute.
- exception: Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/parsing/nexus/nexus.py", line 194, in _populate_data
assert isinstance(
AssertionError: nexus.NXentry.COLLECTION:SubSection is not a Quantity
```
- `nexus.NXelectronanalyser.NXtransformations` has no suitable quantity for `depends_on__field`
```
WARNING nomad.client 2024-03-14T13:46:18 Error while setting attribute.
- exception: Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/parsing/nexus/nexus.py", line 182, in _populate_data
assert metainfo_def, f'{current.m_def} has no suitable property for {parent_field_name}'
AssertionError: nexus.NXelectronanalyser.NXtransformations:Section has no suitable property for depends_on__field
```
- `nexus.NXelectronanalyser.NXtransformations` has no suitable quantity for `transformation_type__field`
```
WARNING nomad.client 2024-03-14T13:46:18 Error while setting attribute.
- exception: Traceback (most recent call last):
File "/Users/markus/Documents/Projects/nomad-fairdi/nomad/parsing/nexus/nexus.py", line 182, in _populate_data
assert metainfo_def, f'{current.m_def} has no suitable property for {parent_field_name}'
AssertionError: nexus.NXelectronanalyser.NXtransformations:Section has no suitable property for transformation_type__field
```
Then there are a few less frequent things:
- The value "HHG laser" is not an enum value for `nexus.NXsource.type__field`.
- NaN value is not set for field on `size__field[size__field]`
# The third error mentioned in #1934
These errors are caused because we try to interpret enum values as floats. This is all for this `nexus.NXtransformations.AXISNAME__field`, which is defined as a float in the schema. I guess it should not be a float?https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1681Draft: Resolve "derived display unit system"2024-03-28T09:58:12ZMohammad NakhaeeDraft: Resolve "derived display unit system"Closes #1898Closes #1898Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1760Check upload groups2024-03-28T15:54:36ZSascha KlawohnCheck upload groupsThis adds checking the existence of a group before adding it to an upload and tests it.
Also fixes a flaky test by adding "yet (another) test user" who is not a writer of the upload.This adds checking the existence of a group before adding it to an upload and tests it.
Also fixes a flaky test by adding "yet (another) test user" who is not a writer of the upload.Sascha KlawohnSascha Klawohnhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1757Draft: Resolve "Consistent labels in archive browser and ELNs"2024-03-28T16:09:24ZMohammad NakhaeeDraft: Resolve "Consistent labels in archive browser and ELNs"Closes #1915Closes #1915Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1666Draft: Resolve "Update developing plugin documentation"2024-03-28T16:39:08ZJoseph RudzinskiDraft: Resolve "Update developing plugin documentation"Closes #1887Closes #1887Joseph RudzinskiJoseph Rudzinskihttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1756Draft: Resolve "Generic parser for H5 files"2024-03-28T17:40:59ZAmir GolparvarDraft: Resolve "Generic parser for H5 files"Closes #1873Closes #1873Amir GolparvarAmir Golparvar