nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2024-02-01T04:50:32Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1655Draft: Resolve "Generic xml parser"2024-02-01T04:50:32ZAlvin Noe LadinesDraft: Resolve "Generic xml parser"Closes #1875Closes #1875Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1653New Atomic Descriptors2024-02-05T10:41:34ZJames DarbyNew Atomic DescriptorsHi,
I've updated the atomic descriptors that are computed. The old approach was to store 4 different variations of the SOAP power spectrum, 2 descriptor vectors for each atom in the system and then 2 structure-wise-averaged descriptor ...Hi,
I've updated the atomic descriptors that are computed. The old approach was to store 4 different variations of the SOAP power spectrum, 2 descriptor vectors for each atom in the system and then 2 structure-wise-averaged descriptor vectors. Instead of this, the new approach is to store 2 structure-wise-averaged descriptor vectors only. One of these is an element-agnostic, radially scaled variant of the SOAP power spectrum and the second is the descriptor extracted from the recent MACE-MP-0 foundation model.
Questions/Discussion points:
1. Total storage requirement is now greatly reduced to 296 floats per system. With 13 million structures currently in NOMAD I make this about 16 GB of data. Is this acceptable?
2. Adding the MACE descriptors has introduced new dependencies via [mace-torch](https://github.com/ACEsuit/mace/tree/88d49f9ed6925dec07d1777043a36e1fe4872ff3?tab=readme-ov-file#installation). mace-torch is pip installable so I'm hoping this won't cause any issues. I haven't updated requirement.txt etc. Should I do this?
3. The MACE descriptors are computed using the MACE-MP-0 foundation model. The model file (42 MB) gets automatically downloaded to ~/.cache/mace/ the first time it is called. Think we need to be careful with how this will work with multiple different "workers" running in parallel. Will they try to write to the same file?
4. Loading the MACE model takes much longer than computing the MACE descriptor. Currently the model is loaded once per system, I don't see a great way of avoiding this but worth being aware of.
5. The current schema for SOAP descriptors (shown below) includes entries for the parameters that were used to generate the descriptors e.g. `r_cut`, `n_max` etc. These will be the same for all structures in the database so this seems inefficient? My proposal is to put this information into the quantity definition of the soap_descriptor itself (see image) so the same data isn't duplicated for each structure. Thoughts?
![image](/uploads/c9c58f497f8e66fda43360f74956d74d/image.png)Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1644Draft: Resolve "Add experimental tutorial"2024-01-25T12:49:38ZHampus NaesstroemDraft: Resolve "Add experimental tutorial"Closes #1864Closes #1864Sebastian BruecknerSebastian Bruecknerhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1640Add molpro parser2024-02-16T12:39:11ZNathan DaelmanAdd molpro parserAdd new Molpro parser to NOMAD base.
Extend the supporting schemas.Add new Molpro parser to NOMAD base.
Extend the supporting schemas.Nathan DaelmanNathan Daelmanhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1637documentation - minor, visual fix2024-01-17T15:02:26ZAdrianna Wojasdocumentation - minor, visual fix- updated screenshots
- softer shadows- updated screenshots
- softer shadowsAdrianna WojasAdrianna Wojashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1629Draft: Pynxtools as dependency2024-03-08T09:07:45ZFlorian DobenerDraft: Pynxtools as dependencyThis PR remoevs pynxtools (aka nexus parser) as a submodule and adds it as a python dependency to `pyproject.toml`.This PR remoevs pynxtools (aka nexus parser) as a submodule and adds it as a python dependency to `pyproject.toml`.Florian DobenerFlorian Dobenerhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1627Draft: Resolve "Refine building process of docker image"2024-01-23T09:15:56ZAdam FeketeDraft: Resolve "Refine building process of docker image"This MR suppose to fix multiple issues with the building process of the noamd docker image.
- [x] ~~Fix the python package dependency conflicts~~
- [x] ~~get rid of the circular dependencies of the submodules (closing: https://gitlab.m...This MR suppose to fix multiple issues with the building process of the noamd docker image.
- [x] ~~Fix the python package dependency conflicts~~
- [x] ~~get rid of the circular dependencies of the submodules (closing: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1579, @ahmil , @himanel1 )
solution: Adding the submodules dependency directly to the main pyproject.toml file. This way we do not rely on the submodules' dependencies. This method might change when these submodules are going to be a proper plugins.~~
- [x] ~~fix the issue with the pybis package (@amgo) Solution: remove it from the dependency and install it without its deps later. `pip install --no-deps pybis`~~
- [x] ~~remove unused/probelmatic pyscf package dependency (@ladinesa )~~
- [x] explicitly defining nomad user and using it propertly for the kubernetes' security context (@afekete) solution: using nomad user for both of the docker build stages
- [x] Using develop stage for debugging or as a replacement for the production image (closing: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1598, @mscheidg ) solution: combining the python and node build stage into a single stage to represent better a full dev environment. It can be used for running the test or the full application if needed.
- [x] ~~Pynxtools as dependency (closing: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1629, @dflor )~~
Update: all python dependency related fix has been removed from this MR.
Closes #1848Adam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1594Legacy nomad v0.10.13 patches2023-12-13T15:52:58ZMarkus ScheidgenLegacy nomad v0.10.13 patchesThis is a standing MR used to facilitate patches to the legacy v0 nomad version.This is a standing MR used to facilitate patches to the legacy v0 nomad version.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1578Draft: Resolve "rework TabularParser"2023-12-04T09:55:26ZAmir GolparvarDraft: Resolve "rework TabularParser"Closes #1806Closes #1806Amir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1551Draft: Resolve "Representing notebooks as custom ELN schema"2024-02-22T08:29:54ZAdam FeketeDraft: Resolve "Representing notebooks as custom ELN schema"Closes #1549
Implementing notebook interface as a python schema.
Plan:
- create a separate repo for the plugins (separate the notebook schema from the ai-toolkit) like nomad-app-aitoolkit and nomad-app-notebooks
- add them to dependenci...Closes #1549
Implementing notebook interface as a python schema.
Plan:
- create a separate repo for the plugins (separate the notebook schema from the ai-toolkit) like nomad-app-aitoolkit and nomad-app-notebooks
- add them to dependencies as a submodule (I think this is mandatory for making sure it works.)
- the plugin will not be included into the python package but it will be integrated into the image by some Dockerfile magic
- create a custom nomad.yaml file for our different deployments (prod, staging, dev) which includes all the app configsAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1550Add pyflake and pyupgrade lints to ruff2023-11-23T09:24:29ZAhmed IlyasAdd pyflake and pyupgrade lints to ruff- Adds pyflakes and pyupgrade linting to ruff
- Pyflakes analyzes programs and detects various errors
- Pyupgrade: Modernizes Python code with new features and idioms.
The diff is very big but the changes are very trivial. (A lot of D...- Adds pyflakes and pyupgrade linting to ruff
- Pyflakes analyzes programs and detects various errors
- Pyupgrade: Modernizes Python code with new features and idioms.
The diff is very big but the changes are very trivial. (A lot of Dict -> dict and so on)Ahmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1538Draft: Resolve "Restructure parser plug ins"2024-02-26T11:31:14ZAlvin Noe LadinesDraft: Resolve "Restructure parser plug ins"Closes #1536Closes #1536Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1492Draft: Added a config option in nomad.yaml to support custom user folder name...2024-02-01T09:49:56ZSherjeel ShabihDraft: Added a config option in nomad.yaml to support custom user folder names for Northhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1490Draft: Resolve "Write Docs for Writing Docs"2023-12-22T08:31:56ZHampus NaesstroemDraft: Resolve "Write Docs for Writing Docs"Closes #1720Closes #1720https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1414Draft: Dispersive Material Database2024-01-31T15:28:52ZFlorian DobenerDraft: Dispersive Material DatabaseCloses #1600
Still missing:
- [x] `useMemo` for energy/wavelength conversion
- ~[ ] Selectors for wavelength/energy and refr. index/diel. func~
- [ ] Search app
- [ ] Loading of model name(s) into titleCloses #1600
Still missing:
- [x] `useMemo` for energy/wavelength conversion
- ~[ ] Selectors for wavelength/energy and refr. index/diel. func~
- [ ] Search app
- [ ] Loading of model name(s) into titlehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1400Draft: Semantic web2023-07-11T21:50:01ZMarkus ScheidgenDraft: Semantic webhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1299Draft: Resolve "Bundle format test and documentation"2023-05-21T23:24:29ZTheodore ChangDraft: Resolve "Bundle format test and documentation"Closes #1498Closes #1498Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1111Draft: Add Feature for Descriptive Array Quantity2023-06-12T14:05:35ZHampus NaesstroemDraft: Add Feature for Descriptive Array Quantity### Aims to:
Closes #1352
### Tasks:
- [ ] Write tests
- [ ] Add classes
- [ ] NumericalArray
- [ ] ContextArray
- [ ] Write documentation### Aims to:
Closes #1352
### Tasks:
- [ ] Write tests
- [ ] Add classes
- [ ] NumericalArray
- [ ] ContextArray
- [ ] Write documentationHampus NaesstroemHampus Naesstroem