nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2024-03-12T15:00:11Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1723temporary fix for removing github nomad from requirements.txt2024-03-12T15:00:11ZAhmed Ilyastemporary fix for removing github nomad from requirements.txtAhmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1722Added new troubleshooting section to OASIS installation.2024-03-12T12:21:58ZLauri HimanenAdded new troubleshooting section to OASIS installation.Closes #1927Closes #1927Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1721Remove cache2024-03-21T15:05:25ZTheodore ChangRemove cacheTheodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1720Resolve "data structure of the .eln file format has changed"2024-03-12T12:37:34ZAmir GolparvarResolve "data structure of the .eln file format has changed"Closes #1928Closes #1928Amir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1719Draft: bump dependencies for pandas and scipy2024-03-11T14:04:41ZAhmed IlyasDraft: bump dependencies for pandas and scipyAhmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1718Documentation: Tiny improvement and typo fix2024-03-11T13:14:18ZFabian SchöppachDocumentation: Tiny improvement and typo fixhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1717Move pynxtools and plugins into parsing packages2024-03-27T17:30:38ZFlorian DobenerMove pynxtools and plugins into parsing packagesThis adds pynxtools and respective plugins to the `parsing` section of `pyproject.toml` and regenerates the requirements files with it. It also removes the nexus submodule as this is no longer needed.
This is a _hotfix_ to have the newe...This adds pynxtools and respective plugins to the `parsing` section of `pyproject.toml` and regenerates the requirements files with it. It also removes the nexus submodule as this is no longer needed.
This is a _hotfix_ to have the newest examples ready for DPG and proper pluginisation of pynxtools will be handled in !1629.
As discussed with @lukaspie @sanbrock.
ToDo:
- [x] Update `pynxtools==0.1.0`Florian DobenerFlorian Dobenerhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1716Resolve "Config refactor"2024-03-14T10:19:46ZLauri HimanenResolve "Config refactor"Closes #1917Closes #1917Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1715Resolve "Include custom quantities from the Perovskite Database schema"2024-03-13T09:09:02ZJose Marquez PrietoResolve "Include custom quantities from the Perovskite Database schema"Closes #1926Closes #1926Jose Marquez PrietoJose Marquez Prietohttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1714Fixed gui data etag.2024-03-11T20:10:22ZMarkus ScheidgenFixed gui data etag.Changelog: FixedChangelog: Fixedhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1713Fixes from review comments2024-03-11T08:07:21ZAlvin Noe LadinesFixes from review commentsSome fixes to address @jrudz review.Some fixes to address @jrudz review.Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1712copy local path to temp directory2024-03-07T12:54:20ZAhmed Ilyascopy local path to temp directoryIf a local path is provided, copy it to a temp dir before importing. Otherwise when the upload is deleted, the original local path file ends up getting deleted too.If a local path is provided, copy it to a temp dir before importing. Otherwise when the upload is deleted, the original local path file ends up getting deleted too.Ahmed IlyasAhmed Ilyashttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1711Add raw_path_exists method to ClientContext2024-03-06T14:23:57ZSarthak KapoorAdd raw_path_exists method to ClientContext`nomad.datamodel.Context` has a `raw_path_exists` method which needs to implemented in it's child classes. But this implementation is missing in one of the child classes `ClientContext`. The current MR fixes it.
_Why is this needed?_ Wh...`nomad.datamodel.Context` has a `raw_path_exists` method which needs to implemented in it's child classes. But this implementation is missing in one of the child classes `ClientContext`. The current MR fixes it.
_Why is this needed?_ When testing a schema class locally using pytest, the context associated with the archive is of class `ClientContext`. If the schema uses `archive.m_context.raw_path_exists(filepath)` in normalize, the test fails as the method is not implemented.Sarthak KapoorSarthak Kapoorhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1710Improved app and helm chart with respect to timeouts and rate limiting.2024-03-11T20:49:57ZMarkus ScheidgenImproved app and helm chart with respect to timeouts and rate limiting.This partially helps with: #1914, it includes the changes of !1701
Changes:
- Timeouts are now consistently applied to ingress and proxy rules based on shared values
- Separate ingress for api and others (gui, docs) for tighter rate lim...This partially helps with: #1914, it includes the changes of !1701
Changes:
- Timeouts are now consistently applied to ingress and proxy rules based on shared values
- Separate ingress for api and others (gui, docs) for tighter rate limits at the api
- Concurrent connections limit in addition to connection per second limit
- ArchiveQuery defaults fit the timeout and rate limiting settings
- Increased the HPC cloud loadbalancer timeouts to be slightly longer than the nomad timeouts (not this MR)
- Removed the joblib based threading for multi entry archive apis. This was a noop due to GIL.
- Added an await call into the multi entry archive loop, allowing requests (e.g. probes) during a running multi entry archive call.
- Multi entry archive apis stop computing the requested archive list after a client disconnect.
- refactored the main app, because HTTP middlewares are prohibiting recognising client disconnects (https://github.com/encode/starlette/discussions/2094). Now the api does not use any HTTP middleware
- more consistent use of parameter free events in api logging
Solutions:
- The app now does stop when a request is canceled (e.g. via timeout).
- Timeouts are a consistent 60s and the rate limit is set to 10 concurrent api requests and 32 requests per second.
- The long running multi entry archive api calls allow concurrent requests. This already worked for all downloads via the used StreamingResponses.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1709updated submodules and gui artifacts2024-03-05T14:30:33ZJoseph Rudzinskiupdated submodules and gui artifactsCloses #1921Closes #1921Joseph RudzinskiJoseph Rudzinskihttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1708Resolve "Reloading link bugs out"2024-03-07T10:07:30ZLauri HimanenResolve "Reloading link bugs out"Closes #1918Closes #1918Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1707Resolve "CP2K parser failure"2024-03-05T11:52:22ZNathan DaelmanResolve "CP2K parser failure"- Update `CP2K` parser with header patch
- Update `Phonopy` parser with non-canonical hexagonal cell patch
Closes #1919- Update `CP2K` parser with header patch
- Update `Phonopy` parser with non-canonical hexagonal cell patch
Closes #1919Nathan DaelmanNathan Daelmanhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1706Draft: Resolve "automatic creation and resolution of hdf5 references for larg...2024-03-22T09:48:10ZAlvin Noe LadinesDraft: Resolve "automatic creation and resolution of hdf5 references for large arrays"Closes #1903 #1789Closes #1903 #1789Alvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1705Draft: Resolve "CP2K parser failure"2024-03-05T11:19:33ZNathan DaelmanDraft: Resolve "CP2K parser failure"- Update `CP2K` parser with header patch
- Update `Phonopy` parser with non-canonical hexagonal cell patch
Closes #1919- Update `CP2K` parser with header patch
- Update `Phonopy` parser with non-canonical hexagonal cell patch
Closes #1919Nathan DaelmanNathan Daelmanhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/1704Draft: Full refactoring of `run` into `data`2024-03-11T12:30:43ZJose PizarroDraft: Full refactoring of `run` into `data`We need to keep track of the changes in files somewhere.
Closes #1728We need to keep track of the changes in files somewhere.
Closes #1728