nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2022-10-31T06:46:57Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/912Added update parameter to useEntryPageContext, simplified Structure component...2022-10-31T06:46:57ZLauri HimanenAdded update parameter to useEntryPageContext, simplified Structure component logic a bit.Closes #1128Closes #1128Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/911Updated craco.2022-10-27T10:06:50ZLauri HimanenUpdated craco.Closes #1147Closes #1147Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/910Resolve "Overview customization"2022-11-17T08:21:44ZLauri HimanenResolve "Overview customization"Closes #1139Closes #1139Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/909Making hysprint the main branch for hysprint eln again2022-10-27T10:28:20ZMichael GötteMaking hysprint the main branch for hysprint eln againMichael GötteMichael Göttehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/908Resolve "Add MSD plot to overview page for MD"2022-10-27T12:32:45ZJoseph RudzinskiResolve "Add MSD plot to overview page for MD"Closes #1144Closes #1144Joseph RudzinskiJoseph Rudzinskihttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/907Resolve "Python defined sections cannot be used as quantity types in yaml sch...2022-10-26T12:59:46ZMarkus ScheidgenResolve "Python defined sections cannot be used as quantity types in yaml schemas"Closes #1140Closes #1140https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/905Resolve "Arbitrary quantities in ES"2022-12-02T14:17:16ZMarkus ScheidgenResolve "Arbitrary quantities in ES"Closes #1097
![image](/uploads/deb8dd6b09485e8ff4172c1b156a4fa5/image.png)
This is the implementation so far:
- all quantities below the `data` section that are string, float, int, date, or enums and have a scalar shape are add to and...Closes #1097
![image](/uploads/deb8dd6b09485e8ff4172c1b156a4fa5/image.png)
This is the implementation so far:
- all quantities below the `data` section that are string, float, int, date, or enums and have a scalar shape are add to and indexed in `metadata.searchable_quantities`
- There is a special filter menu for those "custom" quantities.
- In this menu you can create a long AND of criteria. This could be extended to support ORs and NOTs.
- For each criteria, you can select from all the available quantities found in NOMAD's data. Currently this is a select and should be extended into an autocomplete. The quantities are select by their "path", i.e. position in the data
- For each criteria there might be operators (<,>, >=, <=) to select from. Operators depend on the selected quantity and its type.
- For each criteria there is a value to enter. We reuse the edit quantity components here. It might look different based on the quantity type.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/904Use __future__ module2022-10-28T14:23:18ZTheodore ChangUse __future__ moduleTheodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/903Changed PublictaionReference base section to retrieve all authors from the DOI.2022-10-24T21:42:34ZJose Marquez PrietoChanged PublictaionReference base section to retrieve all authors from the DOI.Changed PublictionReference base section to retrieve all authors from the DOi instead of only the first one.
Closes #1142Changed PublictionReference base section to retrieve all authors from the DOi instead of only the first one.
Closes #1142Jose Marquez PrietoJose Marquez Prietohttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/901Resolve "Search does not work for some metadata fields"2022-10-21T22:30:03ZMarkus ScheidgenResolve "Search does not work for some metadata fields"Closes #1129Closes #1129https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/900First version of Solar Cell app config2022-10-25T10:31:42ZJose Marquez PrietoFirst version of Solar Cell app configCloses #1114
I added a Solar cell explore entries views. At the moment it changes the search filters panel and the Results table. It has some formatting added o it and I think this is already valuable. Future steps in fixing pre-define...Closes #1114
I added a Solar cell explore entries views. At the moment it changes the search filters panel and the Results table. It has some formatting added o it and I think this is already valuable. Future steps in fixing pre-defined widgets and plots could be added sequentially.
Maybe @himanel1 could have a quick look and then discuss with @mscheidg if you want to have this merged now or at a later stage.Jose Marquez PrietoJose Marquez Prietohttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/898Fix some version problems2022-10-21T12:15:46ZDavid SikterFix some version problemsFixes some version conflicts in the main requirements.txtFixes some version conflicts in the main requirements.txtDavid SikterDavid Sikterhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/897Update parser dependencies2022-10-21T07:40:25ZAlvin Noe LadinesUpdate parser dependenciesSync parser dependecies with nomadSync parser dependecies with nomadAlvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/895Resolve "Deployment is broken due to celery update"2022-10-20T10:35:49ZMarkus ScheidgenResolve "Deployment is broken due to celery update"Closes #1136Closes #1136https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/894820 - Oasislink from local to central as TCP server2023-07-05T11:08:40ZDaniel Lehmberg820 - Oasislink from local to central as TCP server@mscheidg
* logtransfer and federation FastAPI backend mostly finished and tested (see open points below)
* rudimentary class for additional statistics (queries to mongodb) that can be transferred
- currently this is located in ``lo...@mscheidg
* logtransfer and federation FastAPI backend mostly finished and tested (see open points below)
* rudimentary class for additional statistics (queries to mongodb) that can be transferred
- currently this is located in ``logtransfer.py`` (it shouldn't be within the main Nomad API as this can run with multiple workers, which turned out to be problematic in the past)
- we agreed that we use reasonable cases from the KPI as well as statistics required for the homepage
* Check documentation for data sharing in oasis.md
* The test case that sometimes used failed for remote CI pipeline is (hopefully) fixed
---
Relates to issues #820 and #886
This merge request follows up the closed merge requests !697, !727 and !784.
Here the logtransfer is implemented in a separate service to the Nomad app - a TCP Server (Python native classes). In particular the service runs on a local Nomad Oasis and receives logs on the same address and port than where logstash logs are sent on the central Oasis.
For Nomad Oasis the server running on logtransfer can therefore be interpreted as a logstash proxy. Unless running the logstash instance itself, all logs are submitted to the central Oasis where the logs are then eventually stored in logstash.
Open questions:
* [x] To identify a local Oasis we read the IP address on the FastAPI receiving end. Do we still need to adapt the field [`deployment_id`](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/blob/develop/nomad/config.py#L357) per local instance (we would make sure that this is unique -- maybe taking a hash from some config parameters)?
To clarify:
* [x] A problem for testing is currently, that when the app is started with `nomad app run app`, then there the logs are different set than to the log config in docker. Should this maybe be fixed? (If starting Nomad with `nomad app run app`, then no API calls are processed in logstash format).
* [x] Use threading TCP server? If a client does not close the connection to the logstash proxy server, it can block the whole server. `ThreadingTCP` server is only supported for UNIX systems.
* [ ] Validate IP address that was received in federation/logs?
* [ ] Protect against malicious (gzipped) content?
* against too large data (load into main memory)
* against gzip bombs (uncompress too much data into main memory)
* too many newlines in the data (long running loop)https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/893Added a new workflow schema with examples and docs.2022-12-14T08:55:05ZMarkus ScheidgenAdded a new workflow schema with examples and docs.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/892Additional descriptions and formatting to sections and quantities in Perovski...2022-10-19T11:05:56ZJose Marquez PrietoAdditional descriptions and formatting to sections and quantities in Perovskite Solar Cell Database schemahttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/891Resolve "DFT and ELN compatible search config"2022-10-20T05:57:45ZMarkus ScheidgenResolve "DFT and ELN compatible search config"Closes #1133Closes #1133Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/890Support for mounts and extra_hosts keyword in tools.json2022-10-20T06:54:46ZFlorian DobenerSupport for mounts and extra_hosts keyword in tools.jsonThis MR allows a mounts and extra_hosts keyword in the tools.json file to mount external directories and to pass additional hosts into the docker container.
See https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub/-/blob/fhi-pcr...This MR allows a mounts and extra_hosts keyword in the tools.json file to mount external directories and to pass additional hosts into the docker container.
See https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub/-/blob/fhi-pcr840/tools.json#L114 for an usage example.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/889Change style of references2022-10-19T07:55:48ZTheodore ChangChange style of referencesChange from `../entries/{entry_id}` to `../uploads/{upload_id}/archive/{entry_id}` style.
Closes #1120Change from `../entries/{entry_id}` to `../uploads/{upload_id}/archive/{entry_id}` style.
Closes #1120Theodore ChangTheodore Chang