nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2021-10-07T06:18:42Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/404Added north launch token to get upload by id API.2021-10-07T06:18:42ZMarkus ScheidgenAdded north launch token to get upload by id API.This is part of the north project: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub/-/issues/22This is part of the north project: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub/-/issues/22https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/408Fix a bug concerning the outcar parser2021-10-07T13:57:18ZMohammad NakhaeeFix a bug concerning the outcar parserThis was discussed in issue #537
When we use the file name as nomad parse arguments it returns empty
The problem was resolved by getting the absolute path of the argumentThis was discussed in issue #537
When we use the file name as nomad parse arguments it returns empty
The problem was resolved by getting the absolute path of the argumenthttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/409Refactoring metadata: using a consistent flat structure in MongoDB2021-10-08T11:16:20ZDavid SikterRefactoring metadata: using a consistent flat structure in MongoDBGetting rid of Calc.metadata and instead storing the fields directly on the Calc object, plus some minor name changes for consistency and cleanup of old code.
**Metadata refactoring:**
- `Dataset.name` -> `Dataset.dataset_name`
- `Datas...Getting rid of Calc.metadata and instead storing the fields directly on the Calc object, plus some minor name changes for consistency and cleanup of old code.
**Metadata refactoring:**
- `Dataset.name` -> `Dataset.dataset_name`
- `Dataset.created` -> `Dataset.dataset_create_time`
- `Dataset.modified` -> `Dataset.dataset_modified_time`
- `Calc.metadata.last_processing` -> `Calc.last_processing_time`
- `Calc.metadata.last_edit` -> `Calc.last_edit_time`
- Fields `calc_hash`, `pid`, `last_processing_time`, `last_edit_time`, `external_id`, `nomad_version`, `nomad_commit`, `external_db`, `comment`, `references`, `coauthors`, `shared_with`, `datasets` are moved from `Calc.metadata` to `Calc`
- `Calc.metadata` (and `Calc.metadata.raw_id`) are removed
**Additionally:**
- Metadata files are only read when an entry is first processed, not when reprocessing or when importing bundles. Note, if the initial processing fails we will not try to read the metadata files again. Maybe this could be smarter...
- Functionality connected to import of metadata from oasis metadata files is removed. Testing of new bundle-based oasis export/import is extended.v1.0.0-feature-freezeMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/411renaming "name" to "upload_name"2021-10-11T06:09:30ZMohammad Nakhaeerenaming "name" to "upload_name"issue #621
The problem was resolved by renaming "name" to "upload_name". I tested the upload page and it works.issue #621
The problem was resolved by renaming "name" to "upload_name". I tested the upload page and it works.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/402Added command to clean publish raw .zip files. #5902021-10-11T10:49:24ZMarkus ScheidgenAdded command to clean publish raw .zip files. #590https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/412Add cli command to quarantine certain raw files. #5902021-10-13T06:21:54ZMarkus ScheidgenAdd cli command to quarantine certain raw files. #590https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/413Some fixes in gui part regarding DataTable2021-10-13T08:32:20ZMohammad NakhaeeSome fixes in gui part regarding DataTableThe merge request is related to the problems discussed in issue #619The merge request is related to the problems discussed in issue #619https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/415V1.0.0 issue5772021-10-14T07:15:52ZMohammad NakhaeeV1.0.0 issue577The first and the third tasks in #577 have been fixed.The first and the third tasks in #577 have been fixed.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/414Add some info about ssh key before git clone2021-10-14T07:16:22ZMohammad NakhaeeAdd some info about ssh key before git cloneI provided some explanation about the NOMAD repo accessibility and GitLab ssh key. Could you please take a look? Please let me know if it needs correction or needs more info.I provided some explanation about the NOMAD repo accessibility and GitLab ssh key. Could you please take a look? Please let me know if it needs correction or needs more info.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/417Fix issue #6312021-10-14T09:13:38ZAlvin Noe LadinesFix issue #631Added support for hybrids in exciting parser.Added support for hybrids in exciting parser.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/416Added better support for EELS data2021-10-14T09:37:32ZLauri HimanenAdded better support for EELS data- Added EELS search filters into the GUI (resolution, energy_window, detector_type, eels spectrum)
- Fixed issue in parser not reporting units for energies
- Updated the experimental metainfo for device settings to include units and smal...- Added EELS search filters into the GUI (resolution, energy_window, detector_type, eels spectrum)
- Fixed issue in parser not reporting units for energies
- Updated the experimental metainfo for device settings to include units and small description.
- Added new EELS metadata into results together with regtests
- Small GUI refactoring for the process of serializing/deserializing filter values (was needed for serializing energy_window which does not correspond to a single metainfo quantity)
- Small GUI refactoring for the components used to display properties in the overview (`PropertyCard`, `PropertyContent`).https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/418Fix issue #6312021-10-14T12:32:29ZAlvin Noe LadinesFix issue #631Added support for hybrids in exciting parser.Added support for hybrids in exciting parser.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/420Elastic search read-only issues2021-10-14T16:27:27ZMohammad NakhaeeElastic search read-only issuesRelated to #630
Not sure about the values for
- cluster.routing.allocation.disk.watermark.flood_stage=1gb
- cluster.routing.allocation.disk.watermark.low=4gb
- cluster.routing.allocation.disk.waterm...Related to #630
Not sure about the values for
- cluster.routing.allocation.disk.watermark.flood_stage=1gb
- cluster.routing.allocation.disk.watermark.low=4gb
- cluster.routing.allocation.disk.watermark.high=2gb
It is also possible to set in percentages but I think when we have a lot of disk space their values can be too much for such thresholds.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/419Some small changes in previous issues2021-10-14T16:28:06ZMohammad NakhaeeSome small changes in previous issuesAdd some margin under the DataTable #619 third task
And a change regarding #577 third taskAdd some margin under the DataTable #619 third task
And a change regarding #577 third taskMohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/423Fixed issues in polling site. Now one request is made immediately and...2021-10-15T05:53:08ZLauri HimanenFixed issues in polling site. Now one request is made immediately and...Fixed issues in polling site. Now one request is made immediately and subsequent ones every 5 seconds.Fixed issues in polling site. Now one request is made immediately and subsequent ones every 5 seconds.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/422Refactor permissions2021-10-15T07:00:49ZDavid SikterRefactor permissionsRefactoring the permission-related fields as discussed in #580
- `Upload.user_id` and `EntryMetadata.uploader` are renamed to `main_author`
- It becomes possible to set coauthors on both the upload and the entry level. The fields are na...Refactoring the permission-related fields as discussed in #580
- `Upload.user_id` and `EntryMetadata.uploader` are renamed to `main_author`
- It becomes possible to set coauthors on both the upload and the entry level. The fields are named `coauthors` and `entry_coauthors`, respectively.
- **NOTE 1**: The field `coauthors` was previously defined on the entry level, but now it is set on the upload level.
- **NOTE 2**: we prefer users to use the field `coauthors` on the upload level, and if possible it would be nice to get rid of the `entry_coauthor` field eventually.
- The field `Calc.shared_with` is renamed and moved to `Upload.reviewers` (the name change is to make it more clear that they only have read permissions, now that there will potentially be multiple readers and writers)
- The derived field `EntryMetadata.owners` is renamed to `viewers` (also to make it clearer that these are the users who can _read_ the data). The field is defined as `main_author` + `coauthors` + `reviewers`.
- Introduces a new derived field `EntryMetadata.writers` to denote users who can write to the upload. Defined as `main_author` + `coauthors`
- The derived field `EntryMetadata.authors` is defined as `main_author` + `coauthors` + `entry_coauthors`
- **NOTE**: being in the `entry_coauthors` doesn't give you any permissions, neither to read from or to write to the upload. It only means you're listed as an author in the search index. This is the point, because we want the permissions to be determined at the upload level, so you either have read/write access to the _whole_ upload or not (would be messy to have different permissions to different entries within the same upload)Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/424Additional cleanup of processing related Mongo fields2021-10-15T13:57:28ZDavid SikterAdditional cleanup of processing related Mongo fields- Removes the field `Proc.current_process_step` and instead use `last_status_message`
- Removes the "hacky" solution with the field `Upload.pending_operations` and use process arguments instead- Removes the field `Proc.current_process_step` and instead use `last_status_message`
- Removes the "hacky" solution with the field `Upload.pending_operations` and use process arguments insteadMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/426add some changes in uploads page and upload page2021-10-20T09:04:18ZMohammad Nakhaeeadd some changes in uploads page and upload pageReprocess button was added to the upload page and a reload button was added to the table of uploads.
Additionally, the NewUploadButton would be disabled when we reach the maximum number of unpublished uploads.Reprocess button was added to the upload page and a reload button was added to the table of uploads.
Additionally, the NewUploadButton would be disabled when we reach the maximum number of unpublished uploads.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/427Set a maximum width for all browser tabs2021-10-20T09:15:48ZMohammad NakhaeeSet a maximum width for all browser tabsConcerning issue #599
I set the maximum width = 340
In my browser, it looks like this
![Screenshot](/uploads/fa64d2725240d978ec523d8b616e0e07/Screenshot.png)
It can be smaller, but I am afraid of masking the keys.Concerning issue #599
I set the maximum width = 340
In my browser, it looks like this
![Screenshot](/uploads/fa64d2725240d978ec523d8b616e0e07/Screenshot.png)
It can be smaller, but I am afraid of masking the keys.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/428Add OpenMX to the list of supported parsers2021-10-20T13:40:55ZMohammad NakhaeeAdd OpenMX to the list of supported parsersIssue #635
Not sure about the followings
"parserSpecific": "",
"tableOfFiles": ""
Where can I check them?Issue #635
Not sure about the followings
"parserSpecific": "",
"tableOfFiles": ""
Where can I check them?Mohammad NakhaeeMohammad Nakhaee