nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2022-03-24T10:33:25Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/593Implemented new full parsers, minor metainfo updates2022-03-24T10:33:25ZAlvin Noe LadinesImplemented new full parsers, minor metainfo updatesdl_poly, cpmd full parsersdl_poly, cpmd full parsersMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/592tabular tree2022-03-15T13:59:55ZFelix Dietrichtabular treeTabular tree metaschema for flexible metaschema definitions during upload.Tabular tree metaschema for flexible metaschema definitions during upload.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/591Add async interface2022-03-15T11:09:31ZTheodore ChangAdd async interfaceCloses #763Closes #763Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/590Added a new util prod deployment.2022-03-15T11:45:15ZMarkus ScheidgenAdded a new util prod deployment.This adds another manual task to the CI/CD "prod deploy util". This allows to deploy the branch to a nomad installation at: https://nomad-lab.eu/prod/v1/util
This installation uses the full nomad data and can be used for a variety of te...This adds another manual task to the CI/CD "prod deploy util". This allows to deploy the branch to a nomad installation at: https://nomad-lab.eu/prod/v1/util
This installation uses the full nomad data and can be used for a variety of tests, e.g. benchmark measurement.
The app is scaled very small to just one replica with a single worker. This hopefully makes it easier to test, as all requests should go to the same process on the server. See `ops/helm/nomad/deployments/prod-util-values.yaml` to change this.Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/589EditMembers accessibility (#764)2022-03-14T15:25:40ZMohammad NakhaeeEditMembers accessibility (#764)Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/588GUI_Test_UploadsPage2022-03-29T06:39:35ZMohammad NakhaeeGUI_Test_UploadsPageAdd GUI tests for uploads page, upload page, and members edit dialog.Add GUI tests for uploads page, upload page, and members edit dialog.Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/587Add async interface2022-03-15T09:50:39ZTheodore ChangAdd async interfaceCloses #763
Add async interface that can be directly used in an async environment.Closes #763
Add async interface that can be directly used in an async environment.Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/586Resolve "Missing/wrong constant shifts in unitsData.json"2022-03-14T09:32:07ZLauri HimanenResolve "Missing/wrong constant shifts in unitsData.json"Closes #762Closes #762Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/585Support parsers creating multiple entries from the same file2022-03-21T08:15:16ZDavid SikterSupport parsers creating multiple entries from the same fileCloses #761
We make it possible for an entry (associated with some mainfile) to have _child entries_. This is done by introducing a new field on the entry level: `mainfile_key`. Main entries have `mainfile_key == None`, child entries h...Closes #761
We make it possible for an entry (associated with some mainfile) to have _child entries_. This is done by introducing a new field on the entry level: `mainfile_key`. Main entries have `mainfile_key == None`, child entries have the same value for `mainfile` as the main/parent entry, plus some non-empty string as value for the `mainfile_key`. Note, however, that most parsers will only produce a main entry without any child entries.
Both the main and the child entries are full-fledged entries, i.e. they are distinct objects in mongo and elastic search, they have their own archive files, their own metadata, and so on.
The combination `(upload_id, mainfile, mainfile_key)` uniquely identifies any entry. For every child entry, a main entry must exist (i.e. an entry with the same `upload_id` and `mainfile`, but with `mainfile_key == None`).
Parsers signal that they want to create child entries by returning a set of keys, one for each child, from the `is_mainfile` function (instead of a boolean, like we have done up until now). Parsers that don't want to create any child entries can just return True, as before.
The parse function is called only once, for the main entry, and we pass an additional argument to it: `child_archives`, a dictionary of the format `{mainfile_key: child_archive}`.David SikterDavid Sikterhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/584Resolve "ElasticSearch upgrade for v1.x"2022-03-18T13:22:27ZLauri HimanenResolve "ElasticSearch upgrade for v1.x"Closes #760
@mscheidg: Luckily I did not have to change that much, see the issue for the changes I did. How do we go from here? Will you just merge and then we deploy this to beta? The old indices should work, at least that's what [ES s...Closes #760
@mscheidg: Luckily I did not have to change that much, see the issue for the changes I did. How do we go from here? Will you just merge and then we deploy this to beta? The old indices should work, at least that's what [ES says in their docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/breaking-changes-7.0.html#_indices_created_before_7_0).Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/583Resolve "Improve ArchiveReader performance"2022-03-20T23:10:59ZTheodore ChangResolve "Improve ArchiveReader performance"Closes #747
Use anyio to implement async file I/O.
anyio handles nested event loops well (celery + asyncio).Closes #747
Use anyio to implement async file I/O.
anyio handles nested event loops well (celery + asyncio).Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/582Edit Quantity Components (#740)2022-03-05T08:50:17ZMohammad NakhaeeEdit Quantity Components (#740)Origin: #719Origin: #719Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/581edit quantity components (#719)2022-03-04T13:54:24ZMohammad Nakhaeeedit quantity components (#719)Modules have been splitted to separate filesModules have been splitted to separate filesMohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/580Merge for release v1.0.32022-03-03T08:21:14ZMarkus ScheidgenMerge for release v1.0.3https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/579Add pdf preview and improved memoizing2022-03-04T09:35:30ZDavid SikterAdd pdf preview and improved memoizingAdds viewer for pdf files.Adds viewer for pdf files.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/578TinyMCE as a rich text edit2022-03-02T08:26:24ZSherjeel ShabihTinyMCE as a rich text editCurrently, we save the images dropped as base64 strings. This should be replaced with URLs to binary files once other things are sorted that can allow that.
I have also added a Nomad skin to TinyMCE. All of this can be moved in a better...Currently, we save the images dropped as base64 strings. This should be replaced with URLs to binary files once other things are sorted that can allow that.
I have also added a Nomad skin to TinyMCE. All of this can be moved in a better build routine as you feel it fits better.Sherjeel ShabihSherjeel Shabihhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/577EditQuantity components (#740)2022-03-14T10:15:12ZMohammad NakhaeeEditQuantity components (#740)Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/576Resolve "NOMAD Oasis with keycloak"2022-03-01T13:21:08ZMarkus ScheidgenResolve "NOMAD Oasis with keycloak"Closes #758Closes #758Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/575Add support for dropping files for file reference quantities2022-03-02T08:26:25ZDavid SikterAdd support for dropping files for file reference quantitiesAdds an EditQuantity-component for file reference quantities. The component allows the user to select/drop local files and upload them.Adds an EditQuantity-component for file reference quantities. The component allows the user to select/drop local files and upload them.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/574Resolve "Metainfo search is broken in v1"2022-03-01T12:57:13ZMarkus ScheidgenResolve "Metainfo search is broken in v1"Closes #752Closes #752Markus ScheidgenMarkus Scheidgen