nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2021-12-10T16:05:38Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/474Resolve "Upload API does not use coauthors and reviewers"2021-12-10T16:05:38ZMarkus ScheidgenResolve "Upload API does not use coauthors and reviewers"Closes #681Closes #681v1.0.0-betaMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/475Parsers2021-12-09T14:32:39ZAlvin Noe LadinesParsersAdded yambo and psi4 parsers. Implemented various parser fixes.Added yambo and psi4 parsers. Implemented various parser fixes.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/476Resolve "VSCode settings in the source code"2021-12-09T10:43:11ZMarkus ScheidgenResolve "VSCode settings in the source code"Closes #687Closes #687Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/477Resolve "Metainfo reference from path"2021-12-15T09:17:50ZMarkus ScheidgenResolve "Metainfo reference from path"Closes #678Closes #678Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/478GUI improvements and new GUI style menu2021-12-10T15:59:28ZLauri HimanenGUI improvements and new GUI style menuContains the following GUI improvements that are also discussed in #674:
- Added a new input component `InputField` that replaces `InputText`, `InputCheckboxes` and `InputSelect` by combining their functionality into a single component.
...Contains the following GUI improvements that are also discussed in #674:
- Added a new input component `InputField` that replaces `InputText`, `InputCheckboxes` and `InputSelect` by combining their functionality into a single component.
- Aggregations are no longer pre-loaded if they are not needed immediately. Aggregations will be loaded on-demand e.g. when opening a menu or displaying a statistic.
- Reduced the size of terms aggregations. Previously a greedy default of 200 items was used, but now only the amount that is necessary for visualization is loaded. The amount is determined from the maximum size requested by any visible component. `InputField` has controls for increasing the requested size (it currently uses a fairly naive approach of just increasing the aggregation size, scrolling could be possible if it does not interfere with `exclude_from_search`).
- Statistics are now normalized to the aggregation maximum.
- The search bar has been restyled to be more recognizable. I moved the search icon to the beginning (like e.g. in GMail search) and made the placeholder text always visible. We could also try using some colour to highlight it more, but blue buttons are not consistent with the rest of the UI and blue outline is not compatible with the focused state.
- Modified the search column defaults to be less method-oriented.
- Fixed issue with exclude_from_search being enabled even though no user interaction had been done to the filters.
- Removed the undefined statistics in the `visibility`-filter.
- Introduced a new style for the GUI:
- Different icon and icon size for the button that enables/disables statistics for a filter.
- Different text input style and size
- Statistics scaling is now shown directly and not hidden inside a menu
- Added a new menu for playing around with the GUI style. This menu is hidden by default, but can be enabled by pressing **Alt+G**.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/479Refactor proc framework2021-12-13T11:40:48ZDavid SikterRefactor proc frameworkMajor refactor of the proc framework. The join functionality is lifted up to the base class and generalized. By marking a process with `is_child = True`, you tell the framework that we should try to join with (resume) the parent Proc (ob...Major refactor of the proc framework. The join functionality is lifted up to the base class and generalized. By marking a process with `is_child = True`, you tell the framework that we should try to join with (resume) the parent Proc (obtained by calling `self.parent` on the child Proc) when he child process is done. When the join succeeds (when all children are done), the parent is put to RUNNING (previously it remained in status WAITING_FOR_RESULT) and the new `join` method is called. The Proc may from there go back to status WAITING_FOR_RESULT again (so that we can implement the idea of "slotted" parsing, see #668).
Further, queuing is made possible (#667). I.e. when a process is running, it may be possible to invoke another process on the same object. The framework should ensure that the processes are queued up and executed one at a time in sequence. By setting `is_blocking = True`, you can ensure that certain processes "block" additional queuing until they are done, as it does not make sense/seem safe to queue up calls in every case. I have marked `delete_upload`, `publish_upload`, and `publish_externally` as "blocking". The queue is persisted to the mongodb object. Measures are taken to ensure that scheduling, starting and completing a process are atomic operations.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/480Resolve "Metainfo-based filter in v1 search GUI"2021-12-15T09:05:55ZMarkus ScheidgenResolve "Metainfo-based filter in v1 search GUI"Closes #691, #688, #695Closes #691, #688, #695Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/481Added save the nomad access token in jhub to fix north/77.2021-12-14T08:03:28ZMarkus ScheidgenAdded save the nomad access token in jhub to fix north/77.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/482V0.10.11 hotfixes2021-12-14T08:04:49ZMarkus ScheidgenV0.10.11 hotfixeshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/483Introducing parser levels, defining the order of processing2021-12-16T10:12:34ZDavid SikterIntroducing parser levels, defining the order of processingParser can define a field `level` which determines which order they will be run in, as requested by #668 . The default value is 0. Level 0 parsers are run first, then level 1, then level 2, and so on. The code defined in `Upload` and `Ca...Parser can define a field `level` which determines which order they will be run in, as requested by #668 . The default value is 0. Level 0 parsers are run first, then level 1, then level 2, and so on. The code defined in `Upload` and `Calc` for special handling of the phonopy parser is therefore also removed (the functionality defined there should be moved to the parser instead).Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/484MR_UploadPage_Members_fix2021-12-17T10:51:36ZMohammad NakhaeeMR_UploadPage_Members_fixIssue #673
The GUI part is also adapted with the changes in the apiIssue #673
The GUI part is also adapted with the changes in the apiMohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/486Resolve "Updated documentation for v1"2021-12-22T12:10:52ZMarkus ScheidgenResolve "Updated documentation for v1"Closes #699Closes #699Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/487Fixes after reprocessing #689 #6902021-12-16T12:16:39ZAlvin Noe LadinesFixes after reprocessing #689 #690Provided fixes for the list indicated in issues #689 #690Provided fixes for the list indicated in issues #689 #690Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/489Resolve "v1 GUI raw files"2021-12-16T15:44:06ZMarkus ScheidgenResolve "v1 GUI raw files"Closes #692Closes #692Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/490Unify logic for setting metadata via files and via the api/process2021-12-17T10:51:03ZDavid SikterUnify logic for setting metadata via files and via the api/processCloses #694. With these changes, `MetadataEditRequestHandler` handles both metadata files and requests to edit metadata through the api/by calling `Upload.edit_upload_metadata`, and we thus have the same validation and json format in bot...Closes #694. With these changes, `MetadataEditRequestHandler` handles both metadata files and requests to edit metadata through the api/by calling `Upload.edit_upload_metadata`, and we thus have the same validation and json format in both cases. Setting metadata via the ArchiveParser is also disallowed, at least for now (if this should be allowed, we should probably use the same class also in this case). Also refactored `MetadataEditRequestHandler` a bit to make it use a simpler and more consistent terminology.David SikterDavid Sikterhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/491GUI improvement2021-12-17T11:05:35ZLauri HimanenGUI improvement- The suggestion list in the search bar now supports custom prioritization of items. I bumped up the priority of certain metainfo (elements, formulas, program name, author etc.)
- Improved the layout for `EntryDetails`: still not perfec...- The suggestion list in the search bar now supports custom prioritization of items. I bumped up the priority of certain metainfo (elements, formulas, program name, author etc.)
- Improved the layout for `EntryDetails`: still not perfect but better.
- Added number formatting (=separators) for the number of results.
- Improved the `Quantity`-component: now it automatically loads a preset based on the quantity name (this preset can still be overridden). This makes sure that quantities look identical no matter where they are used in the app. E.g. the quantities shown by `EntryDetails` and `OverviewView` were slightly different before, now they are identical and require less config.
- Added formatting for timestamps in the results list.
- Completely redesigned filter menu headers (see screenshot):
- The new headers are separated into two "levels": There is a topmost level that shows the common filter panel actions and can also show an overline title text. The second level is reserved for information and actions that are specific to the currently shown menu: it shows the menu title and can also show a list of actions.
- The resource (entries or materials) targeted by the currently active search context is now shown in the overline text of the main filter menu. This resource was not shown explicitly before e.g. in "Search your own data" or the dataset search page. If we want to enable materials search for these pages, it would be natural to make this simple title into a dropdown menu.
- The overline text in the submenu now shows the number of results. This allows the user to directly see the number of results when the submenu is open, but the number of results is never visible twice.
- I added a checkbox for directly filtering for `method_name` in the DFT, GW and EELS menus. With the new layout, there is now space for adding this checkbox along with other actions if needed.
- The filter header is now "sticky": it will not scroll along with the options. This ensures that the actions and the number of results are always shown.
- Added better scaling defaults for statistics.
- Now the "show more" button tracks the aggregation call and will be disabled when the items are loading (previously it was just disappearing incorrectly and there was no way to track the status of an individual aggregation request).
- Fixed issues in the statistics item sizing: `InputList` was sometimes loading an incorrect number of items and the title was overflowing in some extreme cases.
- I added `quantities` also to the materials index because the new Archive search panel was broken for materials search. If this causes ES space issues we could consider removing this data and then disabling the Archive search in materials search.
- Fixed issue with handling value changes in `InputRadio`.
- Added caching for dataset search page.
![example](/uploads/2276d8e0d967fa8b98aa42bb764b7f5b/example.jpg)Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/492Clean API user metadata.2021-12-17T18:16:03ZMarkus ScheidgenClean API user metadata.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/493Clean API user metadata.2021-12-17T18:15:37ZMarkus ScheidgenClean API user metadata.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/494Set gromacs parser to master.2021-12-20T10:03:41ZMarkus ScheidgenSet gromacs parser to master.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/495Improved incremental processing and upload API2021-12-21T15:16:19ZDavid SikterImproved incremental processing and upload APICloses #587. Introduces an additional `path_filter` argument to the `process_upload` that enables limiting the processing to a specific path. This is used to optimize the API calls to put and delete individual files and folders at the sp...Closes #587. Introduces an additional `path_filter` argument to the `process_upload` that enables limiting the processing to a specific path. This is used to optimize the API calls to put and delete individual files and folders at the specified raw path.David SikterDavid Sikter