nomad-FAIR merge requestshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests2022-03-04T09:35:30Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/579Add pdf preview and improved memoizing2022-03-04T09:35:30ZDavid SikterAdd pdf preview and improved memoizingAdds viewer for pdf files.Adds viewer for pdf files.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/578TinyMCE as a rich text edit2022-03-02T08:26:24ZSherjeel ShabihTinyMCE as a rich text editCurrently, we save the images dropped as base64 strings. This should be replaced with URLs to binary files once other things are sorted that can allow that.
I have also added a Nomad skin to TinyMCE. All of this can be moved in a better...Currently, we save the images dropped as base64 strings. This should be replaced with URLs to binary files once other things are sorted that can allow that.
I have also added a Nomad skin to TinyMCE. All of this can be moved in a better build routine as you feel it fits better.Sherjeel ShabihSherjeel Shabihhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/577EditQuantity components (#740)2022-03-14T10:15:12ZMohammad NakhaeeEditQuantity components (#740)Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/576Resolve "NOMAD Oasis with keycloak"2022-03-01T13:21:08ZMarkus ScheidgenResolve "NOMAD Oasis with keycloak"Closes #758Closes #758Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/575Add support for dropping files for file reference quantities2022-03-02T08:26:25ZDavid SikterAdd support for dropping files for file reference quantitiesAdds an EditQuantity-component for file reference quantities. The component allows the user to select/drop local files and upload them.Adds an EditQuantity-component for file reference quantities. The component allows the user to select/drop local files and upload them.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/574Resolve "Metainfo search is broken in v1"2022-03-01T12:57:13ZMarkus ScheidgenResolve "Metainfo search is broken in v1"Closes #752Closes #752Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/573Making file browser editable2022-02-25T07:25:50ZDavid SikterMaking file browser editableExtending the browser with basic edit functionality. For now, when a something is added or deleted, a dialog shows up and blocks further actions until processing is complete. This "modal" dialog is defined in `Browser.js`, and some codec...Extending the browser with basic edit functionality. For now, when a something is added or deleted, a dialog shows up and blocks further actions until processing is complete. This "modal" dialog is defined in `Browser.js`, and some codechanges have also been made to allow rerendering (and possibly re-fetching data via the api) of adaptors/lanes even when the path is the same. This part needs more work, though.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/572Resolve "Archive API raised 500 if NaN values are present"2022-02-25T07:21:16ZMarkus ScheidgenResolve "Archive API raised 500 if NaN values are present"Closes #754Closes #754Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/571Draft: edit Quantity (#740)2022-02-24T13:31:04ZMohammad NakhaeeDraft: edit Quantity (#740)Mohammad NakhaeeMohammad Nakhaeehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/569Moved dcat to fastapi. Added dois to dcat. Removed flask_restplus.2022-02-22T07:22:44ZMarkus ScheidgenMoved dcat to fastapi. Added dois to dcat. Removed flask_restplus.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/568Parsers2022-03-15T12:32:30ZAlvin Noe LadinesParsers1. Implemented new full parsers: gamess, tinker, qbox, onetep, dftbplus, fleur, siesta
2. Group parsers into: electronic, atomistic, workflow
3. Move parser initialization to nomad
4. Move gw results to calculation
5. Various minor fixes1. Implemented new full parsers: gamess, tinker, qbox, onetep, dftbplus, fleur, siesta
2. Group parsers into: electronic, atomistic, workflow
3. Move parser initialization to nomad
4. Move gw results to calculation
5. Various minor fixesMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/567Improved preview2022-02-22T07:49:02ZDavid SikterImproved previewSome minor fixes and improvements of the file browser and preview functionality:
- the `FilePreview` component is moved to a separate file
- code is refactored by introducing viewer objects, one for each file type we support. The viewer ...Some minor fixes and improvements of the file browser and preview functionality:
- the `FilePreview` component is moved to a separate file
- code is refactored by introducing viewer objects, one for each file type we support. The viewer object stores constant information, like matching file extensions, maxSizeAutoPreview etc, plus a `render` function.
- a viewer is introduced for json files (the same viewer used elsewhere).
- txt, yaml and yml files are automatically previewed with the text viewer
- error handling is added. If the viewer fails, we show a message and offers the user to view the file using the text viewer instead.
- optimizing the vertical file browser api calls, using the `include_entry_info` option instead of making a costly second api call.
- Highlighting the current entry mainfileMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/566H5web north2022-02-21T07:59:37ZSandor BrockhauserH5web northhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/565Optimize ArchiveReader read disk performance2022-02-23T07:22:20ZTheodore ChangOptimize ArchiveReader read disk performanceCloses #751Closes #751Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/564Resolve "In-api processing of simple and individual entries"2022-02-18T14:34:22ZDavid SikterResolve "In-api processing of simple and individual entries"Closes #745.
The endpoint for PUT `uploads/{upload_id}/raw/{path}` is extended with two query arguments: `wait_for_processing` and `include_archive`. When specifying `wait_for_processing`, the upload and processing is run locally and th...Closes #745.
The endpoint for PUT `uploads/{upload_id}/raw/{path}` is extended with two query arguments: `wait_for_processing` and `include_archive`. When specifying `wait_for_processing`, the upload and processing is run locally and the endpoint call blocks until the processing is complete (but only the uploaded file is matched and (if a parser matches) processed, nothing else), and we return an additional `processing` key in the response. If `include_archive` is specified, the archive is also included in the response.
The response now has the following format:
```
{
"upload_id": ...,
"data": <upload Proc data>
"processing": {
"upload_id": ...,
"path": <full raw path to the uploaded file>,
"entry_id": ...,
"parser_name": ...,
"entry": <entry proc data>
"archive": <archive data>
}
}
```
Which is identical to before except for the addition of the `processing` key. The `processing` key will be null if not specifying `wait_for_processing`. If we do specify `wait_for_processing`, but the file does not match any parser, the entry-related values inside the `processing` dict will be all null.
I introduced a new decorator `@process_local`, which was maybe a bit ambitious, but I think it was the best way to do it to get a consistent handling of the process attributes, like `process_status`, `sync_counter` etc, and unified error boundary. The local process cannot be started (i.e. the endpoint will fail) if something else is running, and nothing else can be run or scheduled to run while a local process is running.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/563Improved preview, including images and automatically previewing small files (...2022-02-16T07:43:48ZDavid SikterImproved preview, including images and automatically previewing small files (#738)Adds support for simple image preview, and paves the way for adding viewers for additional file types. Images are previewed automatically if they are small enough (<10 MB), otherwise the user has to request it by clicking on a preview bu...Adds support for simple image preview, and paves the way for adding viewers for additional file types. Images are previewed automatically if they are small enough (<10 MB), otherwise the user has to request it by clicking on a preview button. For text files (the fallback viewer) the user always has to press the button to open the viewer.
Also solves some issues with the text preview scrolling.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/561Rewrite ArchiveQuery with asyncio2022-02-17T12:27:52ZTheodore ChangRewrite ArchiveQuery with asyncioCloses #704Closes #704Theodore ChangTheodore Changhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/560Resolve "Improved GUI search tests"2022-03-07T08:54:56ZLauri HimanenResolve "Improved GUI search tests"Closes #748
@mscheidg: Could you take a look and see that my overall approach is reasonable. Especially I'd like some comments on:
- I had to somehow formalize the test configuration, and for now, I did it by creating a config file at...Closes #748
@mscheidg: Could you take a look and see that my overall approach is reasonable. Especially I'd like some comments on:
- I had to somehow formalize the test configuration, and for now, I did it by creating a config file at `nomad-FAIR/nomad-test.yaml`. This configuration is loaded when node.js is instantiating the test states and should also be used when serving the application for test usage (`nomad admin run appworker`). Is this approach OK in your opinion? Btw: I had to put the config file in the repository root folder, because otherwise the volume access is not working.
- Do you think the current configuration in `nomad-FAIR/nomad-test.yaml` is distinct enough so that it does not interfere with other instances?
- Can you take a look at some of the snapshot files (e.g. gui/tests/data/search/min_max_aggregation_n_elements.json) and see if the format is OK?Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/559Merge latest patches2022-02-11T08:27:40ZMarkus ScheidgenMerge latest patchesFixed issue with not getting the full aggregation data whenever the option for 'not processed' is filtered out.Fixed issue with not getting the full aggregation data whenever the option for 'not processed' is filtered out.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/merge_requests/558Fix regressions in setup.sh2022-02-08T09:57:48ZMarkus ScheidgenFix regressions in setup.sh