nomad-FAIR issueshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues2024-03-22T17:14:40Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1957Preview plot from CLI parse2024-03-22T17:14:40ZHampus NaesstroemPreview plot from CLI parseIt would be great with some sort of preview of the plot generated from calling the CLI parse command. Currently we have to either inspect the json manually or run nomad locally to test this during plugin development.It would be great with some sort of preview of the plot generated from calling the CLI parse command. Currently we have to either inspect the json manually or run nomad locally to test this during plugin development.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1953Topology visualization improvement2024-03-25T11:57:34ZLauri HimanenTopology visualization improvementDue to limitations built into the GUI code, certain topologies cannot be shown. The following limitations should be lifted:
- [ ] If a system in topology does not contain atoms, we simply don't display the ngl viewer. This allows logica...Due to limitations built into the GUI code, certain topologies cannot be shown. The following limitations should be lifted:
- [ ] If a system in topology does not contain atoms, we simply don't display the ngl viewer. This allows logical groups that don't have their own visualization.
- [ ] The topology should not be required to have a single root node.Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1945Add unit support for percentages2024-03-21T08:23:23ZLauri HimanenAdd unit support for percentagesRelated to https://github.com/nomad-coe/nomad/issues/100. Starting from 0.21, Pint has built-in support for percentages. Unfortunately, also starting from that exact version, [this Pint issue](https://github.com/hgrecco/pint/issues/1809)...Related to https://github.com/nomad-coe/nomad/issues/100. Starting from 0.21, Pint has built-in support for percentages. Unfortunately, also starting from that exact version, [this Pint issue](https://github.com/hgrecco/pint/issues/1809) is preventing an easy migration as it breaks our parser code.
Tasks:
- [ ] Upgrade Pint, once multiplication issue is solved.
- [x] Ensure that new unit definitions get migrated to the front-end.
- [x] Update JS unit code to support percentages.
- [x] Add tests for the JS unit code.Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1937add reference to elabftw parser2024-03-15T00:49:10ZAmir Golparvaradd reference to elabftw parserAmir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1936Gallery search results2024-03-15T09:07:33ZMarkus ScheidgenGallery search resultsSome pseudo code
```js
const GalleryCard = (title, entryId) => {
return <Paper>
...
{children}
</Paper>
}
const H5WebGalleryCard = (result, archive_path) => {
return <GalleryCard {...}><H5Web path={result.mainfile + arch...Some pseudo code
```js
const GalleryCard = (title, entryId) => {
return <Paper>
...
{children}
</Paper>
}
const H5WebGalleryCard = (result, archive_path) => {
return <GalleryCard {...}><H5Web path={result.mainfile + archive_path}></H5Web></GalleryCard>
}
const DosGalleryCard = (result) => {
return <GalleryCard {...}><DOS entryId={result.entryId}/></GalleryCard>
}
const galleryComponents = {
'h5web': H5WebGalleryCard,
'dos': DosGalleryCard
}
app = {
search: {
gallery: {
component: 'h5web',
props: {
archive_path: '/nexus/NXmpes/entry/data/data'
}
}
}
<SearchGallery> // a new alternative for SearchResults
const {results, app, callbacks...} = useSearchContext() // ask lauri
const [pinnedResults, setPinnedResults] = useState()
<PinnedGallery>
<MUIGrid>
{pinnedResults.map(...)}
</MUIGrid>
</PinnedGallery>
<MainGallery
<MUIGrid>
{results.map(() => React.createEntry(galleryComponents[app.gallery.component], {result, ...app.gallery.props})}
</MUIGrid>
</MainGallery>
</SearchGallery>
```Sherjeel ShabihSherjeel Shabihhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1932Theming, profiles, and launch urls for NORTH2024-03-14T13:32:33ZMarkus ScheidgenTheming, profiles, and launch urls for NORTHThis is a continuation of #1897 (!1677)
- [ ] NOMAD and FAIRmat logos for JupyterHub, Jupyter, and JupyterLab
- [ ] !1677 added code to generate profiles for the k8s Jupyterhub, but it does not play well with NOMAD launching containers ...This is a continuation of #1897 (!1677)
- [ ] NOMAD and FAIRmat logos for JupyterHub, Jupyter, and JupyterLab
- [ ] !1677 added code to generate profiles for the k8s Jupyterhub, but it does not play well with NOMAD launching containers via service, also does not work for webtop tools
- [ ] profiles should allow hub urls that launch a tool (without nomad support, e.g. mounted uploads)
- [ ] hub should use the nomad api to determine the user optionsAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1907Extend workflow tasks for arbitrary raw files, hrefs, etc2024-02-29T10:07:12ZJoseph RudzinskiExtend workflow tasks for arbitrary raw files, hrefs, etcThere are many cases where users have complicated workflows, where the data/calculation from some steps are not supported in NOMAD. Developing support will be encouraged but takes time. So, it would be nice for them to be able to still m...There are many cases where users have complicated workflows, where the data/calculation from some steps are not supported in NOMAD. Developing support will be encouraged but takes time. So, it would be nice for them to be able to still make a complete provenance graph for their workflow in NOMAD, while simply pointing to raw files in their upload or adding an href (possibly other options to be thought of).
@ladinesa thoughts?https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1903automatic creation and resolution of hdf5 references for large arrays2024-03-05T11:05:08ZJoseph Rudzinskiautomatic creation and resolution of hdf5 references for large arraysStarted as a discussions in `NOMAD-Discord#developers`:
```
Joe:
Is there a built-in function to resolve an hdf5 reference in nomad-lab? perhaps @Alvin Noe Ladines you know?
Alvin:
I can implement an external function but I really th...Started as a discussions in `NOMAD-Discord#developers`:
```
Joe:
Is there a built-in function to resolve an hdf5 reference in nomad-lab? perhaps @Alvin Noe Ladines you know?
Alvin:
I can implement an external function but I really think that HDFReference should be implemented at the msection level. My opinion should be that any numpy array should be written in hdf5 when size exceeds some defined value. But we should discuss this in detail in an issue. should @Joseph Rudzinski or i create?
Joe:
Yes, this would be ideal. I don't exactly understand how this would work in terms of defining the quantities in the metainfo. Is there a way that we can get away from explicitly defining an hdf5 ref and link the reference when accessed? I guess you have an approach in mind?
I think it would probably be best if you implemented this automated type feature, since there are aspects that I am not that familiar with, but we could meet first to discuss if you want.
My proposal would be to write the units as a str attribute (UnitRegistry syntax) to the dataset in the hdf5 file and then automatically check and apply them when the reference is resolved. We could use/adapt the functions that I wrote in the H5MD parser.
```
Let's discuss @ladinesahttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1900Support serialization of references to inner sub sections in foreign Python p...2024-02-23T13:54:12ZMarkus ScheidgenSupport serialization of references to inner sub sections in foreign Python packagesIf you serialize a Python metainfo package that contains a reference to an inner section definition in another package you get an `MetainfoReferenceError` stating that this not yet supported.If you serialize a Python metainfo package that contains a reference to an inner section definition in another package you get an `MetainfoReferenceError` stating that this not yet supported.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1886Packages aliases in metainfo2024-03-01T11:03:51ZMarkus ScheidgenPackages aliases in metainfoWe are currently moving a lot of metainfo packages. It is necessary to have some infrastructure that allows to gracefully deprecate old package names. The goal is to not have to reprocess or otherwise migrate data instantly.
"Package al...We are currently moving a lot of metainfo packages. It is necessary to have some infrastructure that allows to gracefully deprecate old package names. The goal is to not have to reprocess or otherwise migrate data instantly.
"Package aliases" would allow to add old package names to a package definition:
- `m_from_dict` will consider package aliases when looking for definition referenced in `m_def`
- the gui will consider package aliases when looking for definitions referenced in `m_def`
- searches targeting `section_defs` will expand into an "or" on all package aliases
- facilities will be made available to clients to easily work with package names and consider package aliases in the process
Property aliases (espsecially for sub sections) is a related functionality, not covered in the issue.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1873Generic parser for H5 files2024-03-27T09:23:14ZAmir GolparvarGeneric parser for H5 filesSame as tabular parser, specialized annotations for a better mapping could be helpful here.Same as tabular parser, specialized annotations for a better mapping could be helpful here.Amir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1872pyiron integration2024-03-22T09:26:26ZAmir Golparvarpyiron integrationPyiron IDE is a jupyter-based code for high performance computing in material sciencePyiron IDE is a jupyter-based code for high performance computing in material scienceAmir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1866Config option to disable optimade API.2024-02-26T11:46:23ZMarkus ScheidgenConfig option to disable optimade API.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1854Add bond information to structure files2024-01-18T09:18:48ZLauri HimanenAdd bond information to structure filesSometimes simulations contain explicit bond information. This is currently stored in `system.atoms.bond_list`. Whenever this information is present, we should add it to the structure files produced by the systems API endpoint. Not all fo...Sometimes simulations contain explicit bond information. This is currently stored in `system.atoms.bond_list`. Whenever this information is present, we should add it to the structure files produced by the systems API endpoint. Not all formats will support this, but this should be added at least to the .pdb format that currently drives the Overview visualization. With the bond information, NGL can also display the bonds.Lauri HimanenLauri Himanenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1849Converter for json to and from pd dataframe2024-02-15T09:12:26ZAmir GolparvarConverter for json to and from pd dataframeflatten json into a pandas dataframe
reconstruct dataframe to json
options: depth of the json to flatten
options: order resultant dataframe alphabetically or by a list of stringsflatten json into a pandas dataframe
reconstruct dataframe to json
options: depth of the json to flatten
options: order resultant dataframe alphabetically or by a list of stringsAmir GolparvarAmir Golparvarhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1845Attribute inheritance2024-02-29T11:46:44ZMarkus ScheidgenAttribute inheritanceThe current metainfo does not support attribute inheritance. Attributes are primarily used in nexus. Here, the lack of attribute inheritance causes an excessive duplication of attribute definitions.
- [x] add inheritance to the metainf...The current metainfo does not support attribute inheritance. Attributes are primarily used in nexus. Here, the lack of attribute inheritance causes an excessive duplication of attribute definitions.
- [x] add inheritance to the metainfo python implementation
- [x] show attribute inheritance in the metainfo browser
- [x] adapt the generated nexus metainfoMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1824Improved unwrapping of structures2024-01-23T11:44:04ZLauri HimanenImproved unwrapping of structuresDinga WonankeDinga Wonankehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1812Scaffolding for the plugin example projects.2024-03-13T10:24:56ZMarkus ScheidgenScaffolding for the plugin example projects.We could add more features to the example plugin projects:
- [x] code style checks
- [ ] autoformatting checks and autoformatting pre-commit hooks (both opt-in)
- [x] linting
- [x] an empty mkdocs + github actions to deploy as gh-pages
...We could add more features to the example plugin projects:
- [x] code style checks
- [ ] autoformatting checks and autoformatting pre-commit hooks (both opt-in)
- [x] linting
- [x] an empty mkdocs + github actions to deploy as gh-pages
- [x] test coverage badge, pipeline badge
The mkdocs should include the theming/styling from the main project. We still need to figure out what should in the docs by default.
Some other ideas:
- maybe gh actions to build pypi package (is there something like a package/release registry in github?)
- maybe gh actions to build the docker file (gh does not have a docker resgistry? this would only make sense as opt-in where users define a registry to use)
The projects are :
https://github.com/nomad-coe/nomad-schema-plugin-example
https://github.com/nomad-coe/nomad-parser-plugin-example
@ladinesa is about to create a third one for normalizers
Just start with one project. Create PR there on github.Markus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1810Set co-author metadata for multiple uploads in one action2023-12-21T15:55:50ZAndrea AlbinoSet co-author metadata for multiple uploads in one actionI'm coming up with this question because some user would like to be able to share (or unshare) multiple uploads with some co-author or reviewer, when the uploads are more than 20 it can be tedious to go inside each of them to modify the ...I'm coming up with this question because some user would like to be able to share (or unshare) multiple uploads with some co-author or reviewer, when the uploads are more than 20 it can be tedious to go inside each of them to modify the metadata.
Would we face particular drawbacks implementing this action on a list of ticked uploads in the dashboard?
![Screenshot_from_2023-12-05_14-25-00](/uploads/b8efdf521bcbe555121a76b6b015ab66/Screenshot_from_2023-12-05_14-25-00.png)https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1805Separating the plugin's options from the nomad.yaml file2024-03-20T10:08:09ZAdam FeketeSeparating the plugin's options from the nomad.yaml fileIt would be nice to separate ui/plugin options in the plugin and nomad.yaml. It would be nice to define `ui > apps > options ...` in nomad_plugin.yaml.
- options' attribute of ui/plugin could be part of nomad_plugin.yaml file
- it would ...It would be nice to separate ui/plugin options in the plugin and nomad.yaml. It would be nice to define `ui > apps > options ...` in nomad_plugin.yaml.
- options' attribute of ui/plugin could be part of nomad_plugin.yaml file
- it would behave as defaults so you can still override with nomad.yaml
- nomad.yaml would mostly control exclude/include attribute
Use cases:
- @sbrueck 's use case: installing multiple plugins using in a nomad oasis installation. some of the plugins depends on python packages not available in the main image so a custom image needs to built which would contain all plugins and them configuration.
- @afekete 's use case: the support for jupyter notebooks will be implemented as an external parser but should be still part of the core nomad package. It would be nice to use the plugin's yaml file to store its configs instead of putting them as default values of the main configuration.