nomad-FAIR issueshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues2021-12-20T07:45:07Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/611Statistics for "marketing"2021-12-20T07:45:07ZMarkus ScheidgenStatistics for "marketing"As an NOMAD PI, I want to generate marketing artefacts that shows the current NOMAD data and usage. Statistics on NOMAD data can be generated via API. Statistics on usage rely on data that is not obtained or stored by NOMAD itself, but t...As an NOMAD PI, I want to generate marketing artefacts that shows the current NOMAD data and usage. Statistics on NOMAD data can be generated via API. Statistics on usage rely on data that is not obtained or stored by NOMAD itself, but that can be acquired depending on how NOMAD is operated.
## data sources
Statistics from NOMAD API:
- how much entries and metadata
- the infamous per code/method chart
- histogram of authors, uploads, growth
Statistics from the NOMAD file system (could be integrated into API)
- how much data on disk (e.g. raw files in TB)
Internal statistics, e.g. from web-server logs
- how much "traffic", DAU, visits on certain parts of the app: encyclopedia, aittoolkit, archive, raw-files
## What kind of artefacts do we want
Depending on the type of statistics/report, we could:
- add a set of statistic components (e.g. to show on the `/about` "dashboard"), also useful to visualize oasis usage
- generated piece of HTML for the web-page (or some iframe stuff)
- generated pdf report
- individual charts as pdfFelix DietrichFelix Dietrichhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1680Annotations as definition properties2024-01-08T07:13:53ZHampus NaesstroemAnnotations as definition propertiesAnnotations are additional key-value pairs that can be added to any metainfo object. In practice however, there are predominantly used on definitions. Them not being definition properties seems to be confusion and creates more issues tha...Annotations are additional key-value pairs that can be added to any metainfo object. In practice however, there are predominantly used on definitions. Them not being definition properties seems to be confusion and creates more issues than it solves.
Topic was changed from the following, based on the discussion below:
### Adding ELN Annotation in Inheriting Section Fails
Trying to add a ELN annotation to an inheriting section using `m_copy()` fails (the NumberEditQuantity is not showing up for entries of type `B`):
```python
class A(EntryData):
some_property = Quantity(type=float)
class B(A):
some_property = A.some_property.m_copy()
some_property.a_eln = ELNAnnotation(component=ELNComponentEnum.NumberEditQuantity)
```
This does work for specifying the type of a sub section.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1014Remove domain from parsers2022-09-01T07:16:00ZMarkus ScheidgenRemove domain from parsersThis is obsolete and not used anymore. These kinda *entry types* are now differentiated by different methods in the section results. We should remove the domain attribute from the parsers (and potentially the search index and metadata)This is obsolete and not used anymore. These kinda *entry types* are now differentiated by different methods in the section results. We should remove the domain attribute from the parsers (and potentially the search index and metadata)https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1710Refactor normalizing workflow tests2023-12-21T15:58:56ZAlvin Noe LadinesRefactor normalizing workflow testsSome tests under normalizing/test_workflow.py should technically be in datamodel/metainfo/test_workflow.py.
It is understandable that the two are confused but I personally think that all tests regarding the metainfo
defs including the im...Some tests under normalizing/test_workflow.py should technically be in datamodel/metainfo/test_workflow.py.
It is understandable that the two are confused but I personally think that all tests regarding the metainfo
defs including the implementation of normalize should be under the latter. Only normalizations done in
nomad/normalizing/workflow.py should be in the other.
@pizarrojAlvin Noe LadinesAlvin Noe Ladineshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1728Full refactoring of run into data2024-03-05T09:32:17ZJose PizarroFull refactoring of run into data@ndaelman @jrudz , and also @ladinesa @himanel1 @mscheidg
We were talking about a full refactoring of the section `run` into `data`. The idea is to slowly build the metadata schema for `data` maintaining the three main sections: `System...@ndaelman @jrudz , and also @ladinesa @himanel1 @mscheidg
We were talking about a full refactoring of the section `run` into `data`. The idea is to slowly build the metadata schema for `data` maintaining the three main sections: `System`, `Method` and `Calculation`.
The goal is to have **`normalize()` specifically in each of the main sections** (with specific defined functions inside), and tackle normalization there (instead of in `SystemNormalizer`, `MethodNormalizer` and all the potential list of normalizers for Calculation).
Furthermore, in this refactoring **we will heavily clean-up the sub-sections and quantities currently deprecated, unused or wrongly defined** in the metainfo.
Lastly, **we should also refactor the plugin projects structure** (as Pavel pointed out and as we know from the EELSDB parser, it is a bit confusing). An open question here: how much of the current parsers we want to maintain? IMO: we should deprecate some of them, as they are actually empty or only populating very simple metadata.
_____
As a dynamic (we can modify this at will):
- [x] Start with `System` and slowly build the quantities needed to define it.
- [ ] After that and add extensive testing, go into `Method`.
- [ ] Finally, we take care of `Calculation`. This is the most heavily refactored one.
Some simple rules on developing:
- Add extensive descriptions in the normalize functions. Be verbose on which quantities are _minimal_ and which ones are _extra_.
- Always add typing for passing arguments to functions.
- Be as general as possible. One of the current problems on `run` is that some quantities are a bit restrictive because of the packing that was done before; in other words, keep as much as possible a clean class structures (e.g., avoid the `Dos` or `BandStructure` current structure).
- Always ask for feedback on some unknown quantities.