Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 218
    • Issues 218
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 31
    • Merge requests 31
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-labnomad-lab
  • nomad-FAIRnomad-FAIR
  • Issues
  • #1037
Closed
Open
Issue created Sep 06, 2022 by Markus Scheidgen@mscheidgOwner

Align metainfo with nexus even more

We still have some nexus situations that are hard to map to nomad even with #837 (closed) done.

We try to press fields into quantities. But fields might contain so many additional data: the actual name, the actual unit used, multiple values, etc. It might be the easiest to differentiate between a simple quantity and a complex quantity. Simple quantities can be serialised with just the value and the definition provides all context, complex quantities are serialized as objects with value and additional metadata.

Now we can put more functions on the definitions that will require additional metadata in the data, like variable, attributes, dimensionality, repeats (fields that can occures multiple times).

Here is some pseudo code that shows the difference between simple and complex serialisation:

q.is_complex = q.variable or q.attributes or q.repeats or (q.dimensionality and not q.unit)
if not q.is_complex:
  q.json = q.value
else:
  q.json = [
    {
      m_source_name = 'realName1',
      m_source_unit = 'A',
      m_value = q.value,
      attr_1 = 'some value'
    },
    {
      m_source_name = 'realName2',
      m_source_unit = 'm',
      m_value = q.value
    }    
  ]

This should be backwards compatible. The archive browser needs to support the complex serialisation as well. It should be checked if this would somehow affects the archive API.

With both serializations at the same time, the serialization would stay lean if possible and be more verbose if necessary.

Other benefits: this would also make us more flexible in the future as more and more metadata can be added to quantities. The complex serialisation is more in line how hdf5 works. We could get rid of the more hacky attribute, source_unit, source_name @-based syntax.

Assignee
Assign to
Time tracking