Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 214
    • Issues 214
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 28
    • Merge requests 28
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-labnomad-lab
  • nomad-FAIRnomad-FAIR
  • Merge requests
  • !409

Refactoring metadata: using a consistent flat structure in MongoDB

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged David Sikter requested to merge flatten-mongo-fields into v1.0.0 Oct 07, 2021
  • Overview 2
  • Commits 16
  • Pipelines 2
  • Changes 33

Getting rid of Calc.metadata and instead storing the fields directly on the Calc object, plus some minor name changes for consistency and cleanup of old code.

Metadata refactoring:

  • Dataset.name -> Dataset.dataset_name
  • Dataset.created -> Dataset.dataset_create_time
  • Dataset.modified -> Dataset.dataset_modified_time
  • Calc.metadata.last_processing -> Calc.last_processing_time
  • Calc.metadata.last_edit -> Calc.last_edit_time
  • Fields calc_hash, pid, last_processing_time, last_edit_time, external_id, nomad_version, nomad_commit, external_db, comment, references, coauthors, shared_with, datasets are moved from Calc.metadata to Calc
  • Calc.metadata (and Calc.metadata.raw_id) are removed

Additionally:

  • Metadata files are only read when an entry is first processed, not when reprocessing or when importing bundles. Note, if the initial processing fails we will not try to read the metadata files again. Maybe this could be smarter...
  • Functionality connected to import of metadata from oasis metadata files is removed. Testing of new bundle-based oasis export/import is extended.
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: flatten-mongo-fields