Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 140
    • Issues 140
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 16
    • Merge requests 16
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-lab
  • nomad-FAIRnomad-FAIR
  • Issues
  • #298

Closed
Open
Created Mar 06, 2020 by Markus Scheidgen@mscheidgOwner4 of 4 tasks completed4/4 tasks

Move data from mongo to the archive

Currently all of EntryMetadata is stored in mongodb (in proc.Calc.metadata). This makes the entries quite big and some bulk operations on mongo are becoming quite slow. This might become even worse, if we think about adding encyclopedia metadata as well.

Only some quantities of EntryMetadata should be stored in mongo (especially what is editable by the user). The rest should be part of the archive. Downside is that optimade API and indexing elastic will need to read from the archive to work.

Tasks:

  • allow to store filtered EntryMetadata in archive with working references to optimade (encyclopedia)
  • new index function
  • adopt processing
  • adopt optimade API

image

Edited Mar 24, 2020 by Markus Scheidgen
Assignee
Assign to
Time tracking