Skip to content

GitLab

Closed
Created by Markus Scheidgen@mscheidgOwner4 of 4 tasks completed4/4 tasks

Move data from mongo to the archive

Currently all of EntryMetadata is stored in mongodb (in proc.Calc.metadata). This makes the entries quite big and some bulk operations on mongo are becoming quite slow. This might become even worse, if we think about adding encyclopedia metadata as well.

Only some quantities of EntryMetadata should be stored in mongo (especially what is editable by the user). The rest should be part of the archive. Downside is that optimade API and indexing elastic will need to read from the archive to work.

Tasks:

  • allow to store filtered EntryMetadata in archive with working references to optimade (encyclopedia)
  • new index function
  • adopt processing
  • adopt optimade API

image

Edited by Markus Scheidgen