Move data from mongo to the archive
Currently all of EntryMetadata
is stored in mongodb (in proc.Calc.metadata
). This makes the entries quite big and some bulk operations on mongo are becoming quite slow. This might become even worse, if we think about adding encyclopedia metadata as well.
Only some quantities of EntryMetadata
should be stored in mongo (especially what is editable by the user). The rest should be part of the archive. Downside is that optimade API and indexing elastic will need to read from the archive to work.
Tasks:
-
allow to store filtered EntryMetadata
in archive with working references to optimade (encyclopedia) -
new index function -
adopt processing -
adopt optimade API
Edited by Markus Scheidgen