Skip to content

Introduce CLI for upgrading and migrating mongo data

David Sikter requested to merge create-dbupgrade-cli into v1.0.0

First version of CLI command for migrating data from one mongodb to another. The data transformations needed are kept track of by #578 (closed) (all requirements concerning mongo specified in this issue until now are handled by the CLI).

The command takes options for specifying a source and a destination mongo DB, and it offers a dry run mode. It migrates selected uploads and all records related to them (entries and datasets). It currently just overwrites any existing records in the destination DB, but it may be a good idea to add some customizable smartness (for example a "don't overwrite" or "overwrite only if newer" mode, etc).

Uploads can be selected by a query or by a file listing upload IDs. Uploads that have a status saying that they are currently processing are ignored. If an entry is marked as processing though the upload is not (or at least was not at the moment when we read it), that particular upload will be reported as failed (neither the upload nor its entries will be updated). The script also performs various validation checks.

Merge request reports