Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 134
    • Issues 134
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 11
    • Merge requests 11
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

On Thursday, 7th July from 1 to 3 pm there will be a maintenance with a short downtime of GitLab.

  • nomad-lab
  • nomad-FAIRnomad-FAIR
  • Merge requests
  • !434

Merged
Created Oct 25, 2021 by David Sikter@dsikterDeveloper

Introduce CLI for upgrading and migrating mongo data

  • Overview 0
  • Commits 2
  • Pipelines 1
  • Changes 6

First version of CLI command for migrating data from one mongodb to another. The data transformations needed are kept track of by #578 (closed) (all requirements concerning mongo specified in this issue until now are handled by the CLI).

The command takes options for specifying a source and a destination mongo DB, and it offers a dry run mode. It migrates selected uploads and all records related to them (entries and datasets). It currently just overwrites any existing records in the destination DB, but it may be a good idea to add some customizable smartness (for example a "don't overwrite" or "overwrite only if newer" mode, etc).

Uploads can be selected by a query or by a file listing upload IDs. Uploads that have a status saying that they are currently processing are ignored. If an entry is marked as processing though the upload is not (or at least was not at the moment when we read it), that particular upload will be reported as failed (neither the upload nor its entries will be updated). The script also performs various validation checks.

Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: create-dbupgrade-cli