Skip to content

New sync api

David Sikter requested to merge new-sync-api into v0.11.0

Introduces new functionality for downloading and uploading "upload bundles" from/to NOMAD deployments. Intended to replace existing CLI and publish-from-oasis functionality.

An "upload bundle" is a file bundle (a zipfile, normally) containing all information (raw files, archive files, metadata) needed for transferring an upload from one NOMAD deployment to another. They are also intended for internal operations and for future CLI commands for doing things like "pulling" all new uploads from the central NOMAD to an oasis.

The main methods for exporting and importing are defined in nomad.processing.Upload, named export_bundle and import_bundle, respectively. They are quite parametrized (maybe too parametrized?) to allow for flexibility. The import method does more extensive checking than the existing functionality for importing uploads from OASIS. In the future, we might also want to include code for conversion between different nomad versions.

Merge request reports