Optimisations
io
-
(1) skip bagit -
(1) pack rawfiles directly into zips -
(1) allow staging and public buckets to be on different fs -
(3) remove package upload zips after successful migration, configure what successful means (e.g. no failures or missing calcs)
computing related
- (not doing this) copy system normalisation if likely to be the same
-
system normalisation only for some systems with old heuristics
parallelisation
-
(1) complete/test package migration parallelisation -
(1) catch/backoff/retry API Gateway timeouts (the api is just overworked) -
rollback publish in repo db, if publish failed for errors
analysis
-
(2) add 'source->target' value to quantity diff log message -
(3) lower more of the log levels -
(1) rethink errors for migration log output