Remove config.fs.tmp and use a fix subdir of config.fs.staging instead
With the new incremental upload, we are extracting files to config.fs.tmp
and then potentially mv those to config.fs.staging
. Depending on the deployment, those directories might be on different volumes/mounts. Move would be actual a copy+delete. Also this might cause issues, e.g. cannot mv files across volumes in k8s/docker (Invalid cross-device link
, mv: setting attribute 'security.selinux' for 'security.selinux': Permission denied
, mv: listing attributes of 'staging/test': Cannot allocate memory
).
Anyhow, config.fs.tmp
is exclusively used for staging purposes. Instead of having it configurable as a separate directory, we could use a fixed tmp
sub-directory of config.fs.staging
and remove config.fs.tmp
. In the mean time, we configure config.fs.tmp
to be a sub-dir of config.fs.staging
on our clusters.
Further there is a config.fs.local_tmp
parameter that is not used.
We should get rid of both config.fs.local_tmp
and config.fs.tmp
.