Skip to content

Build images via CI/CD

Florian Dobener requested to merge ci-cd into develop

This is a preliminary working CI/CD which needs to be fixed for all of our containers (meaning: the Dockerfiles need to be updated to be able to be build via CI/CD!). For now it is fine to be merged. It tags the containers according to the branch or if present tag name (similar to NOMAD). Therefore, we are able to version the containers by introducing git tags and should connect the version to, i.e., nexusutils version numbers and only reference tagged versions there.

I also removed some old folders and moved some files to make the repository cleaner. If this may still be important we can also re-add these files (but they seemed outdated for me).

Follow up issues:

  • Fix all containers to pass CI/CD
  • Enable basic nbmake testing for at least on example file per container
  • Check feasibility of a more complex testing scenario
  • Set an expiration policy (remove older containers if they do not fit in a regex)
  • Set the latest tag to the latest tagged version?
  • Use only versioned containers in tools.json
  • Check webtop baseimage. The choice seems a bit strange (ubuntu-baseimage) to me and maybe should be changed to the versions documented at linuxserver.
  • Use a requirements.in file for python based builds and generate a requirements.txt inside the container to fix the dependencies. There is an example script for generating it with pip-tools in the ellipsometry example.
  • Currently, due to the matrix build I'm using it is not possible to start a test depending on a successful build. We would need a DAG based build for that, which is not compatible with matrix based builds, yet: https://gitlab.com/gitlab-org/gitlab/-/issues/254821. Therefore, all containers need to be build successfully to run the test.

General future considerations:

There will be the option to use a custom tools.json in NOMAD soon: nomad-FAIR!1117 (closed) Therefore, it is possible to disconnect this repository from NOMAD as a submodule and only provide an example tools.json there. Interaction with NOMAD with this repository would then only be via the container registry.

Edited by Florian Dobener

Merge request reports