Draft: Nomad link to send installation information from local to central

Closed Daniel Lehmberg requested to merge 820_nomad_federation_api into develop

Relates to #820 (closed) and #886 (substitues MR !697 (closed))

There is also an alternative implementation which overcomes some of the issues raised in the code review -- !784 (closed)

So far I set up the following structure with stub methods/classes:

  • SubmitInstallationInfoToCentral is currently the class to frequently send data from local to central (it is used in the FastAPI middleware submit_installation_info)
  • The module (/ FastAPI router) federation.py contains the REST/POST where the local Nomad installation sends the data to / central Nomad receives the data.
  • The InstallationInfo class specifies the content and format that is sent from local to central.
  • in test_federation.py there are two tests and conftest.py (at same level) includes fixtures to set up the testing for federation

To clarify (for myself to keep track, but also happy to get comments/hints on these issues):

  • Could there be a race condition on the "last_timestamp" in SubmitInstallationInfoToCentral?
  • Is there a flag to distinguish between the local and central OASIS? This way I would only provide the landscape/POST on the central oasis. (not yet, for now also provide the federation API on local installations)
  • How to get all of the data to submit? oasis id [OK], oasis url [OK], oasis maintainer [OK], nomad version [OK], and a few "telemetry" data [OPEN]
  • How to use mongodb (to be used within federation.py) to store the data at central Nomad?
  • RuntimeError('This event loop is already running') still appears when not using the nest_asyncio package (the import is limited to test_federation.py only though)
  • option to turn feature on/off?
  • What if the frequency to submit data is set to very low in the configuration? Set a minimum value in the source code and make a warning?
Edited by Daniel Lehmberg

Merge request reports