Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • nomad-FAIR nomad-FAIR
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 217
    • Issues 217
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 28
    • Merge requests 28
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • nomad-labnomad-lab
  • nomad-FAIRnomad-FAIR
  • Issues
  • #616
Closed
Open
Issue created Sep 26, 2021 by Markus Scheidgen@mscheidgOwner

Use pydantic settings management for nomad.config

Pydantic has build in support for settings management. This supports everything that our NomadConfig-obj system provides (but better):

  • nested config objects
  • override from first env (limited), than config file, than default values

Here is an example of how it could work:

from typing import Dict, Any
from pydantic import BaseModel, Field, BaseSettings, HttpUrl
import yaml
import os.path
import os


class Client(BaseModel):
    user: str = Field('test', description='The username.')
    password: str = '*'
    url: HttpUrl = Field('http://nomad-lab.eu')


class NomadConfig(BaseSettings):
    client: Client = Client()
    not_nested: str = 'hello'

    class Config:
        env_prefix = 'nomad_'
        case_sensitive = False

        @classmethod
        def customise_sources(
                cls,
                init_settings,
                env_settings,
                file_secret_settings):

            return (
                init_settings,
                env_settings,
                yaml_config_settings_source, file_secret_settings)


def yaml_config_settings_source(settings: BaseSettings) -> Dict[str, Any]:
    if not os.path.exists('nomad.yaml'):
        return {}

    try:
        with open('nomad.yaml') as f:
            data = yaml.load(f, Loader=yaml.FullLoader)
        if data is None:
            return {}
        return data
    except yaml.YAMLError as e:
        raise e


config = NomadConfig()

The support of env vars is limited as they only apply to the first level (e.g. NomadConfig). To set nested fields, the top level env var has to be set with a json value:

export NOMAD_CLIENT='{"password":"mypassword"}'

If necessary this could be replaced by a custom settings source.

Assignee
Assign to
Time tracking