diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bc3c5a8efff1395edc3b3c708fd890516a6728f6..10717081f2fe744666e57070cc6c0104004f7738 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,5 +1,7 @@ # Contributing +See also the more detailed [contributing guide](https://nomad-lab.eu/docs/develop/contrib.html) in the documentation. + We maintain two git repositories that you can contribute to. The default choice for external contributors should be the [GitHUB project](https://github.com/nomad-coe/nomad) project. Please you the normal [GitHub-Flow](https://docs.github.com/get-started/quickstart/github-flow) targeting the `develop` branch. diff --git a/README.md b/README.md index 94026abcf0bf81d53864aa14f648a288a53afb34..98258a70cc29f91412242df157b0386acfd8df5e 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ customized NOMAD version and with their own compute and storage resources. ## Contributing +See also the more detailed [contributing guide](https://nomad-lab.eu/docs/develop/contrib.html) in the documentation. + There are two forks of this repository, one on **GitHUB** and one on MPCDF's **GitLab**. NOMAD's [GitHUB project](https://github.com/nomad-coe/nomad) always contains the current `develop` branch. @@ -39,6 +41,17 @@ Omitted versions are plain bugfix releases with only minor changes and fixes. Th file [`CHANGELOG.md`](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/blob/develop/CHANGELOG.md) contains much more detailed information about changes and fixes in the released versions. +### v1.2.0 +- Restructured documentation +- More how-to guides and learning material +- The NOMAD plugin mechanism +- Initial integration of external ELNs +- Many improvement to the computational materials science parsers +- Log-transfer from OASIS to the public NOMAD infrastructure +- Revised archive storage format +- Workflow model and visualizer, updated computational materials science workflow. +- More flexible Python dependencies to install nomad-lab in various environments + ### v1.1.9 - ELN improvements for safer file handling - many smaller fixes and changes diff --git a/docs/develop/code.md b/docs/develop/code.md new file mode 100644 index 0000000000000000000000000000000000000000..db475e7fce3d5c1f8ff1acdd269956d0f7b431c9 --- /dev/null +++ b/docs/develop/code.md @@ -0,0 +1,73 @@ +NOMAD is a complex project with lots of parts. This guide gives you a rough overview +about the code-base and ideas to what to look at first. + +## Git Projects + +There is one [main NOMAD project](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR) +(and its [fork on GitHub](https://github.com/nomad-coe/nomad)). This project contains +all the framework and infrastructure code. It instigates all checks, builds, and deployments +for the public NOMAD service, the NOMAD Oasis, and the `nomad-lab` Python package. All +contributions to NOMAD have to go through this project eventually. + +All (Git) projects that NOMAD depends on are either a Git submodule (you find +them all in the `/dependencies` directory or its subdirectories) or they are +listed as PyPI packages in the `/pyproject.toml` of the main project (or one of its +submodules). + +You can also have a look at the [list of parsers](../reference/parsers.md) and [built-in plugins](../reference/plugins.md) +that constitute the majority of these projects. The only other projects are +[matid](https://github.com/SINGROUP/matid), [density of state fingerprints](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-dos-fingerprints), and the [NOMAD Remote Tools Hub tools](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub). + +!!! note + The GitLab organization [nomad-lab](https://gitlab.mpcdf.mpg.de/nomad-lab) and the GitHub organizations for [FAIRmat](https://github.com/fairmat-nfdi) and + the [NOMAD CoE](https://github.com/nomad-coe) all represent larger infrastructure and research projects, and + they include many other Git projects that are not related. When navigating the + code-base only follow the submodules. + +## Python code + +There are three main directories with Python code: + +- `/nomad`: The actual NOMAD code. It is structured into more subdirectories and modules. +- `/tests`: Tests ([pytest](https://docs.pytest.org)) for the NOMAD code. It follows the same module structure, but Python files +are prefixed with `test_`. +- `/examples`: This contains a few small Python scripts that might be linked in the documentation. + +The `/nomad` directory contains the following "main" modules. This list is not extensive, but +should help you to navigate the code base: + +- `config`: NOMAD is configured through the `nomad.yaml` file. This contains all the ([pydantic](https://docs.pydantic.dev/)) models and default config parameters. +- `utils`: Somewhat self-explanatory; notable features: the structured logging system ([structlog](https://www.structlog.org/)) and id generation and hashes. +- `units`: The unit and unit conversion system based on [Pint](https://pint.readthedocs.io). +- `app`: The [fast-api](https://fastapi.tiangolo.com/) APIs: v1 and v1.2 NOMAD APIs, [OPTIMADE](https://www.optimade.org/), [DCAT](https://www.w3.org/TR/vocab-dcat-2/), [h5grove](https://github.com/silx-kit/h5grove), and more. +- `cli`: The command line interface (based on [click](https://click.palletsprojects.com)). Subcommands are structured into submodules. +- `parsing`: The base classes for parsers, matching functionality, parser initialization, some fundamental parsers like the *archive* parser. See also the docs on [processing](../learn/basics.md#parsing). +- `normalizing`: All the normalizers. See also the docs on [processing](../learn/basics.md#normalizing). +- `metainfo`: The metainfo system, e.g. the schema language that NOMAD uses. +- `datamodel`: The built-in schemas (e.g. `nomad.datamodel.metainfo.simulation` used by all the theory parsers). The base sections and section for the shared entry structure. See also the docs on the [datamodel](../learn/data.md) and [processing](../learn/basics.md). +- `search`: The interface to [elasticsearch](https://www.elastic.co/guide/en/enterprise-search/current/start.html). +- `processing`: Its all about processing uploads and entries. The interface to [celery](https://docs.celeryq.dev/en/stable/) and [mongodb](https://www.mongodb.com). +- `files`: Functionality to main the files for uploads in staging and published. The interface to the file system. +- `archive`: Functionality to store and access archive files. This is the storage format for all processed data in nomad. See also the docs on [structured data](../learn/data.md). + +## GUI code + +The NOMAD UI is written as a [react](https://react.dev/) single page application (SPA). It uses (among many other libraries) +[MUI](https://mui.com/), [plotly](https://plotly.com/python/), and [D3](https://d3js.org/). The GUI code is maintained in the `/gui` directory. Most relevant code +can be found in `/gui/src/components`. The application entrypoint is `/gui/src/index.js`. + +## Documentation + +The documentation is based on [MkDocs](https://www.mkdocs.org/). The important files +and directories are: + +- [/docs](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/tree/develop/docs): Contains all the markdown files that contribute to the documentation system. +- [/mkdocs.yml](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/tree/develop/mkdocs.yml): The index and configuration of the documentation, new files have to be added here as well. +- [/nomad/mkdocs.py](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/tree/develop/nomad/mkdocs.py): Python code that defines [macros](https://mkdocs-macros-plugin.readthedocs.io/) that can be +used in markdown. + +## Other top-level directories + +- `dependencies`: Contains all the submodules (e.g. the parsers). +- `ops`: Contains artifacts to run NOMAD components, e.g. docker-compose.yaml files and our kubernetes helm chart. +- `scripts`: Contains scripts used during build or for certain development tasks. \ No newline at end of file diff --git a/docs/develop/contrib.md b/docs/develop/contrib.md new file mode 100644 index 0000000000000000000000000000000000000000..9d568a92b01205242b0fb18f841d9c08bd6580d6 --- /dev/null +++ b/docs/develop/contrib.md @@ -0,0 +1,282 @@ +!!! note + + The NOMAD source-code is maintained in two synchronized projects on [GitHub](https://github.com/nomad-coe/nomad) and + a [GitLab run by MPCDF](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR). + Everyone can contribute on GitHub. The GitLab requires an account for active contribution. + This not an ideal situation: there are historic reasons and there is + a lot of buy-in into the GitLab CI/CD system. This guide addresses contributions + to both projects. + +## Issues + +### Issue trackers + +Everyone can open an issue in our main [GitHub project](https://github.com/nomad-coe/nomad): + +- [https://github.com/nomad-coe/nomad/issues/new](https://github.com/nomad-coe/nomad/issues/new) + +Use issues to ask questions, report bugs, or suggest features. If in doubt, use the main project +to engage with us. If you address a specific +plugin (e.g. parser), you can also post into the respective projects. See also the +list of [parsers](../reference/parsers.md) and list of [built-in plugins](../reference/plugins.md). + +If you are a member of FAIRmat, the NOMAD CoE, or are a close collaborator, you +probably have an MPCDF GitLab account (or should ask us for one). Please use the +issue tracker on our main [GitLab project](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR). +This is where most of the implementation work is planned and executed. + +### Issue content +A few tips that will help us to solve your issues quicker: + +- Focus on the issue. You don't need to greet us or say thx and goodbye. Let's keep it technical. +- Use descriptive short issue titles. Use specific words over general words. +- Describe the observed problem and not the assumed causes. Clearly separate speculation from the +problem description. +- **Bugs**: think how we could re-produce the problem: + + - What nomad URL are you using (UI), which package version (Python)? + - Is there an upload or entry id that we can look at? + - Example files or code snippets? + - Don't screenshot code, copy&paste please. + +- **Features**: augment your feature descriptions with a use-case that helps us understand the feature and its scope. + +### Issues labels (GitLab) + +On the main GitLab project, there are three main categories for labels. Ideally each issue gets one of each category: + +- The *state* labels (grey). Those are used to manage when and how the issue is addressed. +This should be given the NOMAD team member who is currently responsible to moderate the +development. If its a simple fix and you want to do it yourself, assign yourself and use +"bugfixes". + +- The *component* label (purple). These denote the part of the NOMAD software that is +most likely effected. Multiple purple labels are possible. + +- The *kind* label (red). Wether this is a bug, feature, refactoring, or documentation issue. + +Unlabeled issues will get labeled by the NOMAD team as soon as possible. You can provide +labels yourself. + +## Documentation + +The documentation is part of NOMAD's main source code project. You can raise issues +about the documentation in the usual way (see [above](#issues)). To make changes, +create a [merge](#merge-requests-gitlab) or [pull](#pull-requests-github) request. + +See also the [documentation part](./code.md#documentation) in our code navigation guide. + +## Plugins + +Also read the [how-to develop, publish, and distribute plugins guide](../plugins/plugins.md). + +### Built-in plugins (and submodules) + +Most plugins that are maintained by the NOMAD team are built-in plugins (e.g. all the parsers). +These plugins are also available on the public NOMAD service. + +These plugins are tight to the main project's source-code via submodules. They are +included in the built and therefore automatically distributed as part of the +NOMAD docker images and Python package. + +To contribute to these plugins, use the respective GitHub projects. See also the +list of [parsers](../reference/parsers.md) and list of [built-in plugins](../reference/plugins.md). +The same rules apply there. A merge request to the main project will also be required to update the +submodule. + +All these submodules are placed in the `/dependencies` directory. After merging or checking out, +you have to make sure that the modules are updated to not accidentally commit old +submodule commits again. Usually you do the following to check if you really have a +clean working directory. + +```sh + git checkout something-with-changes + git submodule update --init --recursive + git status +``` + +### 3rd-party plugins + +Please open an issue, if you want to announce a plugin or contribute a plugin as a +potential built-in plugin (e.g. that is part of the public NOMAD service). + +## Branches and Tags + +On the [main GitLab project](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR), we use *protected* and *feature* branches. +You must not (even if you have the rights) commit to protected branches directly. + +- `develop`, a *protected* branch and the *default* branch. It contains the latest, potentially unreleased, features and fixes. +This is the main working branch. +- `master`, a *protected* branch. Represents the latest stable +release (usually what the current official NOMAD runs on). + +- *feature branches*, this is where you work. Typically they are automatically +(use the *create merge request* button) named after issues: `<issue-number>-<issue-title>`. + +- `vX.X.X` or `vX.X.XrcX` *tags* for (pre-)releases. + +The `develop` branch and release tags are automatically synchronized to the [GitHub project](https://github.com/nomad-coe/nomad). +Otherwise, this project is mostly the target for [pull requests](#pull-requests-github) +and does not contain other relevant branches. + +## Merge requests (GitLab) + +### Create the MR + +Ideally have an issue first and create the merge request (and branch) in the +GitLab UI. There is an *create merge request* button on each issue. +When done manually, branches should be based on `develop` and merge request should target `develop` as well. + +### Commit + +Make sure, you follow our [coding guide-lines](./guides.md) and [setup your IDE](./setup.md#setup-your-ide) to enforce +stylechecks, linting, and static analysis. You can also run [tests locally](./setup.md#running-tests). +Try to keep a clean commit history and follow our [Git tips](#git-tips-for-a-clean-history). + +Usually only one person is working on a feature branch. Rebasing, amendments, and force +pushes are allowed. + +### Changelog +We have an automatically generated changelog in the repository file `CHANGELOG.md`. +This changelog is produced from commit messages and to maintain this file, you +need to write commit messages accordingly. + +To trigger a changelog entry, your commit needs to end with a so called *git trailer* +called `Changelog`. A typical commit message for a changelog entry should look like this: + +``` +A brief one line title of the change. + +A longer *markdown* formatted description of the change. Keep in mind that gitlab +will automatically link the changelog entry with this commit and a respective merge +requests. You do not need to manually link to any gitlab resources. + +This could span multiple paragraphs. However, keep it short. Documentation should +go into the actual documentation, but you should mention breaks in backward compatibility, +deprecation of features, etc. + +Changelog: Fixed +``` + +The trailer value (`Fixed` in the example) has to be one of the following values: + +- `Fixed`, for bugfixes. +- `Added`, for new features. +- `Changed`, for general improvements, e.g. updated documentation, refactoring, +improving performance, etc. + +These categories are consistent with [keepachangelog.com](https://keepachangelog.com/). +For more information about the changelog generation read the [gitlab documentation](https://docs.gitlab.com/ee/api/repositories.html#add-changelog-data-to-a-changelog-file). + +### CI/CD pipeline and review + +If you push to your merge requests, GitLab will run an extensive CI/CD pipeline. +You need to pay attention to failures and resolve them before review. + +To review GUI changes, you can deploy your branch to the dev-cluster via CI/CD actions. + +Find someone on the NOMAD developer team to review your MR and request a review through +GitLab. The review should be performed shortly and should not stall the MR longer than +two full work days. + +The reviewer needs to be able to learn about the merge requests and what it tries +to achieve. This information can come from: + +- the linked issue +- the merge request description (might contain your commit message) and your comments +- threads that were open by the author to attach comments to specific places in the code + +The reviewer will open *threads* that need to be solved by the MR author. If all +threads are resolved, you can re-request a review. + +For complex merge requests, you can also comment your code and create *threads* for +the reviewer to resolve. This will allow you to explain your changes. + +### Merge + +The branch should be recently rebase with develop to allow a smooth merge: + +``` +git fetch +git rebase origin/develop +git push origin <branch> -f +``` + +The merge is usually performed by the merge request author after a positive review. + +If you could not keep a clean Git history with your commits, squash the merge request. +Make sure to added the commit messaged when squashing to have a [changelog](#changelog) entry. + +Make sure to delete the branch on merge. The respective issue usually closes itself, due +to the references put in by GitLab. + +## Pull requests (GitHub) + +You can fork the [main NOMAD project](https://github.com/nomad-coe/nomad) and create pull requests following the usual +GitHub flow. Make sure to target the `develop` branch. A team member will pickup your +pull request and automatically copy it to GitLab to run the pipeline and potentially +perform the merge. This process is made transparent in the pull request discussion. Your +commits and your authorship is maintained in this process. + +Similar rules apply to all the parser and plugin projects. Here pipelines are run +directly on GitHub. A team member will review and potentially merge your pull requests. + +## Git tips for a clean history + +It is often necessary to consider code history to reason about potential problems in +our code. This can only be done, if we keep a "clean" history. + +- Use descriptive commit messages. Use simple verbs (*added*, *removed*, *refactored*, etc.) +name features and changed components. [Include issue numbers](https://docs.gitlab.com/ee/user/project/issues/crosslinking_issues.html) +to create links in gitlab. + +- Learn how to amend to avoid lists of small related commits. + +- Learn how to rebase. Only merging feature-branches should create merge commits. + +- Squash commits when merging. + +- Some videos on more advanced Git usage: https://youtube.be/Uszj_k0DGsg, https://youtu.be/qsTthZi23VE + +### amend +While working on a feature, there are certain practices that will help us to create +a clean history with coherent commits, where each commit stands on its own. + +```sh + git commit --amend +``` + +If you committed something to your own feature branch and then realize by CI that you have +some tiny error in it that you need to fix, try to amend this fix to the last commit. +This will avoid unnecessary tiny commits and foster more coherent single commits. With *amend* +you are basically adding changes to the last commit, i.e. editing the last commit. If +you push, you need to force it `git push origin feature-branch --force-with-lease`. So be careful, and +only use this on your own branches. + +### rebase +```sh + git rebase <version-branch> +``` + +Lets assume you work on a bigger feature that takes more time. You might want to merge +the version branch into your feature branch from time to time to get the recent changes. +In these cases, use rebase and not merge. Rebase puts your branch commits in front of the +merged commits instead of creating a new commit with two ancestors. It basically moves the +point where you initially branched away from the version branch to the current position in +the version branch. This will avoid merges, merge commits, and generally leave us with a +more consistent history. You can also rebase before creating a merge request, which basically +allows no-op merges. Ideally the only real merges that we ever have, are between +version branches. + + +### squash +```sh + git merge --squash <other-branch> +``` + +When you need multiple branches to implement a feature and merge between them, try to +use *squash*. Squashing basically puts all commits of the merged branch into a single commit. +It basically allows you to have many commits and then squash them into one. This is useful +if these commits were made just to synchronize between workstations, due to +unexpected errors in CI/CD, because you needed a save point, etc. Again the goal is to have +coherent commits, where each commits makes sense on its own. diff --git a/docs/develop/gitlab.md b/docs/develop/gitlab.md deleted file mode 100644 index 3457afce65e196234302d2b7c40744217a07f872..0000000000000000000000000000000000000000 --- a/docs/develop/gitlab.md +++ /dev/null @@ -1,196 +0,0 @@ ---- -title: Using Git/GitLab ---- - -## Branches - -We use *protected* and *feature* branches. You must not (even if you have -the rights) commit to it directly. - -- `master`, a *protected* branch and also the *default* branch. Represents the latest stable -release (i.e. what the current official NOMAD runs on). -- `develop`, a quasi *protected* branch. It contains the latest, potentially -unreleased, features and fixes. -- *feature branches*, this is where you work. Typically they are automatically -named after issues: `<issue-number>-<issue-title>`. -- `vX.X.X`, tags for releases. - - -## Flow: From issue to merge - -### Create issues - -Everyone with an MPCDF GitLab account can create issues. - -- Use descriptive short titles. Specific words over general works. Try to describe -the problem and do not assume causes. -- You do not need to greet us or say thx and goodbye. Let's keep it purely technically. -- For bugs: think about how to re-produce the problem and provide examples when applicable. -- For features: in addition to the feature descriptive, try to provide a use-case that -might help us understand the feature and its scope. -- You can label the issue if you know the label system. - -### Labeling issues - -There are three main categories for labels. Ideally each issue gets one of each category: - -- The *state* labels (grey). Those are used to manage when and how the issue is addressed. -This should be given the NOMAD team member who is currently responsible to moderate the -development. If its a simple fix and you want to do it yourself, assign yourself and use -"bugfixes". - -- The *component* label (purple). This denote the part of the NOMAD software that is -most likely effected. Multiple purple labels are possible. - -- The *kind* label (key). Wether this is a bug, feature, refactoring, or documentation issue. - -Unlabeled issues will get labeled by the NOMAD team as soon as possible. - - -### Working on an issue - -Within the development team and during development meetings we decide who is acting on -an issue. The person is assigned to the issue on GitLab. It typically switches its grey *state* label -from *backlog* to *current* or *bugfixes*. The assigned person is responsible to further -discuss the problem with the issue author and involve more people if necessary. - -To contribute code changes, you should create a branch and merge request (MR) from the -GitLab issue via the button offered by GitLab. This will create a branch of *develop* and -create a merge request that targets *develop*. - -You can work on this branch and push as often as you like. Each push will cause a pipeline -to run. The solution can be discussed in the merge request. - - -### Code review and merge - -When you are satisfied with your solution, and your CI/CD pipeline passes you can mark your MR as *ready*. -Make sure that the `delete` option is checked to automatically remove you branch after -merge. In most cases you should also check `squash` commits. This will replace all the -commits in the MR with a single *squash* commit. If you do not want to *squash* your -branch, make sure that you produce a reasonable and [clean version history](#clean-version-history). - -To review GUI changes, you should deploy your branch to the dev-cluster via CI/CD actions. -Find someone on the NOMAD developer team to review your MR and request a review through -GitLab. The review should be performed shortly and should not stall the MR longer than -two full work days. - -The reviewer will open *threads* that need to be solved by the MR author. If all -threads are resolved, you can re-request a review. The reviewer should eventually merge -the MR. Typically we squash MRs to keep the revision history short. -This will typically auto-close the issue. - -## Changelog - -We have an automatically generated changelog in the repository file `CHANGELOG.md`. -This changelog is produced from commit messages and to maintain this file, you -need to write commit messages accordingly. - -To trigger a changelog entry, your commit needs to end with a so called *git trailer* -called `Changelog`. A typical commit message for a changelog entry should look like this: - -``` -A brief one line title of the change. - -A longer *markdown* formatted description of the change. Keep in mind that gitlab -will automatically link the changelog entry with this commit and a respective merge -requests. You do not need to manually link to any gitlab resources. - -This could span multiple paragraphs. However, keep it short. Documentation should -go into the actual documentation, but you should mention breaks in backward compatibility, -deprecation of features, etc. - -Changelog: Fixed -``` - -The trailer value (`Fixed` in the example) has to be one of the following values: - -- `Fixed`, for bugfixes. -- `Added`, for new features. -- `Changed`, for general improvements, e.g. updated documentation, refactoring, -improving performance, etc. - -These categories are consistent with (keepachangelog.com)[https://keepachangelog.com/]. -For more information about the changelog generation read the [gitlab documentation](https://docs.gitlab.com/ee/api/repositories.html#add-changelog-data-to-a-changelog-file). - - -## Clean version history - -It is often necessary to consider code history to reason about potential problems in -our code. This can only be done, if we keep a "clean" history. - -- Use descriptive commit messages. Use simple verbs (*added*, *removed*, *refactored*, etc.) -name features and changed components. [Include issue numbers](https://docs.gitlab.com/ee/user/project/issues/crosslinking_issues.html) -to create links in gitlab. - -- Learn how to amend to avoid lists of small related commits. - -- Learn how to rebase. Only merging feature-branches should create merge commits. - -- Squash commits when merging. - -- Some videos on more advanced git usage: https://youtube.be/Uszj_k0DGsg, https://youtu.be/qsTthZi23VE - -### amend -While working on a feature, there are certain practices that will help us to create -a clean history with coherent commits, where each commit stands on its own. - -```sh - git commit --amend -``` - -If you committed something to your own feature branch and then realize by CI that you have -some tiny error in it that you need to fix, try to amend this fix to the last commit. -This will avoid unnecessary tiny commits and foster more coherent single commits. With *amend* -you are basically adding changes to the last commit, i.e. editing the last commit. If -you push, you need to force it `git push origin feature-branch --force-with-lease`. So be careful, and -only use this on your own branches. - -### rebase -```sh - git rebase <version-branch> -``` - -Lets assume you work on a bigger feature that takes more time. You might want to merge -the version branch into your feature branch from time to time to get the recent changes. -In these cases, use rebase and not merge. Rebase puts your branch commits in front of the -merged commits instead of creating a new commit with two ancestors. It basically moves the -point where you initially branched away from the version branch to the current position in -the version branch. This will avoid merges, merge commits, and generally leave us with a -more consistent history. You can also rebase before creating a merge request, which basically -allows no-op merges. Ideally the only real merges that we ever have, are between -version branches. - - -### squash -```sh - git merge --squash <other-branch> -``` - -When you need multiple branches to implement a feature and merge between them, try to -use *squash*. Squashing basically puts all commits of the merged branch into a single commit. -It basically allows you to have many commits and then squash them into one. This is useful -if these commits were made just to synchronize between workstations, due to -unexpected errors in CI/CD, because you needed a save point, etc. Again the goal is to have -coherent commits, where each commits makes sense on its own. - - -## Submodules - - -The main NOMAD GitLab-project (`nomad-fair`) uses Git-submodules to maintain its -parsers and other dependencies. All these submodules are places in the `/dependencies` -directory. There are helper scripts to install (`./dependencies.sh`) and -commit changes to all submodules (`./dependencies-git.sh`). After merging or checking out, -you have to make sure that the modules are updated to not accidentally commit old -submodule commits again. Usually you do the following to check if you really have a -clean working directory. - -```sh - git checkout something-with-changes - git submodule update --init --recursive - git status -``` - -We typically use the `master`/`main` branch on all dependencies. Of course feature branches -can be used on dependencies to manage work in progress. diff --git a/docs/develop/guides.md b/docs/develop/guides.md index 0277264849165da5671fd6ab65c09013c76a7d79..d03d454487f26b2bbe285efead811213f80f61ea 100644 --- a/docs/develop/guides.md +++ b/docs/develop/guides.md @@ -45,7 +45,7 @@ These *guidelines* are partially enforced by CI/CD. As part of CI all tests are branches; further we run a *linter*, *pep8* checker, and *mypy* (static type checker). You can run `nomad qa` to run all these tests and checks before committing. -See [git/gitlab](./gitlab.md) for more details on how to work with issues, branches, merge +See [the contributing guide](./contrib.md) for more details on how to work with issues, branches, merge requests, and CI/CD. ## Documenting code diff --git a/mkdocs.yml b/mkdocs.yml index 286d166916b299374bf6a2b95ca32e0d098bcca6..a22add2b4a18edb349f0a399b5411f2083ca64d3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -28,8 +28,9 @@ nav: - How-to write parser plugins: plugins/parsers.md - Development: - How-to get started: develop/setup.md + - How-to navigate the code: develop/code.md + - How-to contribute: develop/contrib.md - Code guidelines: develop/guides.md - - How-to contribute: develop/gitlab.md - How-to extend the search: develop/search.md - How-to write a parser: develop/parsers.md - How-to write a normalizer: develop/normalizers.md @@ -90,8 +91,10 @@ plugins: module_name: nomad/mkdocs - redirects: redirect_maps: + 'plugins.md': 'plugins/plugins.md' 'pythonlib.md': 'apis/pythonlib.md' 'oasis.md': 'oasis/install.md' + 'develop/gitlab.md': 'develop/contrib.md' extra_css: - stylesheets/extra.css extra_javascript: