nomad-FAIR issueshttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues2023-12-21T15:38:52Zhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1685Getting complaint in installation of nomad with python3.102023-12-21T15:38:52ZRubel MozumderGetting complaint in installation of nomad with python3.10In the dev environment installation process having a complaint from mdtraj package while `python version 3.10`:
```
File "/home/rubel/NOMAD-FAIRmat/GL/nomad-FAIR/.NF_1674_xrd/lib/python3.10/site-packages/pip/_vendor/pyproject_hoo...In the dev environment installation process having a complaint from mdtraj package while `python version 3.10`:
```
File "/home/rubel/NOMAD-FAIRmat/GL/nomad-FAIR/.NF_1674_xrd/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/rubel/NOMAD-FAIRmat/GL/nomad-FAIR/.NF_1674_xrd/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/rubel/NOMAD-FAIRmat/GL/nomad-FAIR/.NF_1674_xrd/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-ghmqe1s6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 396, in prepare_metadata_for_build_wheel
self.run_setup()
File "/tmp/pip-build-env-ghmqe1s6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-ghmqe1s6/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in run_setup
exec(code, locals())
File "<string>", line 301, in <module>
File "/tmp/pip-build-env-ghmqe1s6/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
cythonize_one(*args)
File "/tmp/pip-build-env-ghmqe1s6/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: mdtraj/geometry/src/_geometry.pyx
```
But no issues are found with `python 3.9`.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1404Operatin Nomad Oasis, manage Rabbitmq storage2023-12-21T15:38:53ZMichael GötteOperatin Nomad Oasis, manage Rabbitmq storageHey Nomad,
I am running a nomad oasis and for some reason rabbitmq has docker volumes with 33GB, that does not seem to be reasonable to me. If this grows in a similar rate it quickly becomes too big.
Here you see what i mean.
![image](...Hey Nomad,
I am running a nomad oasis and for some reason rabbitmq has docker volumes with 33GB, that does not seem to be reasonable to me. If this grows in a similar rate it quickly becomes too big.
Here you see what i mean.
![image](/uploads/9dd36d90a3520b557c4cf1a0619b84fb/image.png)
Do you have an idea what to do?
Best MichaMarkus ScheidgenMarkus Scheidgenhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/935Improve on gitlab ci/cd script2022-09-13T08:21:03ZAdam FeketeImprove on gitlab ci/cd script- [x] define environments for `staging` (aka `develop`) similarly to `dev`
- [x] checking the rules because some of them does not make sense anymore (like: `$CI_COMMIT_REF_NAME =~ /^dev-.*$/`)
- [ ] checking the way how initialisation of...- [x] define environments for `staging` (aka `develop`) similarly to `dev`
- [x] checking the rules because some of them does not make sense anymore (like: `$CI_COMMIT_REF_NAME =~ /^dev-.*$/`)
- [ ] checking the way how initialisation of submodules works
- [x] get rid of docker login waning (`echo $CI_BUILD_TOKEN | docker login --username foo --password-stdin`)
- [x] cleanup (eg. using `before_script` more often)
- [ ] regularly rebuild the image from scratch to avoid issues like #937 (--no-cache)https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/908Sporadically failing python-tests during CI2022-09-15T09:05:27ZMarkus ScheidgenSporadically failing python-tests during CI
This is a list of test cases, where we observed that they failed or errored, but the problem is gone when simply repeating the tests.
- `tests/app/v1/routers/test_uploads.py::test_post_upload[invalid-credentials]`
- `tests/app/v1/route...
This is a list of test cases, where we observed that they failed or errored, but the problem is gone when simply repeating the tests.
- `tests/app/v1/routers/test_uploads.py::test_post_upload[invalid-credentials]`
- `tests/app/v1/routers/test_uploads.py::test_put_upload_raw_path[zip-to-subfolder]`
- `tests/app/v1/routers/test_uploads.py::test_post_upload[stream-no-embargo]`
- `tests/app/v1/routers/test_uploads.py::test_editing_raw_file[conflict_in_concurrent_editing]`
These are probably race-conditions with a singular cause.David SikterDavid Sikterhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1415cannot run the pipeline manually2023-12-21T15:40:10ZMohammad Nakhaeecannot run the pipeline manuallyNormally we should be able to run the pipeline manually for the branches that failed because of unexpected server errors. But the rules do not allow.
![Screenshot_from_2023-03-30_13-39-37](/uploads/1587ce4269d10bb6c03851950b9d4978/Scree...Normally we should be able to run the pipeline manually for the branches that failed because of unexpected server errors. But the rules do not allow.
![Screenshot_from_2023-03-30_13-39-37](/uploads/1587ce4269d10bb6c03851950b9d4978/Screenshot_from_2023-03-30_13-39-37.png)https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1357Running nomad-oasis on the root path2023-12-21T15:40:06ZAdam FeketeRunning nomad-oasis on the root pathissues/workarounds:
- [ ] update nginx.conf
- [x] update north's health check
- [ ] check on mounted folder ownerships
- [x] fixing GUI's double leading slashes("//api/v1/north/apmtools")issues/workarounds:
- [ ] update nginx.conf
- [x] update north's health check
- [ ] check on mounted folder ownerships
- [x] fixing GUI's double leading slashes("//api/v1/north/apmtools")Adam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1186Possible improvements on python packaging2023-12-21T16:06:49ZAdam FeketePossible improvements on python packaging- [ ] using src layout
- [ ] ease (>=) on package dependencies in the submodules and constrain them in the main (~=)
- [ ] python version ?? 3.10 or 3.11 (local oasis)
- [ ] test matrix ?
- [ ] convert parsers into plugins/extensions ...- [ ] using src layout
- [ ] ease (>=) on package dependencies in the submodules and constrain them in the main (~=)
- [ ] python version ?? 3.10 or 3.11 (local oasis)
- [ ] test matrix ?
- [ ] convert parsers into plugins/extensions VS monolithic package (Distribution package VS individual packages)
- [ ] implicit namespaces
```python
/src/nomad_lab/parser
/src/nomad_lab/gui
import nomad_lab.parser
import nomad_lab.gui
```
- [ ] linting: we might ignoring too much warnings (ignore them in the code instead)
alternative: black
- [ ] run `gui` and `jupyterhub` independently
- gui: is an independent application which depends on some artefacts
- [ ] adding `linter` and `pytest` configs to `myproject.toml`
- [ ] examples: storing examples in a separate repository using [git LFS](https://docs.gitlab.com/ee/topics/git/lfs/) for the zipped files like: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-fair-examples
- [ ] docs: The "compiled" documentation should not be part of the python source distribution. Reasoning:
* we need the nomad package to build the documentation which actually should also contain the documentation... Currently the only way to build the python sdist by building it twice.
* most of the other links on the gui are also pointing to external link
* the link of the logo of the documentation redirect to a another page (central deployment) and not to the one where it was actually deployed.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1048Improve on python setup process2022-09-28T12:52:03ZAdam FeketeImprove on python setup process- [x] using new standard `myproject.toml`
- [x] resolve package conflicts (closes: #934)
- [x] use `pip-compile` for generating the `requirement.txt` file
- [ ] use `setuptool_scm` to generate version automatically
- [ ] documentation ab...- [x] using new standard `myproject.toml`
- [x] resolve package conflicts (closes: #934)
- [x] use `pip-compile` for generating the `requirement.txt` file
- [ ] use `setuptool_scm` to generate version automatically
- [ ] documentation about how to bump the version of packages up
- [ ] regularly rebuild the image from scratch to avoid issues like #937 (--no-cache)Adam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1360North/gui: Allow custom tools.json for deployments2023-05-17T13:17:44ZFlorian DobenerNorth/gui: Allow custom tools.json for deploymentsIt would be good to allow custom tools.json files for deployments. It is possible to mount a custom `tools.json` to change the properties of existing containers. However, the changes are only picked up by jupyter hub and not by the gui a...It would be good to allow custom tools.json files for deployments. It is possible to mount a custom `tools.json` to change the properties of existing containers. However, the changes are only picked up by jupyter hub and not by the gui as the `northTools.json` is compiled into the gui js. Therefore, it's not possible to add or remove containers in a deployment.https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1350.volumes folder structure is missing from nomad-oasis.zip2023-02-21T12:53:13ZAdam Fekete.volumes folder structure is missing from nomad-oasis.zipAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1358North fiji tool crashing at stratup2023-02-23T11:08:01ZAdam FeketeNorth fiji tool crashing at stratupAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1347NORTH: the updated tool image has not been updated on the deployment2023-02-21T12:53:53ZAdam FeketeNORTH: the updated tool image has not been updated on the deploymentAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1345NORTH: Websocket connection (webtop based tools) failure on Nginx reverse proxy2023-02-16T15:51:11ZAdam FeketeNORTH: Websocket connection (webtop based tools) failure on Nginx reverse proxyhttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1220Keycloak redirect problem with nomad-oasis behind ssl (traefik) proxy2023-12-21T15:38:48ZAdam FeketeKeycloak redirect problem with nomad-oasis behind ssl (traefik) proxyThe url returns with "502 Bad gateway nginx/1.13.9"
```
https://nomad.eln.data.fau.de/keycloak/auth/realms/nomad/protocol/openid-connect/auth?response_type=code&redirect_uri=http%3A%2F%2Fnomad.eln.data.fau.de%2Fnomad-oasis%2Fnorth%2Fhub%...The url returns with "502 Bad gateway nginx/1.13.9"
```
https://nomad.eln.data.fau.de/keycloak/auth/realms/nomad/protocol/openid-connect/auth?response_type=code&redirect_uri=http%3A%2F%2Fnomad.eln.data.fau.de%2Fnomad-oasis%2Fnorth%2Fhub%2Foauth_callback&client_id=nomad_public&state=eyJzdGF0ZV9pZCI6ICJmOTMxYWIyMWM3NDU0MWUwYTRkNzczYTVmZDU1Y2FiNCIsICJuZXh0X3VybCI6ICIvbm9tYWQtb2FzaXMvbm9ydGgvaHViL2FwaS9vYXV0aDIvYXV0aG9yaXplP2NsaWVudF9pZD1qdXB5dGVyaHViLXVzZXItYWRtaW4tanVweXRlciZyZWRpcmVjdF91cmk9JTJGbm9tYWQtb2FzaXMlMkZub3J0aCUyRnVzZXIlMkZhZG1pbiUyRmp1cHl0ZXIlMkZvYXV0aF9jYWxsYmFjayZyZXNwb25zZV90eXBlPWNvZGUmc3RhdGU9ZXlKMWRXbGtJam9nSW1VME9EVTJOVFU0WkRrNFpUUXhOemc0TkRWaVltUTVNamt6TXpCallXTXlJaXdnSW01bGVIUmZkWEpzSWpvZ0lpOXViMjFoWkMxdllYTnBjeTl1YjNKMGFDOTFjMlZ5TDJGa2JXbHVMMnAxY0hsMFpYSXZkSEpsWlNKOSJ9&scope=openid+profile
```Adam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1848Refine building process of docker image2024-01-10T00:21:46ZAdam FeketeRefine building process of docker image- [ ] Explicitly define the uid and gid for the nomad user
- [ ] Combine python and js development stage- [ ] Explicitly define the uid and gid for the nomad user
- [ ] Combine python and js development stagehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1343Issue: unavailable persistent volume claim of north tools in the staging depl...2023-02-16T13:27:45ZAdam FeketeIssue: unavailable persistent volume claim of north tools in the staging deploymentAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1230quick fix: initialise all git submodules during setup process2022-12-13T11:26:17ZAdam Feketequick fix: initialise all git submodules during setup processAdam FeketeAdam Feketehttps://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1211minor fixes for the pipeline2022-12-02T23:09:04ZAdam Feketeminor fixes for the pipeline