Commit 3a440d93 authored by Markus Scheidgen's avatar Markus Scheidgen
Browse files

Merge remote-tracking branch 'origin/v0.7.10' into v0.8.0

parents a41ac458 eba41f94
Pipeline #70881 passed with stages
in 29 minutes and 7 seconds
Subproject commit 43c28c292079f819f06383af123e2e933f5f55ba
Subproject commit 36378d0cf67b46d28c1c5476c387978143d21367
Subproject commit d73611bc1b16ea71daa3d0fd24ee275c78853557
Subproject commit 95bb8e2a8288aac6fb8d0bfbb3e7d28db43e880b
Subproject commit 022a2af6bad45364dbdfac6b6c913f04186ac7d4
Subproject commit 2e6d473f7776c41df10905f4bee27116d4bc0d99
Command Line Interface (CLI)
----------------------------
The :code:`nomad` python package comes with a command line interface (CLI) that
can be accessed after installation by simply running the :code:`nomad` command
in your terminal. The CLI provides a hiearchy of commands by using the `click
package <https://click.palletsprojects.com/>`_.
This documentation describes how the CLI can be used to manage a NOMAD
installation. For commmon use cases see :ref:`cli_use_cases`. For a full
reference of the CLI commands see :ref:`cli_ref`.
.. toctree::
:maxdepth: 2
cli_use_cases.rst
cli_ref.rst
.. _cli_ref:
CLI Reference
*************
Client CLI commands
""""""""""""""""""""""""""""""""""""""""
.. click:: nomad.cli.client.client:client
:prog: nomad client
:show-nested:
Admin CLI commands
""""""""""""""""""""""""""""""""""""""""
.. click:: nomad.cli.admin.admin:admin
:prog: nomad admin
:show-nested:
.. _cli_use_cases:
Use cases
*********
Mirroring data between production environments
""""""""""""""""""""""""""""""""""""""""""""""
Sometimes you would wish to transfer data between separate deployments of the
NOMAD infrastructure. This use case covers the situation when the deployments
are up and running and both have access to the underlying file storage, part of
which is mounted inside each container under :code:`.volumes/fs`.
With both the source and target deployment running, you can use the
:code::ref:`cli_ref:mirror` command to transfer the data from source to target. The
mirror will copy everything: i.e. the raw data, archive data and associated
metadata in the database.
The data to be mirrored is specified by using a query API path. For example to
mirror the upload from source deployment to target deployment, you would use
the following CLI command inside the target deployment:
.. code-block:: sh
nomad client -n <api_url> -u <username> -w <password> mirror <query_json> --source-mapping <target_docker_path>:<shared_path>
Here is a breakdown of the different arguments:
* :code:`-n <url>`: Url to the API endpoint in the source deployment. This API will
be queried to fetch the data to be mirrored. E.g.
http://repository.nomad-coe.eu/api
* :code:`-u <username>`: Your username that is used for authentication in the API call.
* :code:`-w <password>`: Your password that is used for authentication in the API call.
* :code:`mirror <query>`: Your query as a JSON dictionary. See the documentation for
available keywords. E.g. "{"upload_id: "<upload_id>"}"
* :code:`--source-mapping <mapping>`: The deployments use a separate folder to store
the archive and raw data. To correctly find the data that should be
mirrored, the absolute path on the filesystem that is shared between the
deployments needs to be provided. E.g. *.volumes/fs:/nomad/fairdi/prod/fs*.
The first part of this mapping indicates a docker volume path
(*.volumes/fs* in this example) that should be mapped to the second
filepath on the shared filesystem (*/nomad/fairdi/prod/fs* in this example).
Updating the AFLOW prototype information
""""""""""""""""""""""""""""""""""""""""
NOMAD uses the `AFLOW prototype library
<http://www.aflowlib.org/CrystalDatabase/>`_ to link bulk crystal entries with
prototypical structures based on their symmetry. The
:ref:`cli_ref:prototypes-update` subcommand can be used to update this
database from the online information provided by AFLOW. The command produces a
prototype dataset as a python module.
The dataset should be recreated if the AFLOW dataset has been updated or if the
symmetry matching routine used within NOMAD is updated (e.g. the symmetry
tolerance is modified). To produce a new dataset run the following command:
.. code-block:: sh
nomad admin ops prototypes-update <module_path>
Here is a breakdown of the different arguments:
* :code:`<module_name>`: Name of the python module in which the data should
be stored. If the file does not exist it will be created. The prototype
data used by NOMAD is under the path:
*nomad/normalizing/data/aflow_prototypes.py*
The command also provides a :code:`--matches-only` flag for only updating the
dataset entry that is used for matching the prototypes. This means that the
online information from AFLOW is not queried. This makes the process faster
e.g. in the case when you want only to update the matches after modifying the
symmetry routines.
......@@ -45,6 +45,8 @@ extensions = [
'sphinx.ext.coverage',
'sphinx.ext.ifconfig',
'sphinx.ext.napoleon',
'sphinx.ext.autosectionlabel',
'sphinx_click.ext',
'sphinxcontrib.httpdomain',
'sphinxcontrib.autohttp.flask',
'sphinxcontrib.autohttp.flaskqref',
......@@ -52,6 +54,9 @@ extensions = [
'm2r'
]
# Prefix the automatically generated labels with the document name
autosectionlabel_prefix_document = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['.templates']
......
.. mdinclude:: ../ops/docker-compose/nomad/README.md
.. mdinclude:: ../ops/helm/nomad/README.md
.. mdinclude:: ../ops/containers/README.md
.. mdinclude:: ../ops/docker-compose/nomad-oasis/README.md
Operating NOMAD
===============
###############
.. mdinclude:: ../ops/README.md
.. mdinclude:: ../ops/docker-compose/nomad/README.md
.. mdinclude:: ../ops/helm/nomad/README.md
.. mdinclude:: ../ops/containers/README.md
.. mdinclude:: ../ops/docker-compose/nomad-oasis/README.md
.. toctree::
:maxdepth: 2
depl_docker
depl_helm
depl_images
cli
oasis
## Overview
Read the [introduction](./introduction.html) and [setup](./setup.html) for input on
the different nomad services. This is about how to deploy and operate these services.
......
......@@ -12,12 +12,13 @@ matid
# infrastructue related
msgpack<0.6.0
elasticsearch==6.4.0
elasticsearch-dsl==6.4.0
pyyaml
future
enum34
requests
celery[redis]
elasticsearch-dsl>=6.0.0,<7.0.0
mongoengine==0.18.2
Werkzeug==0.16.1
flask
......@@ -38,7 +39,7 @@ zipstream-new==1.1.5
bagit
bcrypt
filelock
ujson
ujson==1.35
bravado
pyjwt[crypto]
jsonschema[format]
......@@ -59,6 +60,7 @@ python-json-logger
setuptools
sphinx
sphinxcontrib.httpdomain
sphinx-click
sphinx_rtd_theme
gitpython
mypy==0.730
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment