Single-user Jupyter notebook image for analytics

Please note that this documentation is just a draft. Some of the instructions may be incomplete.

This notebook folder contains Dockerfile for a single user notebook image which based on Jupyter docker stack. In addition of the base notebook it contains QUIP with GAP and all of the necessary packages for the tools. The docker image is suitable for running/testing locally and for deploying it by the jupyterhub on a kubernetes cluster.

Short description

• Based on the jupyter/minimal-notebook docker image
• conda: python 2 and 3 environments
• compiled quip with gap

Tutorials

Each tutorial could have it's own repository. For practical reason please use "analytics-" as prefix for your tutorial

You can add a new repository to the tutorial folder like:

cd tutorials
git submodule add https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-grain-boundaries.git grain-boundaries

Link for the old tutorials:tutorials

• https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-lab-base/tree/master/analysis-tools
• https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-toolkit-tutorials

Building/using your own docker image

Cloning the repository

git clone --recursive https://gitlab.mpcdf.mpg.de/nomad-lab/analytics.git

Alternativle you can clone it wothout the submodules:

git clone https://gitlab.mpcdf.mpg.de/nomad-lab/analytics.git

And you can initialise the submodules when its nedded:

git submodule update --init --recursive

Building the docker image locally

docker build -t analytics-notebook:latest .

Testing/Developing the notebooks locally

• Use the following command to run the docker image locally with a password-less sudo access:

  docker run -d --rm \
             -p 8888:8888 \
             -v $PWD/tutorials:/home/jovyan/tutorials \
             -e GRANT_SUDO=yes \
             --user root \
             --name analytics-notebook \
             analytics-notebook:latest 

• To have bash in the container use the following command:

   docker exec -it analytics-notebook start.sh

  more info: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/common.html?highlight=root#alternative-commands

Useful tricks for Linux

• avoiding tokens for development

  docker run --rm \
    -p 8889:8888 \
    -v $PWD/tutorials:/home/jovyan/tutorials \
    --name analytics-notebook \
    analytics-notebook:latest \
      start-notebook.sh \
        --NotebookApp.token=''

• you may consider to use the '--rm' option which deletes the container when it stop. Be careful in this way you will loose everything which was not stored into the mounted folder.

  docker run \
         --rm \
         -p 8888:8888  \
         -v $PWD/tutorials:/home/jovyan/tutorials  \
         --name analytics-notebook \
         analytics-notebook:latest

• For mounting a folder, you may need to use absolute path or other tricks if the PWD environmental variable is not accessible in your shell:

  docker run \
         --rm \
         -p 8888:8888  \
         -v /path/for/the/tutorials:/home/jovyan/tutorials  \
         --name analytics-notebook \
         analytics-notebook:latest

• you may need to change the user id in the container - by adding "-e NB_UID=1001" and "--user root" options to your command - to have access for the mounted folders:

  docker run \
         --rm \
         -p 8888:8888  \
         -v $PWD/tutorials:/home/jovyan/tutorials  \
         --user root \
         -e NB_UID=1001 \
         --name analytics-notebook \
         analytics-notebook:latest

More information about the command line options: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/common.html#notebook-options

Continuous integration

Each commit triggers a build process on GitLab Runner. Beside the latest tag there will be a unique tag (same that as the value of the git commit) available for explicitly tracking the version of the notebook for a cluster deployment.

Using the docker image from the registry

1. Install docker on your machine

2. Login to the image repository

   docker login gitlab-registry.mpcdf.mpg.de 

3. Pull the image:

   docker pull gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest

4. Create a container:

   docker run -p 8888:8888 \
              -v $PWD/tutorials:/home/jovyan/tutorials \
              gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest

Note: The latest tag can be find on the following page: https://gitlab.mpcdf.mpg.de/nomad-lab/analytics/container_registry