Skip to content
Snippets Groups Projects
Commit e17b9f55 authored by Adam Fekete's avatar Adam Fekete
Browse files

improve on the documentaion

parent fca2bf59
Branches
No related tags found
3 merge requests!123testing develop merge,!23Various tutorials,!19cmlkit - new tutorial
# Single-user Jupyter notebook image for analytics
**Please note that this documentation is just a draft. Some of the instructions may be incomplete.**
This notebook folder contains Dockerfile for a single user notebook image which based on Jupyter docker stack.
In addition of the base notebook it contains QUIP with GAP and all of the necessary packages for the tools.
The docker image is suitable for running/testing locally and for deploying it by the jupyterhub on a kubernetes cluster.
## Short description
- Based on the jupyter/minimal-notebook docker image
- conda: python 2 and 3 environments
- compiled quip with gap
## Tutorials
- Based on the [jupyter/tensorflow-notebook](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-tensorflow-notebook) docker image
- conda: python 3 environments
- compiled quip with gap, qmmlpack
Each tutorial could have it's own repository. For practical reason please use "analytics-" as prefix for your tutorial
You can add a new repository to the tutorial folder like:
```bash
cd tutorials
git submodule add https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-grain-boundaries.git grain-boundaries
```
Link for the old tutorials:tutorials
- https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-lab-base/tree/master/analysis-tools
- https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-toolkit-tutorials
## Building/using your own docker image
......@@ -46,7 +31,7 @@ git submodule update --init --recursive
### Building the docker image locally
```bash
docker build -t analytics-notebook:latest .
docker build -t analytics:latest .
```
......@@ -59,16 +44,62 @@ docker build -t analytics-notebook:latest .
-e GRANT_SUDO=yes \
--user root \
--name analytics-notebook \
analytics-notebook:latest
analytics:latest
```
- To have bash in the container use the following command:
```bash
docker exec -it analytics-notebook start.sh
docker exec -it analytics start.sh
```
more info: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/common.html?highlight=root#alternative-commands
### Using the docker image from the registry
Beside the latest tag there will be a unique tags (same that as the value of the git commit) available for explicitly tracking the version of the notebook for a cluster deployment. Each PR has its own tag (name of the source branch) which used for staging.
1. Install docker on your machine
2. Login to the image repository
```bash
docker login gitlab-registry.mpcdf.mpg.de
```
3. Pull the image:
```bash
docker pull gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest
```
4. Create a container:
```bash
docker run -p 8888:8888 \
-v $PWD/tutorials:/home/jovyan/tutorials \
gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest
```
Note: The latest tag can be find on the following page:
https://gitlab.mpcdf.mpg.de/nomad-lab/analytics/container_registry
## How to add a new tutorial - Basic workflow
1) Create a branch and add all the necessary changes
2) Create PR to 'master' branch
- CI tries to build the image and deploys staging hub
3) Manually test the staging hub
4) Merge PR to 'master' branch
- CI deploys production hub
### Template
Each tutorial could have it's own repository and stored in the tutorials folder as a submodule:
```bash
cd tutorials
git submodule add https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-{tutorial}.git
```
To create a new tutorial you can use the skeleton project in the `template`folder.
### Useful tricks for Linux
......@@ -77,8 +108,8 @@ docker build -t analytics-notebook:latest .
docker run --rm \
-p 8889:8888 \
-v $PWD/tutorials:/home/jovyan/tutorials \
--name analytics-notebook \
analytics-notebook:latest \
--name analytics \
analytics:latest \
start-notebook.sh \
--NotebookApp.token=''
```
......@@ -89,8 +120,8 @@ docker build -t analytics-notebook:latest .
--rm \
-p 8888:8888 \
-v $PWD/tutorials:/home/jovyan/tutorials \
--name analytics-notebook \
analytics-notebook:latest
--name analytics \
analytics:latest
```
- For mounting a folder, you may need to use absolute path or other tricks if the PWD environmental variable is not accessible in your shell:
......@@ -99,8 +130,8 @@ docker build -t analytics-notebook:latest .
--rm \
-p 8888:8888 \
-v /path/for/the/tutorials:/home/jovyan/tutorials \
--name analytics-notebook \
analytics-notebook:latest
--name analytics \
analytics:latest
```
- you may need to change the user id in the container - by adding "-e NB_UID=1001" and "--user root" options to your command - to have access for the mounted folders:
......@@ -111,36 +142,10 @@ docker build -t analytics-notebook:latest .
-v $PWD/tutorials:/home/jovyan/tutorials \
--user root \
-e NB_UID=1001 \
--name analytics-notebook \
analytics-notebook:latest
--name analytics \
analytics:latest
```
More information about the command line options: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/common.html#notebook-options
## Continuous integration
Each commit triggers a build process on GitLab Runner. Beside the latest tag there will be a unique tag (same that as the value of the git commit) available for explicitly tracking the version of the notebook for a cluster deployment.
### Using the docker image from the registry
1. Install docker on your machine
2. Login to the image repository
```bash
docker login gitlab-registry.mpcdf.mpg.de
```
3. Pull the image:
```bash
docker pull gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest
```
4. Create a container:
```bash
docker run -p 8888:8888 \
-v $PWD/tutorials:/home/jovyan/tutorials \
gitlab-registry.mpcdf.mpg.de/nomad-lab/analytics:latest
```
Note: The latest tag can be find on the following page:
https://gitlab.mpcdf.mpg.de/nomad-lab/analytics/container_registry
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment