Skip to content
Snippets Groups Projects
Commit efd7b32b authored by Adam Fekete's avatar Adam Fekete
Browse files

Merge branch 'new_python_install' into 'master' 

New python install script + metainfos

See merge request !1
parents 0b1d36c2 7c761885
Branches
No related tags found
1 merge request!1New python install script + metainfos
Hide whitespace changes
Inline Side-by-side
Showing
with 126 additions and 203 deletions
.gitmodules
+ 32
− 32
View file @ efd7b32b
[submodule "tutorials/grain-boundaries"]
path = tutorials/grain-boundaries
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-grain-boundaries.git
[submodule "tutorials/asevisulization"]
path = tutorials/asevisulization
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-asevisualization.git
[submodule "tutorials/compressed-sensing"]
path = tutorials/compressed-sensing
[submodule "3rdparty/quip"]
path = 3rdparty/quip
url = https://github.com/libAtoms/QUIP.git
[submodule "tutorials/analytics-compressed-sensing"]
path = tutorials/analytics-compressed-sensing
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-compressed-sensing.git
[submodule "tutorials/analytics-compressed-sensing-2018"]
path = tutorials/compressed-sensing-2018
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-compressed-sensing-2018.git
[submodule "tutorials/convolutional-nn"]
path = tutorials/convolutional-nn
[submodule "tutorials/analytics-kaggle-competition"]
path = tutorials/analytics-kaggle-competition
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-kaggle-competition.git
[submodule "tutorials/analytics-asevisualization"]
path = tutorials/analytics-asevisualization
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-asevisualization.git
[submodule "tutorials/analytics-band-structure-visualization"]
path = tutorials/analytics-band-structure-visualization
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-band-structure-visualization.git
[submodule "tutorials/analytics-convolutional-nn"]
path = tutorials/analytics-convolutional-nn
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-convolutional-nn.git
[submodule "tutorials/face-of-crystals"]
path = tutorials/face-of-crystals
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-face-of-crystals.git
[submodule "tutorials/gap-si-surface"]
path = tutorials/gap-si-surface
[submodule "tutorials/analytics-gap-si-surface"]
path = tutorials/analytics-gap-si-surface
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-gap-si-surface.git
[submodule "tutorials/kaggle-competetion"]
path = tutorials/kaggle-competetion
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-kaggle-competetion.git
[submodule "tutorials/soap-atomic-charges"]
path = tutorials/soap-atomic-charges
[submodule "tutorials/analytics-soap-atomic-charges"]
path = tutorials/analytics-soap-atomic-charges
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-soap-atomic-charges.git
[submodule "notebook/quip"]
path = 3rdparty/quip
url = https://github.com/libAtoms/QUIP.git
[submodule "tutorials/band-structure-visualization"]
path = tutorials/band-structure-visualization
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-band-structure-visualization.git
[submodule "tutorials/tcmi"]
path = tutorials/tcmi
url = https://gitlab.mpcdf.mpg.de/rbenj/analytics-tcmi
[submodule "tutorials/analytics-grain-boundaries"]
path = tutorials/analytics-grain-boundaries
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-grain-boundaries.git
[submodule "tutorials/analytics-compressed-sensing-2018"]
path = tutorials/analytics-compressed-sensing-2018
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-compressed-sensing-2018.git
[submodule "tutorials/analytics-tcmi"]
path = tutorials/analytics-tcmi
url = https://gitlab.mpcdf.mpg.de/rbenj/analytics-tcmi.git
[submodule "analytics-tutorial-template"]
path = analytics-tutorial-template
url = https://gitlab.mpcdf.mpg.de/nomad-lab/analytics-tutorial-template.git
Dockerfile
+ 85
− 30
View file @ efd7b32b
FROM jupyter/tensorflow-notebook:2ce7c06a61a1
FROM jupyter/tensorflow-notebook:2ce7c06a61a1 AS main
LABEL maintainer="Luca Ghiringhelli <ghiringhelli@fhi-berlin.mpg.de>"
# ================================================================================
# Linux applications and libraries
# ================================================================================
USER root
RUN apt-get update \
&& apt-get install -y -q --no-install-recommends \
......@@ -17,46 +21,37 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
USER $NB_UID
# Dependecies:
# - quippy: gcc gfortran liblapack-dev libblas-dev libnetcdf-dev netcdf-bin libxpm-dev libgsl-dev
# ================================================================================
# Python environment
# ================================================================================
# Add any additional packages you want available for use in a Python 3 notebook
# to the first line here (e.g., pandas, matplotlib, etc.)
# Remove pyqt and qt pulled in for matplotlib since we're only ever going to
# use notebook-friendly backends in these images
# to the first line here (e.g., nglview, jupyter_contrib_nbextensions, etc.)
# https://github.com/ipython-contrib/jupyter_contrib_nbextensions
RUN conda install --quiet --yes \
'pytorch::pytorch-cpu==1.1.0' \
'pytorch::torchvision-cpu==0.3.0' \
'colorcet==2.0.1' \
&& pip install \
'git+https://github.com/raghakot/keras-vis.git' \
'ase==3.17.0' \
'asap3==3.11.10' \
'numpy==1.18.1' \
'scipy==1.4.1' \
'pandas==0.25.3' \
'scikit-learn==0.21.1' \
'joblib==0.14.1' \
'diskcache==4.1.0' \
&& conda clean -tipsy \
&& fix-permissions $CONDA_DIR \
&& fix-permissions /home/$NB_USER
# https://github.com/ipython-contrib/jupyter_contrib_nbextensions
RUN conda install --quiet --yes \
'nglview==2.7.0' \
'jupyter_contrib_nbextensions==0.5.1' \
'jupyter_nbextensions_configurator==0.4.1' \
'ase' \
'asap3' \
&& conda clean -tipsy \
&& jupyter nbextensions_configurator enable --user \
&& jupyter nbextension install nglview --py --sys-prefix \
&& jupyter nbextension enable nglview --py --sys-prefix \
&& jupyter nbextension enable execute_time/ExecuteTime \
&& jupyter nbextension enable init_cell/main
&& jupyter nbextension enable init_cell/main \
&& fix-permissions $CONDA_DIR \
&& fix-permissions /home/$NB_USER
# Dependecies:
# - ase: many
# - asap3: grain-boundaries
# - nglview:
# ================================================================================
# QUIP + GAP + quippy
......@@ -67,9 +62,7 @@ WORKDIR /opt/quip
# QUIP for general use is the OpenMP version.
ENV QUIP_ARCH linux_x86_64_gfortran_openmp
ENV QUIP_INSTALLDIR /opt/quip/bin
ENV PATH $QUIP_INSTALLDIR:$PATH
COPY 3rdparty/quip .
COPY 3rdparty/gap src/GAP
......@@ -82,22 +75,68 @@ RUN chown -R $NB_USER:$NB_GID $PWD
USER $NB_USER
# Installs with no suffix, e.g. quip
RUN make > /dev/null 2>&1 \
RUN make \
&& make install
# Installs quippy using python2 environment
# && make install-quippy > /dev/null 2>&1
RUN pip install -r quippy/requirements.txt \
# Installs quippy
RUN pip install src/f90wrap \
&& make install-quippy
# Make quippy executable available from terminal
ENV PATH /opt/quip/bin:$PATH
# ================================================================================
# Install all of the package dependencies of the tutorials
# ================================================================================
# Note: multi stage build may be used here as well in the future
WORKDIR /opt/tutorials
COPY tutorials/ .
RUN pip install ./analytics-asevisualization; exit 0
RUN pip install ./analytics-band-structure-visualization; exit 0
RUN pip install ./analytics-compressed-sensing; exit 0
RUN pip install ./analytics-compressed-sensing-2018; exit 0
RUN pip install ./analytics-convolutional-nn; exit 0
RUN pip install ./analytics-gap-si-surface; exit 0
RUN pip install ./analytics-grain-boundaries; exit 0
RUN pip install ./analytics-kaggle-competition; exit 0
RUN pip install ./analytics-soap-atomic-charges; exit 0
RUN pip install ./analytics-tcmi; exit 0
USER root
RUN fix-permissions /opt/tutorials \
&& fix-permissions $CONDA_DIR
# ==================
# python tutorials/gap-si-surface/setup.py install
#ARG TUTORIALS_HOME=$HOME/tutorials
#
#WORKDIR $TUTORIALS_HOME
#RUN ln -s /opt/tutorials/*/*.ipynb .
#
## Linking images or other assets may required by the tutorials
#WORKDIR $TUTORIALS_HOME/assets
#RUN ln -s /opt/tutorials/*/assets/* .
#
## Linking data may required by the tutorials
#WORKDIR $TUTORIALS_HOME/data
#RUN ln -s /opt/tutorials/*/data/* $TUTORIALS_HOME/data
#
#RUN fix-permissions $TUTORIALS_HOME/data
# ==================
ARG TUTORIALS_HOME=$HOME/tutorials
WORKDIR $TUTORIALS_HOME
RUN ln -s /opt/tutorials/*/*.ipynb .; exit 0
RUN mkdir assets && ln -s /opt/tutorials/*/assets/* assets/; exit 0
RUN mkdir data && ln -s /opt/tutorials/*/data/* data/; exit 0
RUN fix-permissions $TUTORIALS_HOME
# ================================================================================
# Switch back to jovyan to avoid accidental container runs as root
......@@ -106,4 +145,20 @@ RUN pip install -r quippy/requirements.txt \
WORKDIR $HOME
USER $NB_UID
#
## Unidentified python dependencies
#RUN pip install \
# 'git+https://github.com/raghakot/keras-vis.git' \
# 'joblib==0.14.1' \
# 'diskcache==4.1.0' \
# 'colorcet==2.0.1' \
# This part contains the installation of all common python package
# All the other package will be isntalled by the setup scipt of each invivdual tutorial
# Issues:
# data should be part of the python package or not? (symlink)
# absolute path for tutorial's data
# jsmol has thousends of files (10158 files in total).
# "in adavce" trust of notebook (eg. tcmi)
\ No newline at end of file
analytics-tutorial-template @ 48b7ede9
Subproject commit 48b7ede9bc25ef75a44c77ca6040d5887b62c453
tutorials/Welcome.ipynb deleted 100644 → 0
+ 0
− 135
View file @ 0b1d36c2
%% Cell type:markdown id: tags:
# BIG-DATA ANALYTICS
We develop and implement methods that identify correlations and structure in big data of materials. This will enable scientists and engineers to decide which materials are useful for specific applications or which new materials should be the focus of future studies.
Despite the huge number of possible materials (e.g. GaAs, Al2O3, etc.), we note that “the chemical compound space” is sparsely populated when the focus is on selected properties or functions (e.g. structure: rock salt vs. zinkblende, electrical conductivity, etc.). NOMAD offers big-data analytics tools that will help to sort all of the available materials data to identify trends and anomalies. For more information click the "INTRODUCTION TO" button above.
## Tutorials
The following tutorials are designed to get started with the Analytics Toolkit (click title to show/hide details of the selected tutorial):
### Crystal structure prediction
- [**Tutorial on compressed sensing for materials property prediction**](compressed-sensing/tutorial.ipynb) *(created by: Emre Ahmetcik, Angelo Ziletti, Runhai Ouyang, Luca Ghiringhelli, Matthias Scheffler)*
This tutorial introduces from scratch and step by step: Compressed sensing, LASSO, and SISSO for materials property prediction
### Materials property prediction
- [**Grain boundaries of alpha-Fe tutorial**](grain-boundaries/grain_boundaries.ipynb) *(created by: Ádám Fekete, Martina Stella, Henry Lambert, Alessandro De Vita, Gábor Csányi)*
In this tutorial we will be using a machine learning method (clustering) to analyse results of Grain Boundary (GB) calculations of alpha-iron. Along the way we will learn about different methods to describe local atomic environment in order to calculate properties of GBs. We will use these properties to separate the different regions of the GB using clustering methods. Finally we will determine how the energy of the GB is changing according to the angle difference of the regions.
- [**The SOAP descriptor, Gaussian Approximation Potentials (GAP) and machine-learning of force fields**](gap-si-surface/gap_si_surface.ipynb) *(created by: Adam Fekete, Matrina Stella, Henry Lambert, Alessandro De Vita, Gabor Csanyi)
In this tutorial we will be using a Gaussian Approximation Potentials to analyse results of TB DFT calculations of Si surface. Along the way we will learn about different descriptors (2b, 3b, soap) to describe local atomic environment in order to predict energies and forces of Si surface.
### Organic molecules
- [**Learning atomic charges**](soap-atomic-charges/soap_atomic_charges.ipynb) *(created by: Gábor Csányi, James R. Kermode)*
In this tutorial, we will use Gaussian process regression, GPR (or equivalently, Kernel Ridge Regression, KRR) to train and predict charges of atoms in small organic molecules.
%% Cell type:markdown id: tags:
<h1>BIG-DATA ANALYTICS</h1>
<p></p>
<div class="jumbotron">
<p class="lead">We develop and implement methods that identify correlations and structure in big data of materials. This will enable scientists and engineers to decide which materials are useful for specific applications or which new materials should be the focus of future studies.
</div>
<h2 class="display-4">Tutorials</h2>
The following tutorials are designed to get started with the Analytics Toolkit (click title to show/hide details of the selected tutorial):
<h3 class="display-4">Crystal structure prediction</h3>
<p></p>
<div class="list-group">
<a href="compressed-sensing/tutorial.ipynb" class="list-group-item list-group-item-action active">
<h3>Tutorial on compressed sensing for materials property prediction</h3>
</a>
<div class="list-group-item list-group-item-action">
<i>created by: Emre Ahmetcik, Angelo Ziletti, Runhai Ouyang, Luca Ghiringhelli, Matthias Scheffler</i>
<p>This tutorial introduces from scratch and step by step: Compressed sensing, LASSO, and SISSO for materials property prediction</p>
</div>
</div>
<h3 class="display-4">Materials property prediction</h3>
<p></p>
<div class="list-group">
<a href="grain-boundaries/grain_boundaries.ipynb" class="list-group-item list-group-item-action active">
<h3>Grain boundaries of alpha-Fe tutorial</h3>
</a>
<div class="list-group-item list-group-item-action">
<i>created by: Ádám Fekete, Martina Stella, Henry Lambert, Alessandro De Vita, Gábor Csányi</i>
<p> In this tutorial we will be using a machine learning method (clustering) to analyse results of Grain Boundary (GB) calculations of alpha-iron. Along the way we will learn about different methods to describe local atomic environment in order to calculate properties of GBs. We will use these properties to separate the different regions of the GB using clustering methods. Finally we will determine how the energy of the GB is changing according to the angle difference of the regions.</p>
</div>
<a href="gap-si-surface/gap_si_surface.ipynb" class="list-group-item list-group-item-action active">
<h3>The SOAP descriptor, Gaussian Approximation Potentials (GAP) and machine-learning of force fields</h3>
</a>
<div class="list-group-item list-group-item-action">
<i>created by: Adam Fekete, Matrina Stella, Henry Lambert, Alessandro De Vita, Gabor Csanyi</i>
<p> In this tutorial we will be using a Gaussian Approximation Potentials to analyse results of TB DFT calculations of Si surface. Along the way we will learn about different descriptors (2b, 3b, soap) to describe local atomic environment in order to predict energies and forces of Si surface. </p>
</div>
</div>
<h3 class="display-4">Organic molecules</h3>
<p></p>
<div class="list-group">
<a href="soap-atomic-charges/soap_atomic_charges.ipynb" class="list-group-item list-group-item-action active">
<h3>Learning atomic charges</h3>
</a>
<div class="list-group-item list-group-item-action">
<i>created by: Gábor Csányi, James R. Kermode</i>
<p>In this tutorial, we will use Gaussian process regression, GPR (or equivalently, Kernel Ridge Regression, KRR) to train and predict charges of atoms in small organic molecules.</p>
</div>
</div>
%% Cell type:code id: tags:
``` python
```
analytics-asevisualization @ b066d500
Subproject commit b066d5008def2c7e6448228cc3906fb3e204f68e
analytics-band-structure-visualization @ 6cb5a929
Subproject commit 6cb5a929b8518350bd78437e52b18316e2eddac3
analytics-compressed-sensing @ 4831aadd
Subproject commit 4831aadd969af3a9b4e06de9156d8011c5f3320d
analytics-compressed-sensing-2018 @ 355bb9b9
Subproject commit 355bb9b94250f97c6b0b33ff3dae392a7e96a2ea
analytics-convolutional-nn @ 5f56eeff
Subproject commit 5f56eeff559eb78d566a004542b238f65b94a598
analytics-gap-si-surface @ e44c0a5e
Compare e44c0a5e...e44c0a5e
File moved
analytics-grain-boundaries @ 349edc88
Subproject commit 349edc88d9e18f03b9d3a15cb6f11981919de914
analytics-kaggle-competition @ beff593b
Subproject commit beff593b8420933e59a151fd8e677dcf41e49bf6
analytics-soap-atomic-charges @ fdbadffa
Subproject commit fdbadffaa584cb6f19fe3662c79d474db40b47e2
analytics-tcmi @ 5d21883b
File moved
asevisulization @ 2feb5533
Compare 2feb5533...2feb5533
Subproject commit 2feb55330b1dae4d4e7dc843601bb77d3a76a94f
band-structure-visualization @ 5a93375a
Compare 5a93375a...5a93375a
Subproject commit 5a93375a303c720d94a9ad9ef614f1e748512c44
compressed-sensing @ aac054d7
Compare aac054d7...aac054d7
Subproject commit aac054d78c43162020a159823db7bde8e72c9b4b
compressed-sensing-2018 @ 4c2e3c3b
Compare 4c2e3c3b...4c2e3c3b
Subproject commit 4c2e3c3b43222d807dab87b2f836e22629a10cb0
convolutional-nn @ e564a18b
Compare e564a18b...e564a18b
Subproject commit e564a18bea59bacbddf7746df2c865821ed7fc11
face-of-crystals @ 3843a5c1
Compare 3843a5c1...3843a5c1
Subproject commit 3843a5c1b6874877bf2525ef395821be69aeb8b3
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment