fa2016cc add forwarder to Binder

9f85a470 · Repo Updater · f51e44fd · 9f85a470
Commit 9f85a470 authored 1 year ago by Repo Updater
--- a/notebooks/01--Introduction.ipynb
+++ b/notebooks/01--Introduction.ipynb
@@ -104,10 +104,11 @@
    "\n",
    "* Introduction\n",
    "* Refresher of Python and the Python Ecosystem\n",
+    "* Basics of High Performance Computing\n",
    "* Scientific Computing with Python: NumPy and SciPy\n",
    "* Performance: Cython, JIT (Numba, JAX), C/Fortran interfacing\n",
    "* Parallelism: multithreading, multiprocessing, GPU computing, Dask, mpi4py\n",
-    "* Software Engineering, Packaging"
+    "* Software Engineering, Packaging, Profiling, Visualization"
   ]
  },
  {
@@ -175,7 +176,7 @@
    "\n",
    "* Use the link communicated by email to access a Jupyter service on the MPCDF HPC cloud\n",
    "* The course material *and* software are provided via an interactive JupyterLab interface\n",
-    "* Each instance provides *up to* 4 virtual CPU cores and *up to* 6 GB RAM\n",
+    "* Each instance provides access to several virtual CPU cores and a few GB of RAM\n",
    "* Please keep the following points in mind\n",
    "    * Use the JupyterLab menu **File $\\to$ Shut down** to free resources when finished\n",
    "    * A session is terminated after 12h\n",
@@ -230,7 +231,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.10.12"
  }
 },
 "nbformat": 4,

 %% Cell type:markdown id: tags:

 # Introduction

 **Python for HPC course**

 Sebastian Kehl, Sebastian Ohlmann, Klaus Reuter

 _25-27 July 2023_

 HPC Application Support Group

 Max Planck Computing and Data Facility, Garching

 %% Cell type:markdown id: tags:

 ## Why Python in HPC?
 * Efficiency with respect to development effort:
  $\rightarrow$ Python helps you to get "things done" quickly and focus on your science.
 * Efficiency with respect to hardware utilization:
  $\rightarrow$ Python-based codes may perform well if implemented properly.
 * Using Python may achieve a good overall efficiency for small- to medium-scale codes
 * And finally ... Python is already there, so we have do deal with it!

 %% Cell type:markdown id: tags:

 ## Examples for "real" HPC Codes using Python

 ### [GPAW](https://wiki.fysik.dtu.dk/gpaw/)

 * Density-functional theory code for material science
 * Implemented as a Python module (C core, Python UI)
 * Parallelization: MPI + OpenMP, scales to O(10k) cores

 ### [ESPResSo++](http://www.espresso-pp.de)

 * Molecular dynamics package for soft matter simulation,
  MPI for Polymer Research, Mainz
 * Implemented as a Python module (C++ core, Python UI)
 * Parallelization: Domain decomposition using MPI

 %% Cell type:markdown id: tags:

 ##  Use cases often relevant to our audience

 * prototype implementations of numerical or AI methods that should run (much) faster when going towards production
 * data analysis scripts that should run faster and/or operate in parallel
 * IO-handling and processing of large numerical data

 $\to$ very often, efficient hardware utilization and good speedups can be gained without too much effort by applying the right techniques and tricks

 %% Cell type:markdown id: tags:

 ## Scope of this tutorial
 * Learn about the tools and techniques to write efficient Python code for scientific computing
 ![gain vs effort](fig/gain_vs_effort.svg)

 %% Cell type:markdown id: tags:

 ## Course Outline

 * Introduction
 * Refresher of Python and the Python Ecosystem
+* Basics of High Performance Computing
 * Scientific Computing with Python: NumPy and SciPy
 * Performance: Cython, JIT (Numba, JAX), C/Fortran interfacing
 * Parallelism: multithreading, multiprocessing, GPU computing, Dask, mpi4py
-* Software Engineering, Packaging
+* Software Engineering, Packaging, Profiling, Visualization

 %% Cell type:markdown id: tags:

 ### *Python for HPC* complements advanced HPC courses
 * We cannot cover traditional HPC topics in depth, and would highly recommend the following courses
    * Node-level performance engineering
    * Parallel programming with MPI
    * Parallel programming with OpenMP
    * Advanced parallel programming with OpenMP and MPI
 * Watch out for these courses which are regularly offered by HLRS, LRZ, NHR/RRZE

 %% Cell type:markdown id: tags:

 ## About this Python for HPC tutorial

 * Practical hands-on approach with code examples
 * Presentation is based on Jupyter notebooks
 * Course material available for download at
  https://gitlab.mpcdf.mpg.de/mpcdf/python-for-hpc-exercises

 %% Cell type:markdown id: tags:

 ## Software prerequisites for the Jupyter notebooks and exercises
 * Python >=3.6
 * Jupyter
 * NumPy, SciPy
 * matplotlib
 * Cython
 * gcc, gfortran
 * Numba
 * (CuPy, JAX)
 * (Dask, Dask-MPI, MPI, mpi4py)

 %% Cell type:markdown id: tags:

 ### Software Option 1: Cloud-based MPCDF Jupyter notebooks

 * Use the link communicated by email to access a Jupyter service on the MPCDF HPC cloud
 * The course material *and* software are provided via an interactive JupyterLab interface
-* Each instance provides *up to* 4 virtual CPU cores and *up to* 6 GB RAM
+* Each instance provides access to several virtual CPU cores and a few GB of RAM
 * Please keep the following points in mind
    * Use the JupyterLab menu **File $\to$ Shut down** to free resources when finished
    * A session is terminated after 12h
    * The storage is only temporary, i.e. download your modified notebooks and data in time via the GUI

 %% Cell type:markdown id: tags:

 ### Software Option 2: Python infrastructure on the MPCDF HPC systems
 * Python is provided via the Anaconda Python Distribution
 * software is accessible via environment modules, e.g.
    `module purge`
    `module load gcc/12 impi/2021.9`
    `module load anaconda/3/2023.03`
    `module load mpi4py/3.1.4`

 %% Cell type:markdown id: tags:

 ### Software Option 3: Python environment for your local computer
 * On a recent Linux system, install all the necessary packages via the package manager
 * Alternatively, install the Anaconda Python Distribution (https://www.anaconda.com) and use the `conda` package manager and the file `environment.yml` to add all necessary packages