Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
P
Python for HPC Exercises
Manage
Activity
Members
Code
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
mpcdf
Python for HPC Exercises
Commits
4bd66e9d
Commit
4bd66e9d
authored
3 years ago
by
Repo Updater
Browse files
Options
Downloads
Patches
Plain Diff
2bd4d3df minor updates to 1b and 1c
parent
bcfda603
No related branches found
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
notebooks/1b--Introduction.ipynb
+2
-2
2 additions, 2 deletions
notebooks/1b--Introduction.ipynb
notebooks/1c--Python_Refresher.ipynb
+1
-1
1 addition, 1 deletion
notebooks/1c--Python_Refresher.ipynb
with
3 additions
and
3 deletions
notebooks/1b--Introduction.ipynb
+
2
−
2
View file @
4bd66e9d
...
@@ -29,7 +29,7 @@
...
@@ -29,7 +29,7 @@
"* Efficiency with respect to development effort: \n",
"* Efficiency with respect to development effort: \n",
" $\\rightarrow$ Python helps you to get \"things done\" quickly and focus on science.\n",
" $\\rightarrow$ Python helps you to get \"things done\" quickly and focus on science.\n",
"* Efficiency with respect to hardware utilization: \n",
"* Efficiency with respect to hardware utilization: \n",
" $\\rightarrow$ Python-based codes may perform well if implemented
correct
ly.\n",
" $\\rightarrow$ Python-based codes may perform well if implemented
proper
ly.\n",
"* Using Python may achieve a good overall efficiency for small- to medium-scale codes\n",
"* Using Python may achieve a good overall efficiency for small- to medium-scale codes\n",
"* And finally ... Python is already there, so we have do deal with it!"
"* And finally ... Python is already there, so we have do deal with it!"
]
]
...
@@ -101,7 +101,7 @@
...
@@ -101,7 +101,7 @@
"* Refresher of Python and the Python Ecosystem\n",
"* Refresher of Python and the Python Ecosystem\n",
"* Scientific Computing with Python: NumPy and SciPy\n",
"* Scientific Computing with Python: NumPy and SciPy\n",
"* Performance: Cython, Numba, C/Fortran interfacing\n",
"* Performance: Cython, Numba, C/Fortran interfacing\n",
"* Parallelism: multiprocessing, mpi4py\n",
"* Parallelism:
multithreading,
multiprocessing, mpi4py\n",
"* Software Engineering"
"* Software Engineering"
]
]
},
},
...
...
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# Introduction
# Introduction
**Python for HPC course**
**Python for HPC course**
2018-2021 Sebastian Ohlmann, Klaus Reuter
2018-2021 Sebastian Ohlmann, Klaus Reuter
Max Planck Computing and Data Facility, Garching
Max Planck Computing and Data Facility, Garching
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Why Python in HPC?
## Why Python in HPC?
*
Efficiency with respect to development effort:
*
Efficiency with respect to development effort:
$
\r
ightarrow$ Python helps you to get "things done" quickly and focus on science.
$
\r
ightarrow$ Python helps you to get "things done" quickly and focus on science.
*
Efficiency with respect to hardware utilization:
*
Efficiency with respect to hardware utilization:
$
\r
ightarrow$ Python-based codes may perform well if implemented
correct
ly.
$
\r
ightarrow$ Python-based codes may perform well if implemented
proper
ly.
*
Using Python may achieve a good overall efficiency for small- to medium-scale codes
*
Using Python may achieve a good overall efficiency for small- to medium-scale codes
*
And finally ... Python is already there, so we have do deal with it!
*
And finally ... Python is already there, so we have do deal with it!
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Examples for "real" HPC Codes using Python
## Examples for "real" HPC Codes using Python
### [GPAW](https://wiki.fysik.dtu.dk/gpaw/)
### [GPAW](https://wiki.fysik.dtu.dk/gpaw/)
*
Density-functional theory code for material science
*
Density-functional theory code for material science
*
Implemented as a Python module (C++ core, Python UI)
*
Implemented as a Python module (C++ core, Python UI)
*
Parallelization: MPI + OpenMP, scales to O(10k) cores
*
Parallelization: MPI + OpenMP, scales to O(10k) cores
### [ESPResSo++](http://www.espresso-pp.de)
### [ESPResSo++](http://www.espresso-pp.de)
*
Molecular dynamics package for soft matter simulation,
*
Molecular dynamics package for soft matter simulation,
MPI for Polymer Research, Mainz
MPI for Polymer Research, Mainz
*
Implemented as a Python module (C++ core, Python UI)
*
Implemented as a Python module (C++ core, Python UI)
*
Parallelization: Domain decomposition using MPI
*
Parallelization: Domain decomposition using MPI
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Use cases often relevant to our audience
## Use cases often relevant to our audience
*
prototype implementations of new methods that should run (much) faster
*
prototype implementations of new methods that should run (much) faster
*
data analysis scripts that should run faster and/or operate in parallel
*
data analysis scripts that should run faster and/or operate in parallel
*
IO-handling of large numerical data
*
IO-handling of large numerical data
$
\t
o$ very often, nice speedups can be gained without too much effort by applying the right techniques and tricks
$
\t
o$ very often, nice speedups can be gained without too much effort by applying the right techniques and tricks
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Scope of this tutorial
## Scope of this tutorial
*
Learn about the tools and techniques to write efficient Python code for scientific computing
*
Learn about the tools and techniques to write efficient Python code for scientific computing


%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Course Outline
## Course Outline
*
Introduction
*
Introduction
*
Refresher of Python and the Python Ecosystem
*
Refresher of Python and the Python Ecosystem
*
Scientific Computing with Python: NumPy and SciPy
*
Scientific Computing with Python: NumPy and SciPy
*
Performance: Cython, Numba, C/Fortran interfacing
*
Performance: Cython, Numba, C/Fortran interfacing
*
Parallelism: multiprocessing, mpi4py
*
Parallelism:
multithreading,
multiprocessing, mpi4py
*
Software Engineering
*
Software Engineering
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
### *Python for HPC* complements advanced HPC courses
### *Python for HPC* complements advanced HPC courses
*
We cannot cover traditional HPC topics in depth, and would highly recommend the following courses
*
We cannot cover traditional HPC topics in depth, and would highly recommend the following courses
*
Parallel programming with MPI
*
Parallel programming with MPI
*
Parallel programming with OpenMP
*
Parallel programming with OpenMP
*
Advanced parallel programming with OpenMP and MPI
*
Advanced parallel programming with OpenMP and MPI
*
Node-level performance engineering
*
Node-level performance engineering
*
Watch out for these courses which are regularly offered by HLRS, LRZ, RRZE
*
Watch out for these courses which are regularly offered by HLRS, LRZ, RRZE
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## About this Python for HPC tutorial
## About this Python for HPC tutorial
*
Practical hands-on approach with code examples
*
Practical hands-on approach with code examples
*
Presentation is based on Jupyter notebooks
*
Presentation is based on Jupyter notebooks
*
Course material available for download at
*
Course material available for download at
https://gitlab.mpcdf.mpg.de/mpcdf/python-for-hpc-exercises
https://gitlab.mpcdf.mpg.de/mpcdf/python-for-hpc-exercises
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
## Software prerequisites for the Jupyter notebooks and exercises
## Software prerequisites for the Jupyter notebooks and exercises
*
Python >=3.6
*
Python >=3.6
*
Jupyter
*
Jupyter
*
NumPy, SciPy
*
NumPy, SciPy
*
matplotlib
*
matplotlib
*
Cython
*
Cython
*
gcc, gfortran
*
gcc, gfortran
*
Numba
*
Numba
*
(MPI, mpi4py)
*
(MPI, mpi4py)
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
### Option 1: Cloud-based MPCDF Jupyter notebook service (beta)
### Option 1: Cloud-based MPCDF Jupyter notebook service (beta)
*
Use the link communicated by email to access a Jupyter service on the MPCDF HPC cloud
*
Use the link communicated by email to access a Jupyter service on the MPCDF HPC cloud
*
The course material
*and*
software are provided via an interactive JupyterLab interface
*
The course material
*and*
software are provided via an interactive JupyterLab interface
*
Each instance provides up to 6 virtual CPU cores and up to 12 GB RAM (less guaranteed)
*
Each instance provides up to 6 virtual CPU cores and up to 12 GB RAM (less guaranteed)
*
Please keep the following points in mind
*
Please keep the following points in mind
*
Use the JupyterLab menu
**File $\to$ Shut down**
to free resources when finished
*
Use the JupyterLab menu
**File $\to$ Shut down**
to free resources when finished
*
A session is terminated after 12h
*
A session is terminated after 12h
*
The storage is only temporary, i.e. download your modified notebooks and data in time via the GUI
*
The storage is only temporary, i.e. download your modified notebooks and data in time via the GUI
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
### Option 2: MPCDF Python infrastructure on the HPC systems
### Option 2: MPCDF Python infrastructure on the HPC systems
*
Python (
**3.8**
) is provided via the Anaconda Python Distribution
*
Python (
**3.8**
) is provided via the Anaconda Python Distribution
*
software is accessible via environment modules, e.g.
*
software is accessible via environment modules, e.g.
`module purge`
`module purge`
`module load gcc/10 impi/2019.9`
`module load gcc/10 impi/2019.9`
`module load anaconda/3/2021.05`
`module load anaconda/3/2021.05`
`module load mpi4py/3.0.3`
`module load mpi4py/3.0.3`
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
### Option 3: Python environment for your local computer
### Option 3: Python environment for your local computer
*
On a recent Linux system, install all the necessary packages via the package manager
*
On a recent Linux system, install all the necessary packages via the package manager
*
Alternatively, install the Anaconda Python Distribution (https://www.anaconda.com)
*
Alternatively, install the Anaconda Python Distribution (https://www.anaconda.com)
...
...
This diff is collapsed.
Click to expand it.
notebooks/1c--Python_Refresher.ipynb
+
1
−
1
View file @
4bd66e9d
...
@@ -754,7 +754,7 @@
...
@@ -754,7 +754,7 @@
"metadata": {
"metadata": {
"scrolled": true,
"scrolled": true,
"slideshow": {
"slideshow": {
"slide_type": "
subslide
"
"slide_type": "
fragment
"
}
}
},
},
"outputs": [
"outputs": [
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment