Commit 01902d5c authored by Volker Springel's avatar Volker Springel

some more edits in documentation

parent 82be2881
......@@ -2,77 +2,98 @@ Code development
************************
Scientific software development is a crucial aspect of computational (astro-)physics.
With the progess made over the past decades, numerical methods as well as implementations
have evolved and increased in complexity. The scope of simulation codes has
increased accordingly, such that the question how to organize development becomes important
to scientific progress.
This version of Arepo is intended as a basis, providing an efficient code structure
and state of the art numerical techniques for gravity and hydrodynamics. The code is
completely documented and should allow computational astrophysicists to develop their
own specialized modules on top of it. Practically, this is best done by hosting an
own repository for the development, which is occasonally updated from the main
repository to include latest bugfixes. For this workflow to work properly, it is
important to keep the code-base relatively static. For this reason
**the base version of the code is not meant to be extended in functionality.**
This means that only bugfixes as well as updates in the documentation and examples will
be included.
Scientific software development is a crucial aspect of computational
(astro-)physics. With the progess made over the past decades,
numerical methods as well as implementations have evolved and
increased in complexity. The scope of simulation codes has increased
accordingly, such that the question on how to organize development
becomes important for further scientific progress.
This version of Arepo is intended as a basis, providing an efficient
code structure and state of the art numerical techniques for gravity
and hydrodynamics. The code is completely documented and should allow
computational astrophysicists to develop their own specialized modules
on top of it. Practically, this is best done by hosting an own
repository for the development, which is occasonally updated from the
main repository to include latest bugfixes. For this workflow to work
properly, it is helpful to keep the code-base relatively static. For
this reason
**the base version of the code is not meant to be extended in
functionality.**
This means that only bugfixes as well as updates in the documentation
and examples will be included in revisions of this public release.
Issue reporting
===============
A code of the scope of Arepo will ineviably have a number of issues and so far undetected bugs.
Detection of all issues is very complicated and time-consuming work. This means in practice that
we rely on users that actually report all bugs and issues they come across to improve the quality
of the code. We therefore encourage users to report all issues they have, including things
that remain unclear after reading the documentation. This way we hope to constantly improve the
qualtiy of the code in all its aspects.
A code of the scope of Arepo will inevitably have a number of issues
and so far undiscovered bugs. Detection of all issues is very
complicated and time-consuming work. This means in practice that we
rely on users to actually report all bugs and issues they come across,
helping us to improve the quality of the code. We therefore encourage
users to report all issues they have, including things that remain
unclear after reading the documentation. This way we hope to
constantly improve the qualtiy of the code in all its aspects.
To organize this, we created a code support forum (www.arepo-code.org/forums/forum/arepo-forum)
which is publically visible. To create posts on this forum, a registration with admin-approval
is required. We encourage users to sign up under www.arepo-code.org/register and make
use of the forum whenever they have problems.
To organize this, we set up a code support forum
(www.arepo-code.org/forums/forum/arepo-forum) which is publicly
visible. To create posts on this forum, a registration with
admin-approval is required. We encourage users to sign up under
www.arepo-code.org/register and make use of the forum whenever they
have problems.
Issues reported in the forum will then be confirmed by one of the authors and added to the
repository issue tracker, which serves as a to-do list for bug fixes and improvements to
the code. Therefore, if a problem occurs, the first thing to check is whether there already exists
an open issue, i.e. whether this is a known and confirmed problem.
Issues reported in the forum will then be confirmed by one of the
authors and added to the repository issue tracker, which serves as a
to-do list for bug fixes and improvements to the code. Therefore, if a
problem occurs, the first thing to check is whether there already
exists an open issue, i.e. whether this is a known and confirmed
problem.
**Please use the support forum instead of contacting the authors by email.**
**Please whenever possible use the support forum instead of contacting
the authors by email.**
**We encourage experienced users to provide help and answer some of the open questions on this forum.**
**We encourage experienced users to provide help and answer some of
the open questions on this forum.**
Code extensions
===============
Extensions should be hosted in separate repositories or branches of the main repository.
We highly welcome such extensions and encourage developers to make them publically
available (under GNU GPL v3). While this can be done independently of the authors,
we would encourage developers to inform the authors once there is a publically
available module in order to have a list of available extensions on the code homepage.
Extensions should be hosted in separate repositories or branches of
the main repository. We highly welcome such extensions and encourage
developers to make them publicly available (under GNU GPL v3). While
this can be done independently of the authors, we would encourage
developers to inform the authors once there is a publicly available
module they are willing to share, in order to have a list of available
extensions on the code homepage.
Some guidelines for code extensions and new modules:
* Modularity
* Strive for modularity
* Minimize the number of changes in existing source code files to a few function calls
and structured variables within existing elements.
* The source code of an own modue should largely be in separate (i.e. new) files.
* Minimize the number of changes in existing source code files to a
few function calls and structured variables within existing
elements.
* The source code of a new modue should largely be in separate
(i.e. new) files.
* Documentation
* Document the module
* All parameter and config-options should be clearly explained in this documentation.
Feel free to add an addional page to this documentation explaining how your module
works.
* All parameter and config-options should be clearly explained in
this documentation. Feel free to add an addional page to this
documentation explaining how your module works.
* Document what each function does in the source code.
* The Template-Config.sh file should have a short explanation for each flag.
* The Template-Config.sh file should have a short explanation for
each added flag.
* Verification and examples
* Verification and examples
* If possible, create one or more addional examples illustrating and testing the module.
* If possible, create one or more addional examples illustrating and
testing the module.
......@@ -2,18 +2,18 @@
Diagnostic output
*****************
Arepo will not only output the simulation snapshot and reduced data via
the halo-finder files, but also a number of (mostly ascii) diagnostic log-
files which contain important information about the code performance and
runtime behavior.
In practice, to quickly check the performance of large
production runs, it is useful to check the ``timebins.txt``
and ``cpu.txt`` files. The former will give information how many simulation
elements are on which timestep, i.e. characteristics of the system
simulated, the latter provides information about the computational
time spent in each part of the code, which can be influenced to some
degree by the values of the code parameters.
Arepo will not only output the simulation snapshot and reduced data
via the halo-finder files, but also a number of (mostly ascii)
diagnostic log- files which contain important information about the
code performance and runtime behavior.
In practice, to quickly check the performance of large production
runs, it is useful to check the ``timebins.txt`` and ``cpu.txt``
files. The former will give information how many simulation elements
are evolved with which timestep sizes, i.e. characteristics of the
system simulated, the latter provides information about the
computational time spent in each part of the code, which can be
influenced to some degree by the values of the code parameters.
For ongoing simulations, these can be checked via
......@@ -26,16 +26,16 @@ For ongoing simulations, these can be checked via
stdout
======
The standard output contains general information about the simulation status
and many of the main routines will print general information in it.
The output itself is mainly relevant for reconstructing what the simulation
did, which is needed e.g. for debugging purposes.
The standard output contains general information about the simulation
status and many of the main routines will print general information in
it. The output itself is mainly relevant for reconstructing what the
simulation did, which is needed e.g. for debugging purposes.
balance.txt
===========
Output of fractional cpu time used in each individual step, optimized to be
machine readable (while cpu.txt is more human readable).
Output of fractional cpu time used in each individual step, optimized
to be machine readable (while cpu.txt is more human readable).
Symbol key
......@@ -103,21 +103,24 @@ example:
cpu.txt
=======
Each sync-point, such a block is written. This file
reports the result of the different timers built into Arepo. Each
computationally expensive operation has a different timer attached to it and
this way allows to closely monitor what the computational time is spent on.
Some of the timers (e.g. treegrav) have sub-timers for individual operations.
This is denoted by the indentation hierarchy in the first column.
The distribution of these timings is highly problem dependent, but it is
possible to identify inefficient parts of the overall algorithm and optimize
only the most time-consuming parts of the code. There is the option
``OUTPUT_CPU_CSV`` which also returns this data as a ``cpu.csv`` file.
The different columns are:
name; wallclock time (in s) this step; percentage this step; wallclock time
(in s) cumulative; percentage up to this step. A typical block of cpu.txt looks
the following (here a gravity-only, tree-only run):
For each sync-point, such a block is written. This file reports
measurements of the different timers built into Arepo. Each
computationally expensive operation has a different timer attached to
it, thus allowing to closely monitor where the computational time is
spent. Some of the timers (e.g. treegrav) have sub-timers for
individual operations. This is denoted by the indentation hierarchy
in the first column. The fraction of time spent in different code
parts, as well as the absolute amount, is highly problem
dependent. The timers make it possible to identify inefficient parts
of the overall algorithm and concentrate on the most time-consuming
parts of the code. There is also the option ``OUTPUT_CPU_CSV`` which
returns thes same data as a more easily machine-readable ``cpu.csv``
file.
The different columns are: name; wallclock time (in s) this step;
percentage this step; wallclock time (in s) cumulative; percentage up
to this step. A typical block of cpu.txt looks the following (here a
gravity-only, tree-only run):
.. code-block :: python
......@@ -169,13 +172,13 @@ the following (here a gravity-only, tree-only run):
misc 0.00 0.0% 0.02 0.0%
domain.txt
domain.txt
==========
The load-balancing (cpu work and memory) both in gravity and hydro calculation
are reported for each timebin individually. Reported every sync-point.
Ideally balanced runs have the value 1, the higher the value, the more
imbalanced the simulation.
The load-balancing (cpu work and memory) both for gravity and hydro
calculations is reported for each timebin individually. Reported every
sync-point. Ideally balanced runs have a value 1, the higher the
value, the more imbalanced the simulation.
.. code-block :: python
......@@ -195,15 +198,15 @@ imbalanced the simulation.
--------------------------------------------------------------------
-----------------
energy.txt
energy.txt
==========
In specified intervals (in simulation time, specified by the parameter
`TimeBetStatistics`) the total energy and its components are computed and
written into `energy.txt`. This file also contains the cumulative energy
that had to be injected into the system to ensure positivity in thermal energy.
All output in code units. Note: this only works with up to 6 particle types.
The columns are
In specified intervals (in simulation time, specified by the parameter
`TimeBetStatistics`) the total energy and its components are computed
and written into the file `energy.txt`. This file also contains the
cumulative energy that had to be injected into the system to ensure
positivity in thermal energy. All output is in code units. Note: this
only works with up to 6 particle types. The columns are
.. code-block :: python
......@@ -249,11 +252,11 @@ Two example lines:
4203e+07 0 0 0 0 0 0 0 0 1.76774e+06 0 0 0 503.306 3047.89 0 0 65.75
86 0 7.71477
info.txt
info.txt
========
Every sync-point, the time-bins, time, timestep and number of active particles
are written into this file, e.g.
Every sync-point, the time-bins, time, timestep and number of active
particles are written into this file, e.g.
.. code-block :: python
......@@ -264,14 +267,15 @@ are written into this file, e.g.
memory.txt
==========
Arepo internally uses an own memory manager. This means that one large chunk of
memory is reserved initially for Arepo (specified by the parameter
`MaxMemSize`) and allocation for individual arrays is handled internally.
The reason for introducing this was to avoid memory fragmentation during
runtime on some machines, but also to have detailed information about how much
memory Arepo actually needs and to terminate if this exceeds a pre-defined
threshold. ``memory.txt`` reports this internal memory usage, and how much memory
is actually needed by the simulation.
Arepo uses its own internal memory manager. This means that one large
chunk of memory is reserved initially for Arepo (specified by the
parameter `MaxMemSize`), and the allocation for individual arrays is
then handled internally from this pool. The reason for introducing
this was to avoid memory fragmentation during runtime on some
machines, but also to have detailed information about how much memory
Arepo actually needs and to terminate if this exceeds a pre-defined
threshold. ``memory.txt`` reports this internal memory usage, and how
much memory is actually needed by the simulation.
.. code-block :: python
......@@ -461,13 +465,15 @@ is actually needed by the simulation.
sfr.txt
=======
In case ``USE_SFR`` is active, Arepo will create a ``sfr.txt`` file, which reports
the stars created in every call of the star-formation routine.
In case ``USE_SFR`` is active, Arepo will create a ``sfr.txt`` file,
which reports the stars created in every call of the star-formation
routine.
The individual columns are:
* time (code units or scale factor)
* total stellar mass to be formed in timestepo prior to stochastic sampling (code units),
* total stellar mass to be formed in timestepo prior to stochastic
sampling (code units),
* instantaneous star formation rate of all cells (Msun/yr),
* instantaneous star formation rate of active cells (Msun/yr),
* total mass in stars formed in this timestep (after sampling) (code units),
......@@ -490,14 +496,16 @@ Example:
timebins.txt
============
Arepo is optimized for time-integrating both hydrodynamical as well as
gravitational interactions on the largest possible timestep that is allowed by
the timestep criterion and allowed by the binary hierarchy of time steps.
Each for each timestep, a linked list of particles on this particular
integration step exists, and their statistics are reported in `timebins.txt`.
In this file, the number of gas cells and collisionless particles in each
timebin (i.e. integration timestep) is reported for each sync-point, as well
as the cpu time and fraction spent on each timebin. A typical bock looks like
Arepo is optimized for time-integrating both hydrodynamical as well as
gravitational interactions on the largest possible timestep that is
allowed by the timestep criterion and allowed by the binary hierarchy
of time steps. For each timestep, a linked list of particles on this
particular integration step exists, and their statistics are reported
in `timebins.txt`. In this file, the number of gas cells and
collisionless particles in each timebin (i.e. integration timestep) is
reported for each sync-point, as well as the cpu time and the fraction
of the total cost contributed by each timebin. A typical block looks
like
.. code-block :: python
......@@ -516,12 +524,13 @@ as the cpu time and fraction spent on each timebin. A typical bock looks like
------------------------
Total active: 185 143
timings.txt
timings.txt
===========
The performance of the gravitational tree algorithm is reported in
`timings.txt` for each sync-point. An example of a single sync-point looks
the following
The performance of the gravitational tree algorithm is reported in
`timings.txt` for each sync-point. An example of a single sync-point
looks like the following
.. code-block :: python
......
......@@ -2,50 +2,60 @@
Related codes
*************
The general code structure of Arepo is similar to the Gadget code, but with
significant further developments and improvements. However, keeping in mind
that the code architecture originates from an Tree-PM-SPH code helps to
understand why certain things are done in certain ways.
One of the obvious signs of this are the name of some global structures and
functions. For example the array holding the information for gas cells only is
called SphP. Another example would be the routines in ``src/init/density.c``,
which used to be an SPH density calculation, but now only used for an estimate
for the typical distance to neighbouring cells. Another aspect is the
presence of the neighbour tree, a search-tree structure on gas cells. In
practice this means that many of the routines that depend on the state of
neighbouring cells, including sub-grid feedback models etc., can be based on
this neighbour search. This makes it possible to port these models from earlier
versions of Gadget to Arepo.
However, it is important to realize that routines written for the Gadget code
will in general not work directly when ported to Arepo. We encourage developers
to carefully test that the implementation is still having the same effect.
Apart from the differences in implementation, it is also important to realize
that a grid-based finite volume scheme might react different to a particular to
a sub-grid implementation.
The general high-level code structure of Arepo is quite similar to the
Gadget code, but with significant changes and improvements. However,
keeping in mind that the code architecture originates from an
Tree-PM-SPH code can help to understand why certain things are done in
certain ways.
One of the obvious signs of this are the names of some global
structures and functions. For example the array holding the
information for gas cells is called SphP. Another example would be the
routines in ``src/init/density.c``, which used to be an SPH density
calculation, but is now only used for an estimate for the typical
distance to neighbouring cells. Another aspect is the presence of the
neighbour tree, a search-tree structure on gas cells. In practice this
means that many of the routines that depend on the state of
neighbouring cells, including sub-grid feedback models etc., can be
based on this neighbour search. This makes it possible to port these
models relatively easily from earlier versions of Gadget to Arepo.
However, it is important to also realize that routines written for the
Gadget code will in general not work without some porting adjustments
when inserted in to Arepo. We thus encourage developers to carefully
test that their implementations still have the intended effect when
such a porting is done. Apart from the differences in implementation,
it is also important to realize that a grid-based finite volume scheme
can react differently to certain sub-grid implementations than a
particle-based approach.
Differences to the development Version
======================================
This public version of Arepo was branched off from the development version in
November 2017, substantially cut down, cleand up and the documentation completed
by Rainer Weinberger. The main reason for doing this is to provide a code to the
community that researchers are able to understand and use without the need of
an extensive introduction by one of the authors. We belive that the public
version, being better documented on a developer level, also provides a better
starting point for new developments.
The general idea for this public verson was to preserve the well-tested code
with a limited number of changes. However, some compile-time options have been
eliminated and to recover the same running mode **in the development version
of Arepo**, the following compile-time flags need to be set in `Config.sh`
This public version of Arepo was branched off from the development
version in November 2017, and then Rainer Weinberger substantially cut
it down, cleaned it up and completed the documentation. The main
reason for doing this was to provide a code to the community that
researchers are able to understand and use without the need of an
extensive introduction by one of the authors. We believe that the
public version, which is better documented at a developer level, also
provides a better starting point for new developments.
The general idea for this public verson was to preserve the
well-tested code with a limited number of changes. However, some
compile-time options have been eliminated and to recover the same
running mode **in the development version of Arepo**, the following
compile-time flags need to be set in `Config.sh`
* ``PERIODIC`` (has become obsolete over the years)
* ``VORONOI`` (the development version also has an AMR mode)
* ``VORONOI_DYNAMIC_UPDATE`` (in practice now a default)
* ``AUTO_SWAP_ENDIAN_READIC`` (only relevant for file formats 1 and 2)
* ``CHUNKING``
* the switch ``EXACT_GRAVITY_FOR_PARTICLE_TYPE`` automatically activates ``NO_SELFGRAVITY_TYPE``, ``NO_GRAVITY_TYPE`` and ``EXACT_GRAVITY_REACTION`` (see src/main/allvars.h) in the public version. This is not the case in the development version where all these flags need to be set individually in Config.sh.
* the switch ``EXACT_GRAVITY_FOR_PARTICLE_TYPE`` automatically
activates ``NO_SELFGRAVITY_TYPE``, ``NO_GRAVITY_TYPE`` and
``EXACT_GRAVITY_REACTION`` (see src/main/allvars.h) in the public
version. This is not the case in the development version where all
these flags need to be set individually in Config.sh.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment