Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
ducc
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Incidents
Environments
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Simon Perkins
ducc
Commits
9846e31a
Commit
9846e31a
authored
Jun 04, 2020
by
Martin Reinecke
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
big bunch of reorganization
parent
43d0315c
Changes
24
Hide whitespace changes
Inline
Side-by-side
Showing
24 changed files
with
161 additions
and
55 deletions
+161
-55
.gitlab-ci.yml
.gitlab-ci.yml
+2
-3
MANIFEST.in
MANIFEST.in
+13
-15
README.md
README.md
+109
-0
python/alm.h
python/alm.h
+0
-0
python/demos/fft_bench.py
python/demos/fft_bench.py
+0
-0
python/demos/fft_stress.py
python/demos/fft_stress.py
+0
-0
python/demos/healpix_perftest.py
python/demos/healpix_perftest.py
+0
-0
python/demos/healpix_test.py
python/demos/healpix_test.py
+0
-0
python/demos/sht_demo.py
python/demos/sht_demo.py
+1
-1
python/demos/sht_upsample_demo.py
python/demos/sht_upsample_demo.py
+0
-0
python/ducc.cc
python/ducc.cc
+10
-10
python/fft.cc
python/fft.cc
+5
-5
python/gridder_cxx.h
python/gridder_cxx.h
+0
-0
python/healpix.cc
python/healpix.cc
+5
-5
python/sht.cc
python/sht.cc
+5
-5
python/test/test_fft.py
python/test/test_fft.py
+0
-0
python/test/test_healpix.py
python/test/test_healpix.py
+0
-0
python/test/test_sht.py
python/test/test_sht.py
+0
-0
python/test/test_totalconvolve.py
python/test/test_totalconvolve.py
+0
-0
python/test/test_wgridder.py
python/test/test_wgridder.py
+0
-0
python/totalconvolve.cc
python/totalconvolve.cc
+6
-6
python/totalconvolve.h
python/totalconvolve.h
+0
-0
python/wgridder.cc
python/wgridder.cc
+3
-3
setup.py
setup.py
+2
-2
No files found.
.gitlab-ci.yml
View file @
9846e31a
...
...
@@ -2,7 +2,6 @@ image: $CONTAINER_TEST_IMAGE
variables
:
CONTAINER_TEST_IMAGE
:
gitlab-registry.mpcdf.mpg.de/$CI_PROJECT_PATH:$CI_BUILD_REF_NAME
OMP_NUM_THREADS
:
2
stages
:
-
build_docker
...
...
@@ -21,13 +20,13 @@ test_gcc:
stage
:
testing
script
:
-
python3 setup.py install --user -f
-
pytest-3 -q
pypocketfft/test pysharp/test pyHealpix/test nifty_gridder/test pyinterpol_ng
/test
-
pytest-3 -q
fft/test sht/test healpix/test wgridder/test totalconvolve
/test
test_clang
:
stage
:
testing
script
:
-
CC="clang -fsized-deallocation" python3 setup.py install --user -f
-
pytest-3 -q
pypocketfft/test pysharp/test pyHealpix/test nifty_gridder/test pyinterpol_ng
/test
-
pytest-3 -q
fft/test sht/test healpix/test wgridder/test totalconvolve
/test
release
:
stage
:
build_tarballs
...
...
MANIFEST.in
View file @
9846e31a
prune .
include src/mr_util/infra/aligned_array.h
#include src/mr_util/infra/communication.cc
#include src/mr_util/infra/communication.h
include src/mr_util/infra/error_handling.h
include src/mr_util/infra/mav.h
include src/mr_util/infra/simd.h
...
...
@@ -13,8 +11,6 @@ include src/mr_util/infra/system.h
include src/mr_util/infra/threading.cc
include src/mr_util/infra/threading.h
include src/mr_util/infra/timers.h
#include src/mr_util/infra/types.cc
#include src/mr_util/infra/types.h
include src/mr_util/infra/useful_macros.h
include src/mr_util/bindings/pybind_utils.h
...
...
@@ -52,15 +48,17 @@ include src/mr_util/healpix/healpix_base.h
include src/mr_util/healpix/healpix_tables.cc
include src/mr_util/healpix/healpix_tables.h
include pypocketfft/pypocketfft.cc
include python/fft.cc
include python/sht.cc
include python/healpix.cc
include python/gridder_cxx.h
include python/wgridder.cc
include python/alm.h
include python/totalconvolve.h
include python/totalconvolve.cc
include pysharp/pysharp.cc
include pyHealpix/pyHealpix.cc
include nifty_gridder/gridder_cxx.h
include nifty_gridder/nifty_gridder.cc
include pyinterpol_ng/alm.h
include pyinterpol_ng/interpol_ng.h
include pyinterpol_ng/pyinterpol_ng.cc
include python/test/test_fft.py
include python/test/test_healpix.py
include python/test/test_sht.py
include python/test/test_totalconvolve.py
include python/test/test_wgridder.py
README.md
0 → 100644
View file @
9846e31a
DUCC 0.1
========
This is a collection of basic programming tools which can be handy in many
situations.
Installation
------------
### Requirements
-
[
Python 3
](
https://www.python.org/
)
-
[
pybind11
](
https://github.com/pybind/pybind11
)
-
a C++17-capable C++ compiler (e.g. g++ from version 7 on or a recent clang++)
### Sources
The current version of DUCC can be obtained by cloning the repository via
git clone https://gitlab.mpcdf.mpg.de/mtr/ducc.git
### Installation
In the following, we assume a Debian-based distribution. For other
distributions, the "apt" lines will need slight changes.
DUCC and its mandatory dependencies can be installed via:
sudo apt-get install git python3 python3-pip python3-dev python3-pybind11 pybind11-dev
pip3 install --user git+https://gitlab.mpcdf.mpg.de/mtr/ducc.git
DUCC components
===============
ducc.fft
--------
This package provides Fast Fourier, trigonometric and Hartley transforms with a
simple Python interface.
The central algorithms are derived from Paul Swarztrauber's FFTPACK code
(http://www.netlib.org/fftpack).
Features
-
supports fully complex and half-complex (i.e. complex-to-real and
real-to-complex) FFTs, discrete sine/cosine transforms and Hartley transforms
-
achieves very high accuracy for all transforms
-
supports multidimensional arrays and selection of the axes to be transformed
-
supports single, double, and long double precision
-
makes use of CPU vector instructions when performing 2D and higher-dimensional
transforms
-
supports prime-length transforms without degrading to O(N
**
2) performance
-
has optional multi-threading support for multidimensional transforms
ducc.sht
--------
This package provides efficient spherical harmonic trasforms (SHTs). Its code
is derived from
`libsharp`
.
ducc.healpix
------------
This library provides Python bindings for the most important
functionality in Healpix C++. The design goals are
-
similarity to the C++ interface (while respecting some Python peculiarities)
-
simplicity (no optional function parameters)
-
low function calling overhead
ducc.totalconvolve
------------------
Library for high-accuracy 4pi convolution on the sphere, which generates a
total convolution data cube from a set of sky and beam
`a_lm`
and computes
interpolated values for a given list of detector pointings.
Algorithmic details:
-
the code uses
`ducc.sht`
SHTs to compute the data cube
-
shared-memory parallelization is provided via standard C++ threads.
-
for interpolation, the algorithm and kernel described in
https://arxiv.org/abs/1808.06736 are used. This allows very efficient
interpolation with user-adjustable accuracy.
ducc.wgridder
-------------
Library for high-accuracy gridding/degridding of radio interferometry datasets
Programming aspects
-
written in C++11, fully portable
-
shared-memory parallelization via and C++ threads.
-
Python interface available
-
kernel computation is performed on the fly, avoiding inaccuracies
due to table lookup and reducing overall memory bandwidth
Numerical aspects
-
uses the analytical gridding kernel presented in
https://arxiv.org/abs/1808.06736
-
uses the "improved W-stacking method" described in
https://www.repository.cam.ac.uk/handle/1810/292298 (p. 139ff)
-
in combination these two aspects allow extremely accurate gridding/degridding
operations (L2 error compared to explicit DFTs can go below 1e-12) with
reasonable resource consumption
py
interpol_ng
/alm.h
→
py
thon
/alm.h
View file @
9846e31a
File moved
py
pocketfft/demos/
bench.py
→
py
thon/demos/fft_
bench.py
View file @
9846e31a
File moved
py
pocketfft/demos/
stress.py
→
py
thon/demos/fft_
stress.py
View file @
9846e31a
File moved
py
Healpix/pyH
ealpix_perftest.py
→
py
thon/demos/h
ealpix_perftest.py
View file @
9846e31a
File moved
py
Healpix/pyH
ealpix_test.py
→
py
thon/demos/h
ealpix_test.py
View file @
9846e31a
File moved
py
sharp/
demo.py
→
py
thon/demos/sht_
demo.py
View file @
9846e31a
# Elementary demo for
pysharp
interface using a Gauss-Legendre grid
# Elementary demo for
the ducc_0_1.sht
interface using a Gauss-Legendre grid
# I'm not sure I have a perfect equivalent for the DH grid(s) at the moment,
# since they apparently do not include the South Pole. The Clenshaw-Curtis
# and Fejer quadrature rules are very similar (see the documentation in
...
...
py
sharp/
upsample_demo.py
→
py
thon/demos/sht_
upsample_demo.py
View file @
9846e31a
File moved
ducc.cc
→
python/
ducc.cc
View file @
9846e31a
...
...
@@ -13,19 +13,19 @@
#include "mr_util/healpix/healpix_base.cc"
#include <pybind11/pybind11.h>
#include "py
sharp/pysharp
.cc"
#include "py
pocketfft/pypocket
fft.cc"
#include "py
interpol_ng/pyinterpol_ng
.cc"
#include "
nifty_gridder/nifty_
gridder.cc"
#include "py
Healpix/pyH
ealpix.cc"
#include "py
thon/sht
.cc"
#include "py
thon/
fft.cc"
#include "py
thon/totalconvolve
.cc"
#include "
python/w
gridder.cc"
#include "py
thon/h
ealpix.cc"
using
namespace
mr
;
PYBIND11_MODULE
(
PKGNAME
,
m
)
{
add_
pypocket
fft
(
m
);
add_
pysharp
(
m
);
add_
pyinterpol_ng
(
m
);
add_
nifty_
gridder
(
m
);
add_
pyH
ealpix
(
m
);
add_fft
(
m
);
add_
sht
(
m
);
add_
totalconvolve
(
m
);
add_
w
gridder
(
m
);
add_
h
ealpix
(
m
);
}
py
pocketfft/pypocket
fft.cc
→
py
thon/
fft.cc
View file @
9846e31a
...
...
@@ -21,7 +21,7 @@
namespace
mr
{
namespace
detail_py
pocket
fft
{
namespace
detail_py
module_
fft
{
namespace
{
...
...
@@ -346,7 +346,7 @@ PyObject * good_size(PyObject * /*self*/, PyObject * args)
real
?
util1d
::
good_size_real
(
n
)
:
util1d
::
good_size_cmplx
(
n
));
}
const
char
*
pypocket
fft_DS
=
R"""(Fast Fourier and Hartley transforms.
const
char
*
fft_DS
=
R"""(Fast Fourier and Hartley transforms.
This module supports
- single, double, and long double precision
...
...
@@ -649,11 +649,11 @@ out : int
}
// unnamed namespace
void
add_
pypocket
fft
(
py
::
module
&
msup
)
void
add_fft
(
py
::
module
&
msup
)
{
using
namespace
pybind11
::
literals
;
auto
m
=
msup
.
def_submodule
(
"fft"
);
m
.
doc
()
=
pypocket
fft_DS
;
m
.
doc
()
=
fft_DS
;
m
.
def
(
"c2c"
,
c2c
,
c2c_DS
,
"a"
_a
,
"axes"
_a
=
None
,
"forward"
_a
=
true
,
"inorm"
_a
=
0
,
"out"
_a
=
None
,
"nthreads"
_a
=
1
);
m
.
def
(
"r2c"
,
r2c
,
r2c_DS
,
"a"
_a
,
"axes"
_a
=
None
,
"forward"
_a
=
true
,
...
...
@@ -678,6 +678,6 @@ void add_pypocketfft(py::module &msup)
}
using
detail_py
pocketfft
::
add_pypocket
fft
;
using
detail_py
module_fft
::
add_
fft
;
}
nifty_gridder
/gridder_cxx.h
→
python
/gridder_cxx.h
View file @
9846e31a
File moved
py
Healpix/pyH
ealpix.cc
→
py
thon/h
ealpix.cc
View file @
9846e31a
...
...
@@ -41,7 +41,7 @@
namespace
mr
{
namespace
detail_py
H
ealpix
{
namespace
detail_py
module_h
ealpix
{
using
namespace
std
;
using
namespace
mr
;
...
...
@@ -245,7 +245,7 @@ py::array local_v_angle (const py::array &v1, const py::array &v2)
return
move
(
angle
);
}
const
char
*
pyH
ealpix_DS
=
R"""(
const
char
*
h
ealpix_DS
=
R"""(
Python interface for some of the HEALPix C++ functionality
All angles are interpreted as radians.
...
...
@@ -345,11 +345,11 @@ that their last dimension is removed.
The employed algorithm is highly accurate, even for angles close to 0 or pi.
)"""
;
void
add_
pyH
ealpix
(
py
::
module
&
msup
)
void
add_
h
ealpix
(
py
::
module
&
msup
)
{
using
namespace
pybind11
::
literals
;
auto
m
=
msup
.
def_submodule
(
"healpix"
);
m
.
doc
()
=
pyH
ealpix_DS
;
m
.
doc
()
=
h
ealpix_DS
;
py
::
class_
<
Pyhpbase
>
(
m
,
"Healpix_Base"
,
py
::
module_local
())
.
def
(
py
::
init
<
int
,
const
string
&>
(),
"nside"
_a
,
"scheme"
_a
)
...
...
@@ -385,6 +385,6 @@ void add_pyHealpix(py::module &msup)
}
using
detail_py
Healpix
::
add_pyH
ealpix
;
using
detail_py
module_healpix
::
add_h
ealpix
;
}
py
sharp/pysharp
.cc
→
py
thon/sht
.cc
View file @
9846e31a
...
...
@@ -44,7 +44,7 @@
namespace
mr
{
namespace
detail_py
sharp
{
namespace
detail_py
module_sht
{
using
namespace
std
;
using
namespace
mr
;
...
...
@@ -188,7 +188,7 @@ template<typename T> class py_sharpjob
}
};
const
char
*
pysharp
_DS
=
R"""(
const
char
*
sht
_DS
=
R"""(
Python interface for some of the libsharp functionality
Error conditions are reported by raising exceptions.
...
...
@@ -268,11 +268,11 @@ py::array py_upsample_to_cc(const py::array &in, size_t nrings_out, bool has_np,
return
move
(
out
);
}
void
add_
pysharp
(
py
::
module
&
msup
)
void
add_
sht
(
py
::
module
&
msup
)
{
using
namespace
pybind11
::
literals
;
auto
m
=
msup
.
def_submodule
(
"sht"
);
m
.
doc
()
=
pysharp
_DS
;
m
.
doc
()
=
sht
_DS
;
py
::
class_
<
py_sharpjob
<
double
>>
(
m
,
"sharpjob_d"
,
py
::
module_local
())
.
def
(
py
::
init
<>
())
...
...
@@ -306,7 +306,7 @@ void add_pysharp(py::module &msup)
}
using
detail_py
sharp
::
add_pysharp
;
using
detail_py
module_sht
::
add_sht
;
}
py
pocketfft/test/test_pypocket
fft.py
→
py
thon/test/test_
fft.py
View file @
9846e31a
File moved
py
Healpix/test/test_pyH
ealpix.py
→
py
thon/test/test_h
ealpix.py
View file @
9846e31a
File moved
py
sharp/test/test_pysharp
.py
→
py
thon/test/test_sht
.py
View file @
9846e31a
File moved
py
interpol_ng/test/test_interpol_ng
.py
→
py
thon/test/test_totalconvolve
.py
View file @
9846e31a
File moved
nifty_gridder/test/test_
gridder.py
→
python/test/test_w
gridder.py
View file @
9846e31a
File moved
py
interpol_ng/pyinterpol_ng
.cc
→
py
thon/totalconvolve
.cc
View file @
9846e31a
...
...
@@ -5,11 +5,11 @@
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include "
interpol_ng
.h"
#include "
totalconvolve
.h"
namespace
mr
{
namespace
detail_py
interpol_ng
{
namespace
detail_py
module_totalconvolve
{
using
namespace
std
;
using
namespace
mr
;
...
...
@@ -94,7 +94,7 @@ template<typename T> py::array pyrotate_alm(const py::array &alm_, int64_t lmax,
}
#endif
constexpr
const
char
*
pyinterpol_ng
_DS
=
R"""(
constexpr
const
char
*
totalconvolve
_DS
=
R"""(
Python interface for total convolution/interpolation library
All arrays containing spherical harmonic coefficients are assumed to have the
...
...
@@ -234,12 +234,12 @@ Notes
- must be the last call to the object
)"""
;
void
add_
pyinterpol_ng
(
py
::
module
&
msup
)
void
add_
totalconvolve
(
py
::
module
&
msup
)
{
using
namespace
pybind11
::
literals
;
auto
m
=
msup
.
def_submodule
(
"totalconvolve"
);
m
.
doc
()
=
pyinterpol_ng
_DS
;
m
.
doc
()
=
totalconvolve
_DS
;
using
inter_d
=
PyInterpolator
<
double
>
;
py
::
class_
<
inter_d
>
(
m
,
"PyInterpolator"
,
py
::
module_local
(),
pyinterpolator_DS
)
...
...
@@ -272,6 +272,6 @@ void add_pyinterpol_ng(py::module &msup)
}
using
detail_py
interpol_ng
::
add_pyinterpol_ng
;
using
detail_py
module_totalconvolve
::
add_totalconvolve
;
}
py
interpol_ng/interpol_ng
.h
→
py
thon/totalconvolve
.h
View file @
9846e31a
File moved
nifty_gridder/nifty_
gridder.cc
→
python/w
gridder.cc
View file @
9846e31a
...
...
@@ -26,7 +26,7 @@
namespace
mr
{
namespace
detail_
nifty_
gridder
{
namespace
detail_
pymodule_w
gridder
{
using
namespace
std
;
using
namespace
gridder
;
...
...
@@ -190,7 +190,7 @@ np.array((nrows, nchan,), dtype=complex of same precision as `dirty`)
the measurement set data.
)"""
;
void
add_
nifty_
gridder
(
py
::
module
&
msup
)
void
add_
w
gridder
(
py
::
module
&
msup
)
{
using
namespace
pybind11
::
literals
;
auto
m
=
msup
.
def_submodule
(
"wgridder"
);
...
...
@@ -205,6 +205,6 @@ void add_nifty_gridder(py::module &msup)
}
using
detail_
nifty_gridder
::
add_nifty_
gridder
;
using
detail_
pymodule_wgridder
::
add_w
gridder
;
}
setup.py
View file @
9846e31a
...
...
@@ -22,7 +22,7 @@ def _get_files_by_suffix(directory, suffix):
return
list
(
itertools
.
chain
.
from_iterable
(
iterable_sources
))
include_dirs
=
[
'./src/'
,
include_dirs
=
[
'.
'
,
'.
/src/'
,
_deferred_pybind11_include
(
True
),
_deferred_pybind11_include
()]
extra_compile_args
=
[
'--std=c++17'
,
'-march=native'
,
'-ffast-math'
,
'-O3'
]
...
...
@@ -47,7 +47,7 @@ def get_extension_modules():
depfiles
=
_get_files_by_suffix
(
'.'
,
'h'
)
+
_get_files_by_suffix
(
'.'
,
'cc'
)
+
[
'setup.py'
]
return
[
Extension
(
pkgname
,
language
=
'c++'
,
sources
=
[
'ducc.cc'
],
sources
=
[
'
python/
ducc.cc'
],
depends
=
depfiles
,
include_dirs
=
include_dirs
,
define_macros
=
define_macros
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment