ducc issueshttps://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues2022-08-10T18:18:34Zhttps://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/1Windows does not support symlinks created by Git2022-08-10T18:18:34ZMaurizio TomasiWindows does not support symlinks created by GitGit for Windows does not support symlinks, which replaces with text files containing the path to the linked entry. This makes the compilation of pyHealpix fail since commit 34138251713d85a7b32a96e975b52be75a3411e6 ("Improve Python packag...Git for Windows does not support symlinks, which replaces with text files containing the path to the linked entry. This makes the compilation of pyHealpix fail since commit 34138251713d85a7b32a96e975b52be75a3411e6 ("Improve Python packaging").
Since Windows *does* support symlinks, it seems possible to circumvent this limitation:
https://stackoverflow.com/questions/5917249/git-symlinks-in-windows
However, I have yet to test this feature.https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/2Remove temporary features2020-06-05T13:39:43ZMartin ReineckeRemove temporary featuresSome of the functionality exposed by the Python modules is supposed to be only temporary, mostly because it needs optimization or should move to a different package, but it's currently exposed to allow unit testing. This needs to go away...Some of the functionality exposed by the Python modules is supposed to be only temporary, mostly because it needs optimization or should move to a different package, but it's currently exposed to allow unit testing. This needs to go away again at some point.
- [ ] upsampling in `pysharp` (don't forget to remove file dependencies!)
- [ ] a_lm rotation in `interpol_ng`Martin ReineckeMartin Reineckehttps://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/4Finalize names2023-01-12T14:50:10ZMartin ReineckeFinalize namesNow that we seem to reach a conclusion about naming (ducc0, ducc1, etc.), the repository needs to be adjusted:
- [x] branch `ducc_0_1` renamed to `ducc0`; `ducc0` becomes the default branch
- [x] directory `mr_util` renamed to `ducc0`;...Now that we seem to reach a conclusion about naming (ducc0, ducc1, etc.), the repository needs to be adjusted:
- [x] branch `ducc_0_1` renamed to `ducc0`; `ducc0` becomes the default branch
- [x] directory `mr_util` renamed to `ducc0`; include statements must be adjusted
- [x] macro names need to be changed from MRUTIL to DUCC0
- [x] namespace `mr` becomes `ducc0`
- [ ] probably more...https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/5Checklist for first release2020-09-06T09:28:43ZMartin ReineckeChecklist for first releaseI think we are pretty close to a first (experimental) release. Let's stick to `ducc0` for this one, to indicate that there may be more changes than could be expected from a "version 1" release.
The interfaces for the individual submodul...I think we are pretty close to a first (experimental) release. Let's stick to `ducc0` for this one, to indicate that there may be more changes than could be expected from a "version 1" release.
The interfaces for the individual submodules are OK, I think. In the longer run I have a few ideas how to improve the `sht` interface in particular, but this needs some time to mature. Landman's proposed enhancements for `wgridder` probably also won't be implemented very soon.
@parras, @g-simonperkins, @g-landmanbester, is anything missing from your point of view? If you have suggestions, e.g. for the docstrings, I'll be happy to implement them!https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/6[Feature request] Take maximum k-length of sampling points into account for s...2023-01-25T09:43:49ZPhilipp Arrasparras@mpa-garching.mpg.de[Feature request] Take maximum k-length of sampling points into account for saving some 1D Fourier transformationsIn order to achieve what radio astronomers call "super resolution", they choose the resolution of space to be finer than the Nyquist frequency associated with the longest baseline, i.e. the sample in Fourier space which is the farthest a...In order to achieve what radio astronomers call "super resolution", they choose the resolution of space to be finer than the Nyquist frequency associated with the longest baseline, i.e. the sample in Fourier space which is the farthest away from the zero mode. By taking this into account, a considerable fraction of the 1D Fourier transformations could be saved.https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/7Implement more "apply" variants for mav/fmav classes2021-11-22T10:29:20ZMartin ReineckeImplement more "apply" variants for mav/fmav classes`mav` and `fmav` should have full support for unary, binary and ternary element-wise operations, with variants that
- leave all arguments unchanged and produce no output array (typically reductions)
- leave arguments unchanged and create...`mav` and `fmav` should have full support for unary, binary and ternary element-wise operations, with variants that
- leave all arguments unchanged and produce no output array (typically reductions)
- leave arguments unchanged and create a new array (perhaps taking an extra template argument to specify the element type of the new array) (+, -, * etc.)
- change the leading argument (*=, +=, etc.).
Special shortcuts could be used if
- all arrays are contiguous with equal strides (just use a single loop over all elements)
- all arrays have equal strides (only calculate the offset once)
If possible, the implementation should process the smallest strides in the innermost loop.https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/8Gridder: add automatic selection of index type2020-08-27T11:44:53ZMartin ReineckeGridder: add automatic selection of index typeCurrently, we use `uint32_t` to store the indices of visibilities we want to grid/degrid. For very large measurement sets, this may not be enough (everything under 1 billion visibilities should be safe though).
It's possible to make `Ti...Currently, we use `uint32_t` to store the indices of visibilities we want to grid/degrid. For very large measurement sets, this may not be enough (everything under 1 billion visibilities should be safe though).
It's possible to make `Tidx` a template parameter and call code with `Tidx=uint64_t` for really large MS without any user interaction. This increases compiled code size a bit, but that should not be a problem.https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/9Gridder: add special kernel evaluation functions for 2D kernels2023-02-13T08:02:23ZMartin ReineckeGridder: add special kernel evaluation functions for 2D kernelsWhen gridding a single visibility, we currently call the kernel's `eval` method twice (and `eval_single` once if W-gridding if active). By interleaving these operations in a single call, it's probably possible to save quite some time.When gridding a single visibility, we currently call the kernel's `eval` method twice (and `eval_single` once if W-gridding if active). By interleaving these operations in a single call, it's probably possible to save quite some time.https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/10Let the gridder choose appropriate nu and nv on its own2020-09-17T05:58:19ZMartin ReineckeLet the gridder choose appropriate nu and nv on its ownThe `wgridder` interface currently contains the dimensions `nu` and `nv` of the Fourier space grid, although they are just an implementation detail and are not used in any data products passed to or returned from the gridder. This has th...The `wgridder` interface currently contains the dimensions `nu` and `nv` of the Fourier space grid, although they are just an implementation detail and are not used in any data products passed to or returned from the gridder. This has the potential for user errors:
- the user specifies a combination of `epsilon`, `nu` and `nv`, for which no suitable kernel can be found. Increasing `nu` and `nv` would make the problem solvable.
- the user specifies large `epsilon` and high `nu/nv`; this usually leads to a gridding task which spends more time than necessary on FFTs.
In principle the gridder has all necessary information to determine a good oversampling factor from the user-provided data. It won't be easy to compute the absolute best possible choice, but we can probably get close (and may be better than user intuition).
@parras, @g-landmanbester, @g-simonperkins what do you think?https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/11Mask feature of wgridder is not tested2020-08-27T11:42:39ZPhilipp Arrasparras@mpa-garching.mpg.deMask feature of wgridder is not testedPhilipp Arrasparras@mpa-garching.mpg.dePhilipp Arrasparras@mpa-garching.mpg.dehttps://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/12[ducc1] Rename `do_wstacking` to `do_wgridding` in ducc.wgridder2023-03-16T11:16:02ZPhilipp Arrasparras@mpa-garching.mpg.de[ducc1] Rename `do_wstacking` to `do_wgridding` in ducc.wgridderhttps://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/13Publish ducc on the PyPI2021-01-18T13:27:58ZMaurizio TomasiPublish ducc on the PyPIThe Python Package Index (PyPI) is the default repository queried by `pip` whenever a new Python package is to be installed.
Because of security reasons explained [here](https://github.com/pypa/pip/issues/6301), it is currently not poss...The Python Package Index (PyPI) is the default repository queried by `pip` whenever a new Python package is to be installed.
Because of security reasons explained [here](https://github.com/pypa/pip/issues/6301), it is currently not possible to register Python packages in the PyPI if they list git repositories among their dependencies. Therefore, no package that depends on `ducc0` can be registered in the PyPI, unless `ducc` is registered there too.
I realize that registering `ducc` on the PyPI is going to put some burden on the developers, as publishing a package requires some effort to carefully track version numbers and ensure that the rules of semantic versioning be satisfied for every release.
However, the more `ducc` is being used by the scientific community, the more likely this request will come up regularly. Would the developers consider this request?
(I stumbled on this problem while trying to publish the LiteBIRD Simulation Framework, which lists `ducc` as a [direct dependency](https://github.com/litebird/litebird_sim/blob/2f2f2cb36bbf02eaf5629b6295e9a47684c16905/pyproject.toml#L39).)https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/14Add support for complex-valued images2021-10-27T08:28:40ZLandman BesterAdd support for complex-valued imagesI know we are not quite there yet but I wanted to bring up the possibility of gridding complex-valued images. This is required for the off-diagonal terms when doing full polarisation imaging. Are you planning to support that at any stage?I know we are not quite there yet but I wanted to bring up the possibility of gridding complex-valued images. This is required for the off-diagonal terms when doing full polarisation imaging. Are you planning to support that at any stage?https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/15Unable to use Healpix function `ang2pix` in Python with 32-bit floating-point...2023-06-26T11:41:03ZMaurizio TomasiUnable to use Healpix function `ang2pix` in Python with 32-bit floating-point typeThe function `ducc0.healpix.Healpix_Base.ang2pix` seems to accepts only 64-bit floating point numbers:
```python
import numpy as np
import ducc0.healpix
resol = ducc0.healpix.Healpix_Base(256, "RING")
# Generate random pointings
point...The function `ducc0.healpix.Healpix_Base.ang2pix` seems to accepts only 64-bit floating point numbers:
```python
import numpy as np
import ducc0.healpix
resol = ducc0.healpix.Healpix_Base(256, "RING")
# Generate random pointings
pointings = np.random.rand(100, 3)
# This works
pixidx = resol.ang2pix(pointings[:, 0:2])
# Convert the pointings to 32-bit numbers
pointings = np.array(pointings, dtype="float32")
# This does *not* work
pixidx = resol.ang2pix(pointings[:, 0:2])
```
The error is the following:
```
RuntimeError Traceback (most recent call last)
Input In [13], in <module>
----> 1 resol.ang2pix(pointings[:, 0:2])
RuntimeError:
./src/ducc0/bindings/pybind_utils.h: 51 (pybind11::array_t<T> ducc0::detail_pybind::toPyarr(const pybind11::object&) [with T = double]):
Assertion failure
error during array conversion
```
Is there the possibility to pass 32-bit arrays to `ang2pix`?https://gitlab.mpcdf.mpg.de/mtr/ducc/-/issues/16Can't install from source using pip on python3.8 ubuntu20.042023-07-13T05:59:59ZLandman BesterCan't install from source using pip on python3.8 ubuntu20.04I am getting the following error when trying to build locally in a python3.8 virtual environment
```
(pfb) ╭─bester@oates ~/software
╰─➤ pip install -e ducc/
Obtaining file:///home/bester/software/ducc
Installing build dependencies .....I am getting the following error when trying to build locally in a python3.8 virtual environment
```
(pfb) ╭─bester@oates ~/software
╰─➤ pip install -e ducc/
Obtaining file:///home/bester/software/ducc
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... error
error: subprocess-exited-with-error
× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> [37 lines of output]
Build environment:
Platform: Linux-5.18.10-051810-generic-x86_64-with-glibc2.27
Machine: x86_64
System: Linux
Architecture: ('64bit', 'ELF')
Traceback (most recent call last):
File "/home/bester/.venv/pfb/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/bester/.venv/pfb/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/bester/.venv/pfb/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
return hook(config_settings)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 450, in get_requires_for_build_editable
return self.get_requires_for_build_wheel(config_settings)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 118, in <module>
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 159, in setup
dist.parse_config_files()
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 898, in parse_config_files
pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/config/pyprojecttoml.py", line 66, in apply_configuration
config = read_configuration(filepath, True, ignore_option_errors, dist)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/config/pyprojecttoml.py", line 128, in read_configuration
validate(subset, filepath)
File "/tmp/pip-build-env-dn94kqj7/overlay/lib/python3.8/site-packages/setuptools/config/pyprojecttoml.py", line 55, in validate
raise ValueError(f"{error}\n{summary}") from None
ValueError: invalid pyproject.toml config: `project`.
configuration error: `project` must contain ['name'] properties
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
```
This definitely wasn't a problem before. Looks like something going wrong in the pyproject.toml file which I know almost nothing about. Any ideas on how to get this working?