mpikat merge requestshttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests2024-03-07T07:56:59Zhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/244Pulsar pipeline configurable outputfiles + cal data from data store2024-03-07T07:56:59ZTobias WinchenPulsar pipeline configurable outputfiles + cal data from data store* Outptufile locations are configurable with place-holders such as `{obs_id)/{project_id}` to fit needs of different deployments
* Cal data is taken from data store automatically
* Created outptufiles are reported back on measurement pre...* Outptufile locations are configurable with place-holders such as `{obs_id)/{project_id}` to fit needs of different deployments
* Cal data is taken from data store automatically
* Created outptufiles are reported back on measurement preparehttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/243refactoring dig pack controller and alveo pfb controller from katcp, tornado ...2024-02-27T20:07:16ZJason Wurefactoring dig pack controller and alveo pfb controller from katcp, tornado to aiokatcp, asynciohttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/242Provision tests2024-02-23T16:03:25ZNiclas EsserProvision testsAdded the capability to parse MasterController responses in provision testsAdded the capability to parse MasterController responses in provision testshttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/241Add mechanism to obtain list of outputfiles on measurement stop and create a ...2024-02-22T11:11:57ZTobias WinchenAdd mechanism to obtain list of outputfiles on measurement stop and create a unique run idhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/240New release to production 240220.02024-02-20T14:30:59ZNiclas EsserNew release to production 240220.0- Bug fixes and refactoring of data plotting.
- Unify gated_spectrometer utils into a module- Bug fixes and refactoring of data plotting.
- Unify gated_spectrometer utils into a moduleNiclas EsserNiclas Esserhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/239Refactor plotter2024-02-19T16:50:12ZNiclas EsserRefactor plotterRefactored plotter for HDF5_Writer and FitsInterface
- Refactored design -> use ProcessIO
- Created ProcessIO -> a multiprocessing.Process with an in- and output Queue
- New Plotter
- GatedSpectrometer-utils (Spead handling, data stream ...Refactored plotter for HDF5_Writer and FitsInterface
- Refactored design -> use ProcessIO
- Created ProcessIO -> a multiprocessing.Process with an in- and output Queue
- New Plotter
- GatedSpectrometer-utils (Spead handling, data stream format, plotter etc.) unified and moved to mpikat/pipelines/gated_spectrometerNiclas EsserNiclas Esserhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/238Moved FitsWriter from KATCP to aiokatcp; Generalized spectrum plotter for...2024-02-05T13:16:01ZNiclas EsserMoved FitsWriter from KATCP to aiokatcp; Generalized spectrum plotter for...1. Refactor from katcp -> aiokatcp / tornado -> asyncio [2e483412](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/2e483412622e9880b9b3070f960937712b3d3529)
- FitsInterface
- HDF5Writer /HDF5FileWriter
- MockFitsWriter
- SensorWatc...1. Refactor from katcp -> aiokatcp / tornado -> asyncio [2e483412](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/2e483412622e9880b9b3070f960937712b3d3529)
- FitsInterface
- HDF5Writer /HDF5FileWriter
- MockFitsWriter
- SensorWatchdog
2. Generalized plotting for GS [228900e3](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/228900e3bdb1db68f76f0e1bd9253c030b5106b9)
- The same bandpass plotter is used in FitsInterface and HDF5Writer
- No plotting config is required anymore (removed from the DEFAULT_CONFIG)
3. New real-time plotting feature using Redis (GSPlotter-Pipeline) [b7ec40a0](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/b7ec40a0cf3752d97f6b3fc6539dedf5cdab3a00)
4. New fits sub-module containing receiver, sending, stream and packet description [04022e10](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/04022e10a00009f47527e84b093df81cc8588bed)
5. Gated Spectrometer - Error state on 100% buffer fill level [16068311](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/16068311f0ee7d572c7816cb3bb6971f9a283d37)
6. SKAMPI Telescope Status Server - Try reconnecting to katportal after broken pipe [2f20dcd6](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/2f20dcd68f7b85dd30c972ce09fd8d44b406bcaa)
7. Dropped Support for SKARAB Client and firmware [bc492e2e](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/bc492e2e3a4764da423c472817cbe6307de3108c)
8. Improved exception handling for asyncio coroutines [90b0eb57](https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/commit/90b0eb576b17684ecd78117fedc5ba9cfa342ab5)
9. Testing
- Tests for the fits-module (new/improved)
- Tests for Redis JSON sender (new)
- Tests for HDF5Writer (improved)
10. Minor bug fixes, Pylint, resolve deprecation warning
Generalized spectrum plotter for fitsinterface and hdf5writer; Added FitsWriter to state tests; Added missing dependencies to pyproject.tomlNiclas EsserNiclas Esserhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/237Issue 412024-02-02T14:20:13ZNiclas EsserIssue 41Fixes issue #41, potentially #40 and timeouts of the HDF5WriterFixes issue #41, potentially #40 and timeouts of the HDF5Writerhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/236Realtime plotter2024-02-01T12:35:38ZNiclas EsserRealtime plotterNiclas EsserNiclas Esserhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/235Hdf5 writer fix2023-11-29T15:00:48ZNiclas EsserHdf5 writer fix- Resolved bug in file creation -> on measurement-prepare, a katcp sensor tried to access the filesize, which did not exist. An exception was thrown.
- Resolved plotting bug -> The plotter aborted because of bad code revision when moving...- Resolved bug in file creation -> on measurement-prepare, a katcp sensor tried to access the filesize, which did not exist. An exception was thrown.
- Resolved plotting bug -> The plotter aborted because of bad code revision when moving from tornado to asynciohttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/234Moved FitsWriter from katcp to aiokatcp; Generalized spectrum plotter for...2023-11-16T11:53:43ZNiclas EsserMoved FitsWriter from katcp to aiokatcp; Generalized spectrum plotter for...Fits-module:
- receiver, sender, formats
- modularized folder
Asyncio/aiokatcp:
- HDF5Writer
- FitsInterface
- MockFitsWriter
Dependencies (pyproject.toml)Fits-module:
- receiver, sender, formats
- modularized folder
Asyncio/aiokatcp:
- HDF5Writer
- FitsInterface
- MockFitsWriter
Dependencies (pyproject.toml)https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/233Dev to production2023-10-13T21:00:04ZJason WuDev to production- Fix Alveo search mode band flip issue
- Fix core sorting to int-based
- Fix zapping first integration for pulsar scrunched archive- Fix Alveo search mode band flip issue
- Fix core sorting to int-based
- Fix zapping first integration for pulsar scrunched archivehttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/232Pulsar feature 202309122023-09-28T11:39:21ZJason WuPulsar feature 20230912Alveo PFB filterbank fix and timing archive plotting feature.Alveo PFB filterbank fix and timing archive plotting feature.https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/231Dev to production2023-08-21T11:05:26ZJason WuDev to productionUpdate for pulsar pipeline:
- pfb merge code name update
- update plotting for pulsar diagnostic plotUpdate for pulsar pipeline:
- pfb merge code name update
- update plotting for pulsar diagnostic plothttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/230bug fix pulsar plotting2023-08-21T09:23:27ZJason Wubug fix pulsar plottinghttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/228Pulsar dual pol plotting2023-08-16T07:39:25ZJason WuPulsar dual pol plottingupdate TScrunch plot for pulsar processing, now showing both polarisation to ensure Alveo card is running fine.update TScrunch plot for pulsar processing, now showing both polarisation to ensure Alveo card is running fine.https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/227dev to production2023-08-08T21:03:02ZJason Wudev to productionReplacing "edd_roach_merge_leap" with "edd_pfb_merge" in pulsar_processing, refracting the changes in psrdada_cpp.Replacing "edd_roach_merge_leap" with "edd_pfb_merge" in pulsar_processing, refracting the changes in psrdada_cpp.https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/226New release to production2023-07-25T07:53:58ZNiclas EsserNew release to production- Updates in pulsar pipeline
1. increase logging for running command.
2. Including SKARABPFB for data format.
3. Making #core for mkrecv configurable.
4. folder_size_monitor to async
- Use default logger for stdout
- Extension ...- Updates in pulsar pipeline
1. increase logging for running command.
2. Including SKARABPFB for data format.
3. Making #core for mkrecv configurable.
4. folder_size_monitor to async
- Use default logger for stdout
- Extension of process_tool -> ManagedProcessPoolhttps://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/225Pulsar pipeline update.2023-07-13T09:35:49ZJason WuPulsar pipeline update.Increase logging for running mkrecv command.
Fixing instrument name in dspsr header.
Making #core for mkrecv configurable.
Making folder size monitor async.Increase logging for running mkrecv command.
Fixing instrument name in dspsr header.
Making #core for mkrecv configurable.
Making folder size monitor async.https://gitlab.mpcdf.mpg.de/mpifr-bdg/mpikat/-/merge_requests/224Dev to production2023-06-22T10:13:29ZJason WuDev to production-Patched pulsar_processing dada header MODE bug.
-Pulsar_processing PFB baseband recording buffer size automatic calculation.
-Update mpikat/utils/dada_tools/reader.py
-Alveo stream definition update.-Patched pulsar_processing dada header MODE bug.
-Pulsar_processing PFB baseband recording buffer size automatic calculation.
-Update mpikat/utils/dada_tools/reader.py
-Alveo stream definition update.