Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
C
ctapipe_io_magic
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 3
    • Issues 3
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • Ievgen Vovk
  • ctapipe_io_magic
  • Issues
  • #12

Closed
Open
Opened Dec 30, 2020 by Alessio Berti@g-alessiobertiDeveloper

Finding mono and stereo events take too much time when loading events

When performing a profiling of ctapipe_io_magic (as of commit c676ef1c) taking in one run (composed of multiple subruns), I found out that >50% of the time is spent in finding the indices of mono and stereo events (_find_mono_events and _find_stereo_events methods of the MarsRun class), see attached screenshot (produced with snakeviz and the profiling file attached).

From inspection of the two methods code, the bottleneck is due to the several for loops, which are looping over a consistent number of events. In this sense, there should be a huge boost if those loops are replaced by numpy functions, since the starting data structures are numpy arrays.

The script I tested (called test_ctapipe_io_magic.py) is very simple:

from ctapipe_io_magic import MarsRun, MAGICEventSource

onerun = MarsRun("/storage/gpfs_data/ctalocal/aberti/MAGIC_LST/Analysis/Crab_campaign/CrabNebula/2020-01-19/Calibrated_ON/20200119*05088541*.root")

and to create the profile file I run the script with:

python -m cProfile -o profile.prof test_ctapipe_io_magic.py

test_ctapipe_io_magic.prof

ctapipe_io_magic_profiling_c676ef1cd40a5c7c8b41848b4d9fa90225b2f34c

Edited Dec 30, 2020 by Alessio Berti
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: ievo/ctapipe_io_magic#12