Major refactor of the proc framework. The join functionality is lifted up to the base class and generalized. By marking a process with is_child = True, you tell the framework that we should try to join with (resume) the parent Proc (obtained by calling self.parent on the child Proc) when he child process is done. When the join succeeds (when all children are done), the parent is put to RUNNING (previously it remained in status WAITING_FOR_RESULT) and the new join method is called. The Proc may from there go back to status WAITING_FOR_RESULT again (so that we can implement the idea of "slotted" parsing, see #668 (closed)).
Further, queuing is made possible (#667 (closed)). I.e. when a process is running, it may be possible to invoke another process on the same object. The framework should ensure that the processes are queued up and executed one at a time in sequence. By setting is_blocking = True, you can ensure that certain processes "block" additional queuing until they are done, as it does not make sense/seem safe to queue up calls in every case. I have marked delete_upload, publish_upload, and publish_externally as "blocking". The queue is persisted to the mongodb object. Measures are taken to ensure that scheduling, starting and completing a process are atomic operations.