Refactored processing

Merged processing and reprocessing code, and extending to handle deletion of files. Adding endpoints for putting and removing files, and updating metadata when in staging. Extending functionality of raw path get endpoint, by 1) adding support for seeking within a file using offset and length, 2) mime types, 3) on-the-fly decompression and 4) folder size info (as requested in issue #545 (closed)).