Caching implementation does not work as intended
check_path_and_make_folders goes into an infinite loop. I suspect this is because os.path.join(os.path.dirname(file), 'cache') does not produce a string and the *path.rpartition('/') therefore does not work as intended. The string seperation will also not work on other operating systems using a different path seperator.
I suggest three changes, in decreasing order of importance:
- Use os library to handle the traversing of levels in check_path_and_make_folders.
- Move the construction of the cache filename (from data_folder, save_path and formatted filename) into a seperate function such that there is no duplicate code in extract_heatflux_profile_from_cache and cache_heatflux_by_program.
- Move options such as cache_folder into a user_settings file each user can modify and load from there. (4. Make it possible to load already downloaded, longer time window cache files and overwrite shorter time window cache files. Either build by string comparison or include lightweight database.)