Update to V3.4.1:\n downloadversionIRdata: improved documentation, adding of...

Update to V3.4.1:\n downloadversionIRdata: improved documentation, adding of request for all ports, change of request paths, files; improved warning handling with repect to issue #14\n changelog: updated to V3.4.1\n setup.py: updated requierements to w7xarchive instead of archivedb\n plot_heatflux_example: added calling of the heatflux download to agree with the code.\n IR_iamge_tools: documentation update

CHANGELOG

 <<<<<<< HEAD
+XX.06.2020: Update to V3.5.0	
+	- add download maximum heat flux and wetted area
+16.06.2020: Update to V3.4.1
+	- updated and improved documentation
+	- include chache function and a request for all ports for heat flux and divertor load
 08.02.2020: Update to V3.4.0
 	- Bugfix: download_images_by_times produzierte doppelte Werte im Zeit und Bildvektor
-	- 
+	- Bugfix: download divertor load gave print back that request was made, although no request was made (fixed)
+	- Bugfix: correction of version bug from last commit and change of default value for the profile extraction functions to None
+	- Readme added
 	Update to V3.3.3:
-	- 
+	- added function for peaking factor and strike line width calculation in tools
+	- bugfix: bug fix get_trigger_from_PID which caused an error in get_latest_version if it was used for non IR data
 	Update to V3.3.2:
-	-
+	- extended feedback for download_heatflux
+	- changes of wetted area calculation
+	- changes of wetted area calculation
+	- bugfix: get_trigger_PID, get_finger and during import (see issues 6,7)
 	Update to V3.3.1:
-	-
+	- bugfix: FOV for AEF50 was not working correctly
 	Update to V3.3.0:
-	- 
+	- IR_config_constants: added path to Test archive and use this variable in downloadversionIRdata when building an archive path
+	- downloadversionIRdata: consistent use of testmode parameter
+	- downloadversionIRdata: added testmode to all reading routines allowing reading from Test archive
+	- IR_image_tools: no longer sets working directory to script file location
+	- IR_config_constants: sets parameter_file_path as absolute path
+	- downloadversionIRdata: new function get_trigger_from_PID() wraps get_program_from_PID() and allows getting timestamps for lab data
+	- downloadversionIRdata: download_images_by_time_via_png() now uses nanosecond timestamps to call AKF_2.get_time_intervals()
+	- downloadversionIRdata: disable DeprecationWarning in import of archivedb (timezone issue on Windows)
+	- downloadversionIRdata: get_NUC_by_times() uses the reference cold frame in case no cold frame (NUC frame) was found for this time
+	- downloadversionIRdata: unified use of stoptime instead of endtime
+	- downloadversionIRdata: expanded testing sections (profile, coldframe, timestamps and scene model)
 	Update to V3.2.5:
-	
+	- remove of old, not needed function and dependencies, update of download_background function
 	Update to V3.2.4:
-	
+	- smaller bugfixes and setup.py got added
 	Update to V3.2.3:
 	- bugfix: finding right version for meta informations
 	Update to V3.2.2:
@@ -84,7 +105,8 @@ bugfixes in downloadversionIRdata:
     - fixed: wrong time intervall for TC divertor data
 
 Versions:
-V3.4.0: chaching functionality included, request for all cameras included + Bugfixes
+V3.5.0: download of wetted area, peak loads for different targets and 
+V3.4.1: chaching functionality included, request for all cameras included + Bugfixes
 V3.3.0: code-cleaning, unification of variable names, new functions: get_trigger_from_PID, 
 V3.2.0: download of scene models possible, temperature profiles can be extracted, downsampled temperature images available
 V3.1.0: add of divertor loads in the upload and the download

IR_image_tools.py

@@ -25,14 +25,16 @@ def get_OP_by_time(time_ns=None, shot_no=None, program_str=None):
     '''Derives operation phase (OP) of W7-X based on either:
        a nanosacond time stamp, a MDSplus style shot no. or a program ID.
        
-       IN:
-          time_ns      - integer of nanosecond time stamp,
+       INPUT
+       -----
+          time_ns:      - integer of nanosecond time stamp,
                          e.g. 1511972727249834301 (OPTIONAL)
           shot_no      - integer of MDSplus style shot number,
                          e.g. 171207022 (OPTIONAL)
           program_str  - string of CoDaQ ArchiveDB style prgram number or date,
                          e.g. '20171207.022' or '20171207' (OPTIONAL)
-       RETURN:
+       RESULT
+       ------
           conn         - MDSplus connection object, to be used in e.g. 1511972727249834301
                          read_MDSplus_image_simple(), read_MDSplus_metadata()
    '''
@@ -639,7 +641,13 @@ def reconstruct_coldframe (exposuretime, sT, a, bnew, coldref):
     """
     INPUT
     ------
-    
+        exposuretime: integer
+            the exposure time
+        sT: 
+        a:
+        bnew:
+        coldref: numpy array
+            the reference cold frame as the base for the reconstruction        
     RESULT
     ------
     
@@ -689,17 +697,27 @@ def get_work_list(pipepath,typ="q"):
     """
     INPUT
     ------
-    
+        pipepath: string
+            the path to the folder where the files are located
+        typ: string
+            the typ of data which is requested in the working list\n 
+            possiblities: q, Aw, qpeak, width, load\n 
+            or anything else for the problematic programs
     RESULT
     ------
-    
+        cam_programs: list
+            a list containing two coloumns, cameras and programs
+        reasons: list, optional, only for problematic programs
+            a list showing the reasons, why data are not processed
     NOTE
     ------
     """
     today=datetime.datetime.now()    
     cam_programs=[]
-    if typ in ('q','load'):
+    if typ in ('q_old','load_old'):
         f=open(pipepath+str(today.year)+str(today.month)+"_"+typ+"_requests.txt")
+    elif typ in ('q','load','qpeak','Aw','width'):
+        f=open(pipepath+"Auto_"+typ+"_requests.txt")
     else:
         reasons=[]
         f = open(pipepath+"problematic_programs.txt")

ToDO.txt

@@ -6,3 +6,6 @@ download:
 - implement download of the stored temperature data (After the upload)
 - implement download of the stored heat flux data --> done in V3.0.0
 - implement download of FLIR data --> Done in V2.8.0, in testing phase
+- implement caching
+- implement download of wetted area, peak heat flux and average strike line width
+- implement request of all ports

downloadversionIRdata.py

@@ -4,7 +4,7 @@ Created on Wed Oct 25 15:51:46 2017
 updated on Tue Aug 21 10:20:00 2018
 last update on Fr Nov 23 15:37:00 2018
 
-Version: 3.4.0
+Version: 3.4.1
 (Numbering: #of big changes(OP1.2a download V1, OP1.2b download V2, heatflux V3)
             .
             #of updates to add functionalities
@@ -12,7 +12,7 @@ Version: 3.4.0
             #number of updates for bug fixes )
 @author: holn
 """
-version="V3.4.0"
+version = "V3.4.1"
 
 import numpy as np
 import IR_image_tools as IR_tools
@@ -28,7 +28,7 @@ from IR_config_constants import archivepath, testarchivepath, portcamdict, camli
 import datetime
 import urllib.request
 import urllib.error
-
+import logging
 import json
 from PIL import Image
 from io import BytesIO
@@ -36,7 +36,7 @@ from os.path import join
 import matplotlib.pyplot as plt
 try:
     import threading
-    import archivedb as AKF_2
+    import w7xarchive as AKF_2
     # disbale deprecation warnings, because of timezone warning spam
     import warnings
     warnings.filterwarnings("ignore", category=DeprecationWarning)
@@ -81,12 +81,12 @@ def get_latest_version(stream, project="W7X", testmode=False, t_from=None,
              return int of version number of None if non-versioned or non-existing stream
      author: G. Schlisio, edit by holn
      """
-    if testmode:
+    if testmode:#check which archive should be used
         base = testarchivepath
     else:
         base = archivepath
 
-    if program == None and t_from == None and t_to == None:
+    if program == None and t_from == None and t_to == None:# check wheather any time information was given
         request = urllib.request.Request(base +project+"/" + stream + "/_versions.json", headers={"Accept": "application/json"})
     else:
         if program is not None:
@@ -192,42 +192,56 @@ def download_LUT(port, time, exposure=0, emissivity=0, camera_filter=0, version=
                  testmode=False, verbose=0):
     """
     download_LUT(camera, port, time, exposure=0, emissivity=0, camera_filter=0, version=1):
-        time in ns
+        download of the look up table for the infrared cameras from the database for OP1.2(a+b)
     Have to swap 11, 21 until correction in the database
     INPUT
     ------
-        port:
-        time:
-        exposure:
-        emissivity:
-        camera_filter:
-        version:
-        testmode:
-        verbose:
-    
+        port: interger
+            number of the AEF camera port
+        time: integer
+            nanosecond timestamp for the LUT            
+        exposure: integer
+            requested exposure time in microseconds
+        emissivity: float, string or integer
+            emissivity value to identify which LUT is requested
+        camera_filter: inter or string, only needed for port 50
+            filter number of the Infratec camera in port AEF50
+        version: integer, optional, default 0
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional, default False
+            if True, the data will be loaded from the test archive
+        verbose: integer, optional, default 0
+            feedback level (details of print messages)     
     RESULT
     ------
-    
+        exist: boolean
+            True if Look up table was found, False if not
+        LUT: list
+            contains the calibration information: [digital level, temperature, temperature error]
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     """
-    OP = IR_tools.get_OP_by_time(time_ns=time)
-    if port == 21:
+    OP = IR_tools.get_OP_by_time(time_ns=time)#getting operation phase
+    if port == 21:# swapping both ports (21 and 11)
         port = 11
     elif port == 11:
         port = 21
-    if testmode:
+    if testmode: #check which archive should be used
         larchivepath = testarchivepath+project+"/"+portpathdict[OP]["AEF"+str(port)]+"LUT_"
     else:
         larchivepath = archivepath+project+"/"+portpathdict[OP]["AEF"+str(port)]+"LUT_"
+    #camera dependend request string generation
     if port == 50 and OP == "OP1.2a":# camera=="INFRATEC" or camera=="infratec" or camera=="Infratec":
         query = "Filter_"+str(camera_filter)+"_Texp_"+str(int(exposure))+"us_e_"+str(float(emissivity))
     elif port in [10, 11, 20, 21, 30, 31, 40, 41, 51] or OP == "OP1.2b":#camera=="IRCAM" or camera=="IRcam" or camera=="ircam":
         query = "Texp_"+str(int(exposure))+"us_e_"+str(float(emissivity))
     else:
-        print("download_LUT: Error! Camera unknown, stopping here.")
-        raise Exception
-    if version == 0:
+#        print("download_LUT: Error! Camera unknown, stopping here.")
+        logging.warning("download_LUT: Error! Camera unknown, stopping here.")
+        return False, 0
+#        raise Exception
+    if version == 0:#version check, if 0
         version = get_latest_version(portpathdict[OP]["AEF"+str(port)]+"LUT_DATASTREAM", t_from=time, testmode=testmode)
     if verbose > 0:
         print("download_LUT: LUT V"+str(version)+" is used")
@@ -259,12 +273,27 @@ def read_LUT_from_file(port, this_time, t_exp, emissivity, cfilter, verbose=0):#
     Read LUT V3 from local files for testing new calibration.
     INPUT
     ------
-    
+        port: integer or string
+            the port number of the AEF port        
+        this_time: integer
+            nanosecond timestamp for the LUT            
+        t_exp: integer
+            requested exposure time in microseconds
+        emissivity: float, string or integer
+            emissivity value to identify which LUT is requested
+        cfilter: inter or string, only needed for port 50
+            filter number of the Infratec camera in port AEF50
+        verbose: integer, optional, default 0 
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if Look up table was found, False if not
+        LUT: list
+            contains the calibration information: [digital level, temperature, temperature error]
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     """
     LUT_dir = '\\\\sv-e4-fs-1\\E4-Mitarbeiter\\E4 Diagnostics\\QIR\\testingLUT'
     OP = IR_tools.get_OP_by_time(time_ns=this_time)
@@ -284,14 +313,32 @@ def read_LUT_from_file(port, this_time, t_exp, emissivity, cfilter, verbose=0):#
 
 def download_NUC_by_program(port, program, exposure, version=0, testmode=False, verbose=0):
     '''
+    download the NUC data for a given port, program and exposure time of the camera
     INPUT
     ------
-    
+        port: integer or string
+            port number of the AEF port of the camera
+        program: string
+            program id in the form of 'YYYYMMDD.XXX', e.g. '20181016.016'
+        exposure: integer
+            exposure time in microseconds
+        version: integer, optional, default 0
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional, default False
+            if True, the data will be loaded from the test archive
+        verbose: integer, optional, default 0
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if NUC data was found, False if not
+        NUC: list
+            a list vontaining images of gain, offset, cold, badpixels, gain_error and offset_error
+        NUC_describtion: list
+            a list which entry in NUC is what, looks like ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     '''
 #    prog=get_program_from_PID(program)
 #    try:
@@ -310,14 +357,34 @@ def download_NUC_by_program(port, program, exposure, version=0, testmode=False,
 
 def download_NUC_by_times(port, starttime, stoptime, exposure, testmode=False, version=0, verbose=0):
     '''
+    download the NUC data for a given port, start and endtime in ns and exposure time of the camera
     INPUT
     ------
-    
+        port: integer or string
+            port number of the AEF port of the camera
+        starttime: integer
+            time in ns where the search should start
+        stoptime: integer
+            time in sn where the search should end
+        exposure: integer
+            exposure time in microseconds
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional
+            if True, the data will be loaded from the test archive
+        verbose: integer, optional    
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if NUC data was found, False if not
+        NUC: list
+            a list vontaining images of gain, offset, cold, badpixels, gain_error and offset_error
+        NUC_describtion: list
+            a list which entry in NUC is what, looks like ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     '''
     OP = IR_tools.get_OP_by_time(time_ns=starttime)
     if testmode:
@@ -370,12 +437,29 @@ def get_NUC_by_program(port, program, exposure, version=0, testmode=False, verbo
     the cold frame and/or compute the gain, offset from the hot and cold frame.
     INPUT
     ------
-    
+        port: integer or string
+            port number of the AEF port of the camera
+        program: string
+            program id in the form of 'YYYYMMDD.XXX', e.g. '20181016.016'
+        exposure: integer
+            exposure time in microseconds
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional
+            if True, the data will be loaded from the test archive
+        verbose: integer, optional    
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if NUC data was found, False if not
+        NUC: list
+            a list vontaining images of gain, offset, cold, badpixels, gain_error and offset_error
+        NUC_describtion: list
+            a list which entry in NUC is what, looks like ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     '''
 #    prog=get_program_from_PID(program)
 #    try:
@@ -399,23 +483,46 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
     Loads NUC elements (gain, offset cold image, bad pixels) for an IR camera
     in a given port and time interval. Depending on time and camera, reconstruct
     the cold frame and/or compute the gain, offset from the hot and cold frame.
+    the result is operation phase dependent, due to changes in the aquisition and saving programs
+    
     INPUT
     ------
-    
+        port: integer or string
+            port number of the AEF port of the camera
+        starttime: integer
+            time in ns where the search should start
+        stoptime: integer
+            time in sn where the search should end
+        exposure: integer
+            exposure time in microseconds
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional
+            if True, the data will be loaded from the test archive
+        OP: string, optional, default None
+            name of operation phase
+        verbose: integer, optional    
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if NUC data was found, False if not
+        NUC: list
+            a list vontaining images of gain, offset, cold, badpixels, gain_error and offset_error
+        NUC_describtion: list
+            a list which entry in NUC is what, looks like ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     """
-    if OP is None:
+    if OP is None: #get operation phase if not given
         OP = IR_tools.get_OP_by_time(time_ns=stoptime)
-    if OP == "OP1.2a":
+    if OP == "OP1.2a": # check for operation phase
         t1 = stoptime
         t0 = starttime
         prog = read_program(t1)
         program = prog[1]['id']
-        if (port == 31) or (port == 21 and float(program[4:]) > 1110):
+        if (port == 31) or (port == 21 and float(program[4:]) > 1110):#camera 31 and 21 need after 20171110 new coldframes due to a open shutter during the recording of the coldframes
             if verbose > 0:
                 print("get_NUC_by_times: rebuilding coldframe")
             #use any way the rebuilt coldframe.
@@ -427,7 +534,9 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
                 coldref = hotcold[1]
                 hotref = hotcold[0]
             else:
-                raise Exception("get_NUC_by_times: unable to download reference frames")
+#                raise Exception("get_NUC_by_times: unable to download reference frames")
+                logging.warning("get_NUC_by_times: unable to download reference frames")
+                return False, 0, 0
             filestring = 'AEF' + str(port) + '_et' + str(int(t_exp))
             amap = np.load(join(IRCamColdframes_fittingpath, filestring + '_a.npy'))
             bmap = np.load(join(IRCamColdframes_fittingpath, filestring + '_b.npy'))
@@ -441,7 +550,7 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
             gain_error = 0
             offset_error = 0
             return True, [gain, offset, cirebuild, badpixels, gain_error, offset_error], ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
-        elif (port == 11) or (port == 21):
+        elif (port == 11) or (port == 21): # case of the swaped cameras
             if verbose > 0:
                 print('get_NUC_by_times: downloading NUC')
     #                        coldref, hotref = IR_tools.load_ref_images('AEF' + str(port), t_exp)
@@ -450,10 +559,14 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
                 coldref = hotcold[1]
                 hotref = hotcold[0]
             else:
-                raise Exception("get_NUC_by_times: unable to download reference frames")
+#                raise Exception("get_NUC_by_times: unable to download reference frames")
+                logging.warning("get_NUC_by_times: unable to download reference frames")
+                return False, 0, 0
             NUC_DL = download_NUC_by_times(port, t0, t1, t_exp, version, testmode)
             if NUC_DL[0] == False:
-                raise Exception("get_NUC_by_times: NUC was not found")
+#                raise Exception("get_NUC_by_times: NUC was not found")
+                logging.warning("get_NUC_by_times: NUC was not found")
+                return False, 0, 0
             else:
                 ci = NUC_DL[1][2]
             gain, offset = IR_tools.calculate_gain_offset_image(ci, None, coldref, hotref, verbose=verbose-1)
@@ -463,7 +576,7 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
             gain_error = 0
             offset_error = 0
             return True, [gain, offset, ci, badpixels, gain_error, offset_error], ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
-        else:
+        else: #all other cases, the "normal" case
             if verbose > 0:
                 print('get_NUC_by_times: downloading NUC')
             NUC_DL = download_NUC_by_times(port, starttime, stoptime, t_exp, testmode=testmode)
@@ -480,13 +593,15 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
             coldref = hotcold[1]
             hotref = hotcold[0]
         else:
-            raise Exception("get_NUC_by_times: unable to download reference frames")
+#            raise Exception("get_NUC_by_times: unable to download reference frames")
+            logging.warning("get_NUC_by_times: unable to download reference frames")
+            return False, 0, 0
         ### get the cold frame: 1. get the metastream frametype and exposuretime and check in it for the positions, if -1 type is unknown, when if needs to be identified in a different way
         try:
             exist, _, coldframes = download_calibration_raw_files_by_time(port, t_exp, starttime, stoptime, frametype=0, version=0, testmode=testmode, verbose=verbose-1)
         except:
             exist = False
-        if exist:
+        if exist:#okay it exist, average the cold frames
             cold = np.zeros(np.shape(coldframes[0]), dtype=np.uint64)
             for ele in coldframes:
                 cold += ele
@@ -499,30 +614,54 @@ def get_NUC_by_times(port, starttime, stoptime, t_exp, version=0, testmode=False
         badpixels = find_badpixels(port, gain, offset, niterations=10, tolerance=10, verbose=verbose-1)
         return True, [gain, offset, cold, badpixels, gain_error, offset_error], ['gain', 'offset', 'cold', 'badpixels', 'gain_error', 'offset_error']
     else:
-        raise Exception("get_NUC_by_times: unknown Operation phase or NUC method not implemented for this OP")
+#        raise Exception("get_NUC_by_times: unknown Operation phase or NUC method not implemented for this OP")
+        logging.warning("get_NUC_by_times: unknown Operation phase or NUC method not implemented for this OP")
+        return False, 0, 0
 
 def download_calibration_raw_files_by_time(port, t_exp, starttime, stoptime, frametype=0, version=0, testmode=False, verbose=0):
     """
-    download_calibration_raw_files_by_time(port, t_exp, starttime, stoptime, frametype=0, version=0)
+    download of the calibration raw files which are recorded between t0 and t1 trigger
     frametype: 0 for closed shutter frames (cold), 1 for open shutter frames (background)
+    
     INPUT
     ------
-    
+        port: integer 
+            port number of the AEF port of the camera
+        t_exp: integer
+            exposure time in microseconds
+        starttime: integer
+            time in ns where the search should start        
+        stoptime: integer
+            time in sn where the search should end 
+        frametype: integer, default 0
+            defines which type of frames should be returned, 0 for cold frames, 1 for background frames
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional
+            if True, the data will be loaded from the test archive        
+        verbose: integer, optional    
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if data was found, False if not
+        time: list
+            a list containing the time stamp of each frame
+        frames: list
+            a list of frames, 2D numpy arrays
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     """
     gotit, time_t, texp_t = get_exposure_by_times(port, int(starttime-100), int(stoptime-20e6), testmode=testmode)
     OP = IR_tools.get_OP_by_time(time_ns=stoptime)
     if gotit:
         expinds = np.where(texp_t == t_exp)[0]
         if len(expinds) == 0:
-            print("download_calibration_raw_files_by_time: Error! cannot find the exposure time in the given data")
+            logging.warning("download_calibration_raw_files_by_time: Error! cannot find the exposure time in the given data")
             return False, [0], [0]
     else:
-        print("download_calibration_raw_files_by_time: Error! exposure time not found")
+        logging.warning("download_calibration_raw_files_by_time: Error! exposure time not found")
         return False, [0], [0]
     gotitf, timef, values_f = get_frametype_by_times(port, int(starttime-100), int(stoptime-20e6), testmode=testmode)
     if gotitf:
@@ -545,7 +684,9 @@ def download_calibration_raw_files_by_time(port, t_exp, starttime, stoptime, fra
                 ref_t = [np.min(frametimes[turnpoint+1:]), np.max(frametimes[turnpoint+1:])]
 #                print((ref_t[1]-ref_t[0])/1e9)
             else:
-                raise Exception("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+#                raise Exception("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+                logging.warning("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+                return False, [0], [0]
     else:
         print("download_calibration_raw_files_by_time: Error! frametype not found")
         frametimes = time_t[expinds]
@@ -559,7 +700,9 @@ def download_calibration_raw_files_by_time(port, t_exp, starttime, stoptime, fra
             ref_t = [np.min(frametimes[turnpoint+1:]), np.max(frametimes[turnpoint+1:])]
             print((ref_t[1]-ref_t[0])/1e9)
         else:
-            raise Exception("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+#            raise Exception("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+            logging.warning("download_calibration_raw_files_by_time: requested Frametype unknown and not implemented!")
+            return False, [0], [0]
 #        return False, [0], [0]
     t1date = datetime.datetime.utcfromtimestamp((stoptime-100)/1e9)
     t1date = t1date.isoformat()
@@ -588,12 +731,29 @@ def download_hot_cold_reference_by_times(port, exposure, starttime=1503907200000
     Uses first calibration frames if time is not defined.
     INPUT
     ------
-    
+        port: integer or string
+            port number of the AEF port of the camera, or string containing the full port name
+        exposure: integer
+            exposure time in microseconds
+        starttime: integer
+            time in ns where the search should start
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional
+            if True, the data will be loaded from the test archive        
+        verbose: integer, optional    
+            feedback level (details of print messages) 
     RESULT
     ------
-    
+        exist: boolean
+            True if data was found, False if not
+        images: list
+            a list vontaining images of gain, offset, cold, badpixels, gain_error and offset_error
+        image_describtion: list
+            a list which entry in NUC is what, looks like ['hot','cold']
     NOTE
     ------
+        function need to be adapted for OP2, if AEK ports come into opperation
     """
     OP = IR_tools.get_OP_by_time(time_ns=starttime)
     if isinstance(port, int):
@@ -648,15 +808,37 @@ def download_hot_cold_reference_by_times(port, exposure, starttime=1503907200000
 
 def download_background_by_program(port, program, exposure, camera_filter=0, version=0, testmode=False, verbose=0):
     '''
+    download the background from the AEF cameras in OP1.2a, which are recorded between t0 and t1 trigger
     returned exist, time, frame
     INPUT
     ------
-    
+        port: integer 
+            port number of the AEF port of the camera        
+        program: string
+            program id in the form of 'YYYYMMDD.XXX', e.g. '20181016.016'
+        exposure: integer
+            exposure time in microseconds        
+        camera_filter: integer or string, only needed for port 50
+            filter number of the Infratec camera in port AEF50
+        version: integer, optional
+            Version of the data in the archiveDB, in case of 0 the highest version will be used
+        testmode: boolean, optional