python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 735745 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2818118'] with mtr_portfolio_ids : ['22864312'] and first list_photo_ids : [] new path : /proc/735745/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 4 ; length of list_pids : 4 ; length of list_args : 4 time to download the photos : 0.9642820358276367 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Sat May 10 19:41:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2916 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-10 19:41:33.965948: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-10 19:41:33.995380: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-10 19:41:33.998003: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff448000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-10 19:41:33.998084: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-10 19:41:34.003145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-10 19:41:34.128277: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x41b5ff0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-10 19:41:34.128372: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-10 19:41:34.129561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:41:34.130062: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:41:34.133595: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:41:34.136816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:41:34.137255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:41:34.140007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:41:34.141334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:41:34.146772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:41:34.148156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:41:34.148283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:41:34.148880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-10 19:41:34.148899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-10 19:41:34.148909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-10 19:41:34.149794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2464 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-10 19:41:34.468204: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:41:34.468392: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:41:34.468417: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:41:34.468439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:41:34.468459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:41:34.468478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:41:34.468497: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:41:34.468517: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:41:34.469520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:41:34.471056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:41:34.471122: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:41:34.471145: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:41:34.471164: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:41:34.471204: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:41:34.471225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:41:34.471244: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:41:34.471265: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:41:34.472254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:41:34.472308: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-10 19:41:34.472319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-10 19:41:34.472329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-10 19:41:34.473386: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2464 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque'2025-05-10 19:41:47.009194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:41:47.199515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:41:48.718636: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.718723: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.725485: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.725554: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.779313: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.779369: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.822779: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.822833: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.868737: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.868803: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:41:48.870819: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.41G (1511915520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:48.871400: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.27G (1360723968 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.190491: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-10 19:41:49.229238: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.230330: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.231789: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.232585: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.243102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.243865: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.244498: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.245081: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.245686: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.246293: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.261171: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.261783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.284831: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.284895: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-10 19:41:49.285928: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.286707: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.293794: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.294545: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.303661: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.304770: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.320829: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.321817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.322795: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.323759: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.328409: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.329370: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.330389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.331446: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.333118: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.343315: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.344117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.353191: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.353750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.354308: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.354854: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.355459: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:41:49.356045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 7 Detection mask done ! Trying to reset tf kernel 735974 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1723 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2916 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.03629803657531738 nb_pixel_total : 756628 time to create 1 rle with new method : 0.18666362762451172 length of segment : 1049 time for calcul the mask position with numpy : 0.0009329319000244141 nb_pixel_total : 78050 time to create 1 rle with old method : 0.0862579345703125 length of segment : 250 time for calcul the mask position with numpy : 0.0032286643981933594 nb_pixel_total : 236489 time to create 1 rle with new method : 0.011718034744262695 length of segment : 716 time for calcul the mask position with numpy : 0.0018155574798583984 nb_pixel_total : 80157 time to create 1 rle with old method : 0.10259270668029785 length of segment : 518 time for calcul the mask position with numpy : 0.0012352466583251953 nb_pixel_total : 87351 time to create 1 rle with old method : 0.09954595565795898 length of segment : 441 time for calcul the mask position with numpy : 0.0013015270233154297 nb_pixel_total : 59540 time to create 1 rle with old method : 0.06355142593383789 length of segment : 212 time for calcul the mask position with numpy : 0.00040721893310546875 nb_pixel_total : 20740 time to create 1 rle with old method : 0.023622512817382812 length of segment : 237 time for calcul the mask position with numpy : 0.0006494522094726562 nb_pixel_total : 11754 time to create 1 rle with old method : 0.013889789581298828 length of segment : 227 time for calcul the mask position with numpy : 0.0003540515899658203 nb_pixel_total : 17836 time to create 1 rle with old method : 0.020238637924194336 length of segment : 152 time for calcul the mask position with numpy : 0.0005195140838623047 nb_pixel_total : 27030 time to create 1 rle with old method : 0.030483007431030273 length of segment : 267 time for calcul the mask position with numpy : 0.0004260540008544922 nb_pixel_total : 22071 time to create 1 rle with old method : 0.024358272552490234 length of segment : 195 time for calcul the mask position with numpy : 0.010831356048583984 nb_pixel_total : 643404 time to create 1 rle with new method : 0.2557084560394287 length of segment : 676 time for calcul the mask position with numpy : 0.0008828639984130859 nb_pixel_total : 43165 time to create 1 rle with old method : 0.04966592788696289 length of segment : 198 time for calcul the mask position with numpy : 0.0005075931549072266 nb_pixel_total : 18265 time to create 1 rle with old method : 0.02101445198059082 length of segment : 233 time for calcul the mask position with numpy : 0.0008404254913330078 nb_pixel_total : 49018 time to create 1 rle with old method : 0.05533909797668457 length of segment : 403 time for calcul the mask position with numpy : 0.0015828609466552734 nb_pixel_total : 67164 time to create 1 rle with old method : 0.09399890899658203 length of segment : 367 time for calcul the mask position with numpy : 0.0038099288940429688 nb_pixel_total : 240568 time to create 1 rle with new method : 0.014640569686889648 length of segment : 587 time for calcul the mask position with numpy : 0.013282299041748047 nb_pixel_total : 574815 time to create 1 rle with new method : 0.053617000579833984 length of segment : 676 time for calcul the mask position with numpy : 0.002422332763671875 nb_pixel_total : 110566 time to create 1 rle with old method : 0.12418031692504883 length of segment : 382 time for calcul the mask position with numpy : 0.0012097358703613281 nb_pixel_total : 49051 time to create 1 rle with old method : 0.05248093605041504 length of segment : 228 time for calcul the mask position with numpy : 0.0017702579498291016 nb_pixel_total : 53570 time to create 1 rle with old method : 0.05952143669128418 length of segment : 260 time for calcul the mask position with numpy : 0.0012161731719970703 nb_pixel_total : 47223 time to create 1 rle with old method : 0.052532196044921875 length of segment : 202 time for calcul the mask position with numpy : 0.003050565719604492 nb_pixel_total : 115033 time to create 1 rle with old method : 0.12451529502868652 length of segment : 586 time for calcul the mask position with numpy : 0.0012898445129394531 nb_pixel_total : 51546 time to create 1 rle with old method : 0.05828666687011719 length of segment : 186 time for calcul the mask position with numpy : 0.002290010452270508 nb_pixel_total : 83349 time to create 1 rle with old method : 0.09632253646850586 length of segment : 411 time for calcul the mask position with numpy : 0.0009949207305908203 nb_pixel_total : 7336 time to create 1 rle with old method : 0.008756875991821289 length of segment : 177 time for calcul the mask position with numpy : 1.0968363285064697 nb_pixel_total : 8605122 time to create 1 rle with new method : 1.3104496002197266 length of segment : 4503 time for calcul the mask position with numpy : 0.25183916091918945 nb_pixel_total : 2397973 time to create 1 rle with new method : 0.28307199478149414 length of segment : 1742 time for calcul the mask position with numpy : 0.00311279296875 nb_pixel_total : 238153 time to create 1 rle with new method : 0.005000114440917969 length of segment : 205 time for calcul the mask position with numpy : 0.0009837150573730469 nb_pixel_total : 46672 time to create 1 rle with old method : 0.05311012268066406 length of segment : 181 time for calcul the mask position with numpy : 0.015297412872314453 nb_pixel_total : 911095 time to create 1 rle with new method : 0.026029348373413086 length of segment : 539 time for calcul the mask position with numpy : 0.505868673324585 nb_pixel_total : 3960924 time to create 1 rle with new method : 0.4774808883666992 length of segment : 2116 time for calcul the mask position with numpy : 0.23660802841186523 nb_pixel_total : 5134671 time to create 1 rle with new method : 0.7160820960998535 length of segment : 2091 time spent for convertir_results : 14.20518159866333 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 33 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 51.59082293510437 time spend to save output : 0.08360576629638672 total time spend for step 1 : 51.67442870140076 step2:mask_detect Sat May 10 19:42:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2916 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-10 19:42:26.997085: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-10 19:42:27.035107: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-10 19:42:27.037430: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff450000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-10 19:42:27.037496: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-10 19:42:27.041726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-10 19:42:27.189805: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5341840 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-10 19:42:27.189871: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-10 19:42:27.190799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:42:27.193232: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:42:27.223219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:42:27.242034: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:42:27.246689: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:42:27.272510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:42:27.277405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:42:27.325842: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:42:27.327190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:42:27.327287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:42:27.328379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-10 19:42:27.328400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-10 19:42:27.328411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-10 19:42:27.329934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2464 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-10 19:42:27.531749: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:42:27.538065: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:42:27.538129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:42:27.538148: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:42:27.538164: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:42:27.538179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:42:27.538208: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:42:27.538248: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:42:27.539250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:42:27.540327: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-10 19:42:27.540389: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-10 19:42:27.540408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:42:27.540424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-10 19:42:27.540439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-10 19:42:27.540455: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-10 19:42:27.540470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-10 19:42:27.540486: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:42:27.541421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-10 19:42:27.541460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-10 19:42:27.541472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-10 19:42:27.541481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-10 19:42:27.542544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2464 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-10 19:42:41.489408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-10 19:42:41.957541: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-10 19:42:44.970330: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:44.970749: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:44.977303: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:44.977333: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.029175: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.029236: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.073003: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.073052: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.125640: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.125687: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-10 19:42:45.128409: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.41G (1511915520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.128951: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.27G (1360723968 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.456890: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-10 19:42:45.497556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.498711: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.500255: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.501305: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.519208: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.519917: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.520526: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.521116: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.521729: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.522323: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.537546: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.538178: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.578714: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.578773: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-10 19:42:45.579569: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.580336: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.587441: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.588232: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.596427: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.597049: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.625862: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.626846: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.627783: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.628481: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.632820: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.633556: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.634339: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.635103: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.637528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.646668: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.647403: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.656946: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.657658: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.658363: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.659141: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.659884: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-10 19:42:45.660602: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.91G (2048786432 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 78 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 70 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 54 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 737943 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2916 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4109 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.004388570785522461 nb_pixel_total : 54013 time to create 1 rle with old method : 0.07080268859863281 length of segment : 308 time for calcul the mask position with numpy : 0.0012946128845214844 nb_pixel_total : 49822 time to create 1 rle with old method : 0.05560183525085449 length of segment : 271 time for calcul the mask position with numpy : 0.0013079643249511719 nb_pixel_total : 61283 time to create 1 rle with old method : 0.06750226020812988 length of segment : 235 time for calcul the mask position with numpy : 0.0010304450988769531 nb_pixel_total : 37750 time to create 1 rle with old method : 0.04128384590148926 length of segment : 367 time for calcul the mask position with numpy : 0.0008985996246337891 nb_pixel_total : 37768 time to create 1 rle with old method : 0.043300628662109375 length of segment : 254 time for calcul the mask position with numpy : 0.002254009246826172 nb_pixel_total : 50750 time to create 1 rle with old method : 0.056281089782714844 length of segment : 331 time for calcul the mask position with numpy : 0.001010894775390625 nb_pixel_total : 46901 time to create 1 rle with old method : 0.05303955078125 length of segment : 257 time for calcul the mask position with numpy : 0.0014824867248535156 nb_pixel_total : 58446 time to create 1 rle with old method : 0.06644177436828613 length of segment : 303 time for calcul the mask position with numpy : 0.0011925697326660156 nb_pixel_total : 49449 time to create 1 rle with old method : 0.05475759506225586 length of segment : 236 time for calcul the mask position with numpy : 0.007212400436401367 nb_pixel_total : 359862 time to create 1 rle with new method : 0.07542896270751953 length of segment : 636 time for calcul the mask position with numpy : 0.004131793975830078 nb_pixel_total : 170207 time to create 1 rle with new method : 0.01199197769165039 length of segment : 434 time for calcul the mask position with numpy : 0.005072832107543945 nb_pixel_total : 186000 time to create 1 rle with new method : 0.014807939529418945 length of segment : 753 time for calcul the mask position with numpy : 0.0023164749145507812 nb_pixel_total : 105745 time to create 1 rle with old method : 0.11471247673034668 length of segment : 286 time for calcul the mask position with numpy : 0.0008590221405029297 nb_pixel_total : 33734 time to create 1 rle with old method : 0.03767800331115723 length of segment : 272 time for calcul the mask position with numpy : 0.010404586791992188 nb_pixel_total : 433034 time to create 1 rle with new method : 0.051171302795410156 length of segment : 1094 time for calcul the mask position with numpy : 0.0015528202056884766 nb_pixel_total : 67378 time to create 1 rle with old method : 0.09279584884643555 length of segment : 219 time for calcul the mask position with numpy : 0.004438638687133789 nb_pixel_total : 112676 time to create 1 rle with old method : 0.19249176979064941 length of segment : 589 time for calcul the mask position with numpy : 0.008525371551513672 nb_pixel_total : 332365 time to create 1 rle with new method : 0.02717876434326172 length of segment : 809 time for calcul the mask position with numpy : 0.0017230510711669922 nb_pixel_total : 77392 time to create 1 rle with old method : 0.08580565452575684 length of segment : 250 time for calcul the mask position with numpy : 0.009511232376098633 nb_pixel_total : 277635 time to create 1 rle with new method : 0.027434587478637695 length of segment : 890 time for calcul the mask position with numpy : 0.012666463851928711 nb_pixel_total : 464788 time to create 1 rle with new method : 1.0037431716918945 length of segment : 964 time for calcul the mask position with numpy : 0.0026619434356689453 nb_pixel_total : 112080 time to create 1 rle with old method : 0.1244649887084961 length of segment : 615 time for calcul the mask position with numpy : 0.0010509490966796875 nb_pixel_total : 43435 time to create 1 rle with old method : 0.06627273559570312 length of segment : 265 time for calcul the mask position with numpy : 0.002366781234741211 nb_pixel_total : 102072 time to create 1 rle with old method : 0.12537670135498047 length of segment : 363 time for calcul the mask position with numpy : 0.0060672760009765625 nb_pixel_total : 203422 time to create 1 rle with new method : 0.016350984573364258 length of segment : 550 time for calcul the mask position with numpy : 0.0016469955444335938 nb_pixel_total : 30854 time to create 1 rle with old method : 0.04531455039978027 length of segment : 313 time for calcul the mask position with numpy : 0.0054607391357421875 nb_pixel_total : 188791 time to create 1 rle with new method : 0.016090869903564453 length of segment : 541 time for calcul the mask position with numpy : 0.0013053417205810547 nb_pixel_total : 35511 time to create 1 rle with old method : 0.057552337646484375 length of segment : 195 time for calcul the mask position with numpy : 0.003116607666015625 nb_pixel_total : 70542 time to create 1 rle with old method : 0.09158968925476074 length of segment : 292 time for calcul the mask position with numpy : 0.0029861927032470703 nb_pixel_total : 110692 time to create 1 rle with old method : 0.12392759323120117 length of segment : 525 time for calcul the mask position with numpy : 0.0015807151794433594 nb_pixel_total : 61291 time to create 1 rle with old method : 0.07656025886535645 length of segment : 291 time for calcul the mask position with numpy : 0.0007085800170898438 nb_pixel_total : 30458 time to create 1 rle with old method : 0.03524327278137207 length of segment : 185 time for calcul the mask position with numpy : 0.0005600452423095703 nb_pixel_total : 17626 time to create 1 rle with old method : 0.02109241485595703 length of segment : 138 time for calcul the mask position with numpy : 0.010143518447875977 nb_pixel_total : 290280 time to create 1 rle with new method : 0.03195691108703613 length of segment : 1398 time for calcul the mask position with numpy : 0.006522417068481445 nb_pixel_total : 331220 time to create 1 rle with new method : 0.016366004943847656 length of segment : 1059 time for calcul the mask position with numpy : 0.008080720901489258 nb_pixel_total : 272549 time to create 1 rle with new method : 0.023320436477661133 length of segment : 861 time for calcul the mask position with numpy : 0.0064737796783447266 nb_pixel_total : 265540 time to create 1 rle with new method : 0.01628589630126953 length of segment : 686 time for calcul the mask position with numpy : 0.0012154579162597656 nb_pixel_total : 56171 time to create 1 rle with old method : 0.06274771690368652 length of segment : 207 time for calcul the mask position with numpy : 0.0015745162963867188 nb_pixel_total : 40748 time to create 1 rle with old method : 0.04787254333496094 length of segment : 319 time for calcul the mask position with numpy : 0.0006051063537597656 nb_pixel_total : 20624 time to create 1 rle with old method : 0.024061203002929688 length of segment : 184 time for calcul the mask position with numpy : 0.0033588409423828125 nb_pixel_total : 150191 time to create 1 rle with new method : 0.009537220001220703 length of segment : 415 time for calcul the mask position with numpy : 0.002178192138671875 nb_pixel_total : 79058 time to create 1 rle with old method : 0.09150433540344238 length of segment : 404 time for calcul the mask position with numpy : 0.003286123275756836 nb_pixel_total : 116062 time to create 1 rle with old method : 0.13177132606506348 length of segment : 498 time for calcul the mask position with numpy : 0.0008592605590820312 nb_pixel_total : 36570 time to create 1 rle with old method : 0.05942964553833008 length of segment : 172 time for calcul the mask position with numpy : 0.0033686161041259766 nb_pixel_total : 155202 time to create 1 rle with new method : 0.007966756820678711 length of segment : 422 time for calcul the mask position with numpy : 0.004786491394042969 nb_pixel_total : 204032 time to create 1 rle with new method : 0.012671709060668945 length of segment : 603 time for calcul the mask position with numpy : 0.001499176025390625 nb_pixel_total : 72550 time to create 1 rle with old method : 0.08227849006652832 length of segment : 239 time for calcul the mask position with numpy : 0.0027511119842529297 nb_pixel_total : 127050 time to create 1 rle with old method : 0.14401888847351074 length of segment : 559 time for calcul the mask position with numpy : 0.011053323745727539 nb_pixel_total : 373721 time to create 1 rle with new method : 0.02657794952392578 length of segment : 946 time for calcul the mask position with numpy : 0.011775016784667969 nb_pixel_total : 591441 time to create 1 rle with new method : 0.02928781509399414 length of segment : 1043 time for calcul the mask position with numpy : 0.0027534961700439453 nb_pixel_total : 89678 time to create 1 rle with old method : 0.10359978675842285 length of segment : 495 time for calcul the mask position with numpy : 0.002232789993286133 nb_pixel_total : 101100 time to create 1 rle with old method : 0.12111258506774902 length of segment : 366 time for calcul the mask position with numpy : 0.002577066421508789 nb_pixel_total : 90454 time to create 1 rle with old method : 0.1035313606262207 length of segment : 281 time for calcul the mask position with numpy : 0.0022258758544921875 nb_pixel_total : 99271 time to create 1 rle with old method : 0.13624167442321777 length of segment : 546 time for calcul the mask position with numpy : 0.001634359359741211 nb_pixel_total : 76210 time to create 1 rle with old method : 0.08877372741699219 length of segment : 316 time for calcul the mask position with numpy : 0.0005981922149658203 nb_pixel_total : 25866 time to create 1 rle with old method : 0.02969813346862793 length of segment : 162 time for calcul the mask position with numpy : 0.0024344921112060547 nb_pixel_total : 128479 time to create 1 rle with old method : 0.1428983211517334 length of segment : 433 time for calcul the mask position with numpy : 0.0016689300537109375 nb_pixel_total : 70443 time to create 1 rle with old method : 0.07862305641174316 length of segment : 388 time for calcul the mask position with numpy : 0.004851341247558594 nb_pixel_total : 178239 time to create 1 rle with new method : 0.015545129776000977 length of segment : 589 time for calcul the mask position with numpy : 0.0024857521057128906 nb_pixel_total : 21647 time to create 1 rle with old method : 0.02463555335998535 length of segment : 369 time for calcul the mask position with numpy : 0.010622501373291016 nb_pixel_total : 550443 time to create 1 rle with new method : 0.027160167694091797 length of segment : 1042 time for calcul the mask position with numpy : 0.0030646324157714844 nb_pixel_total : 124329 time to create 1 rle with old method : 0.1360633373260498 length of segment : 397 time for calcul the mask position with numpy : 0.0010979175567626953 nb_pixel_total : 31046 time to create 1 rle with old method : 0.03535890579223633 length of segment : 247 time for calcul the mask position with numpy : 0.0006947517395019531 nb_pixel_total : 31288 time to create 1 rle with old method : 0.03484082221984863 length of segment : 212 time for calcul the mask position with numpy : 0.0014138221740722656 nb_pixel_total : 69100 time to create 1 rle with old method : 0.08133482933044434 length of segment : 268 time for calcul the mask position with numpy : 0.002026796340942383 nb_pixel_total : 59619 time to create 1 rle with old method : 0.06568336486816406 length of segment : 261 time for calcul the mask position with numpy : 0.0021483898162841797 nb_pixel_total : 95105 time to create 1 rle with old method : 0.10761880874633789 length of segment : 393 time for calcul the mask position with numpy : 0.0025827884674072266 nb_pixel_total : 86817 time to create 1 rle with old method : 0.09942221641540527 length of segment : 367 time for calcul the mask position with numpy : 0.003043651580810547 nb_pixel_total : 113726 time to create 1 rle with old method : 0.12815356254577637 length of segment : 507 time for calcul the mask position with numpy : 0.005604982376098633 nb_pixel_total : 187061 time to create 1 rle with new method : 0.016277790069580078 length of segment : 610 time for calcul the mask position with numpy : 0.001893758773803711 nb_pixel_total : 77066 time to create 1 rle with old method : 0.08679699897766113 length of segment : 339 time for calcul the mask position with numpy : 0.002490997314453125 nb_pixel_total : 105669 time to create 1 rle with old method : 0.11816215515136719 length of segment : 446 time for calcul the mask position with numpy : 0.0030341148376464844 nb_pixel_total : 110173 time to create 1 rle with old method : 0.12318229675292969 length of segment : 450 time for calcul the mask position with numpy : 0.001953125 nb_pixel_total : 88582 time to create 1 rle with old method : 0.10059118270874023 length of segment : 360 time for calcul the mask position with numpy : 0.003359079360961914 nb_pixel_total : 159004 time to create 1 rle with new method : 0.011751174926757812 length of segment : 982 time for calcul the mask position with numpy : 0.0029616355895996094 nb_pixel_total : 129206 time to create 1 rle with old method : 0.14391708374023438 length of segment : 846 time for calcul the mask position with numpy : 0.0018949508666992188 nb_pixel_total : 96280 time to create 1 rle with old method : 0.10720491409301758 length of segment : 313 time for calcul the mask position with numpy : 0.0014395713806152344 nb_pixel_total : 65182 time to create 1 rle with old method : 0.07475638389587402 length of segment : 296 time for calcul the mask position with numpy : 0.00047850608825683594 nb_pixel_total : 17179 time to create 1 rle with old method : 0.018750667572021484 length of segment : 165 time for calcul the mask position with numpy : 0.004201173782348633 nb_pixel_total : 154871 time to create 1 rle with new method : 0.01113438606262207 length of segment : 432 time for calcul the mask position with numpy : 0.008946657180786133 nb_pixel_total : 404127 time to create 1 rle with new method : 0.022275447845458984 length of segment : 1193 time for calcul the mask position with numpy : 0.0023479461669921875 nb_pixel_total : 87895 time to create 1 rle with old method : 0.10231566429138184 length of segment : 280 time for calcul the mask position with numpy : 0.0033330917358398438 nb_pixel_total : 161879 time to create 1 rle with new method : 0.007712364196777344 length of segment : 397 time for calcul the mask position with numpy : 0.006693601608276367 nb_pixel_total : 277736 time to create 1 rle with new method : 0.01660442352294922 length of segment : 668 time for calcul the mask position with numpy : 0.008769512176513672 nb_pixel_total : 248692 time to create 1 rle with new method : 0.022487878799438477 length of segment : 1253 time for calcul the mask position with numpy : 0.0007581710815429688 nb_pixel_total : 11773 time to create 1 rle with old method : 0.015908241271972656 length of segment : 159 time for calcul the mask position with numpy : 0.0023238658905029297 nb_pixel_total : 53664 time to create 1 rle with old method : 0.10323572158813477 length of segment : 293 time for calcul the mask position with numpy : 0.0021026134490966797 nb_pixel_total : 62102 time to create 1 rle with old method : 0.07168436050415039 length of segment : 348 time for calcul the mask position with numpy : 0.0030879974365234375 nb_pixel_total : 87474 time to create 1 rle with old method : 0.0997006893157959 length of segment : 464 time for calcul the mask position with numpy : 0.010001897811889648 nb_pixel_total : 318571 time to create 1 rle with new method : 0.023713111877441406 length of segment : 696 time for calcul the mask position with numpy : 0.011717796325683594 nb_pixel_total : 372946 time to create 1 rle with new method : 0.032318115234375 length of segment : 1041 time for calcul the mask position with numpy : 0.002603292465209961 nb_pixel_total : 103120 time to create 1 rle with old method : 0.11544489860534668 length of segment : 472 time for calcul the mask position with numpy : 0.001316070556640625 nb_pixel_total : 42077 time to create 1 rle with old method : 0.04816460609436035 length of segment : 249 time for calcul the mask position with numpy : 0.001201629638671875 nb_pixel_total : 54689 time to create 1 rle with old method : 0.06247901916503906 length of segment : 216 time for calcul the mask position with numpy : 0.0014979839324951172 nb_pixel_total : 39328 time to create 1 rle with old method : 0.04459190368652344 length of segment : 524 time for calcul the mask position with numpy : 0.0008873939514160156 nb_pixel_total : 33460 time to create 1 rle with old method : 0.03895998001098633 length of segment : 242 time for calcul the mask position with numpy : 0.0036470890045166016 nb_pixel_total : 166002 time to create 1 rle with new method : 0.009378433227539062 length of segment : 505 time for calcul the mask position with numpy : 0.0030517578125 nb_pixel_total : 125321 time to create 1 rle with old method : 0.13807916641235352 length of segment : 603 time for calcul the mask position with numpy : 0.0008554458618164062 nb_pixel_total : 26406 time to create 1 rle with old method : 0.029308557510375977 length of segment : 186 time for calcul the mask position with numpy : 0.0018763542175292969 nb_pixel_total : 58118 time to create 1 rle with old method : 0.06413054466247559 length of segment : 458 time for calcul the mask position with numpy : 0.0007498264312744141 nb_pixel_total : 26623 time to create 1 rle with old method : 0.02981853485107422 length of segment : 211 time for calcul the mask position with numpy : 0.0012135505676269531 nb_pixel_total : 60922 time to create 1 rle with old method : 0.07018876075744629 length of segment : 276 time for calcul the mask position with numpy : 0.00042819976806640625 nb_pixel_total : 16391 time to create 1 rle with old method : 0.019533157348632812 length of segment : 118 time for calcul the mask position with numpy : 0.0004467964172363281 nb_pixel_total : 16361 time to create 1 rle with old method : 0.01948261260986328 length of segment : 153 time for calcul the mask position with numpy : 0.0006000995635986328 nb_pixel_total : 10832 time to create 1 rle with old method : 0.013059854507446289 length of segment : 203 time for calcul the mask position with numpy : 0.0005176067352294922 nb_pixel_total : 21834 time to create 1 rle with old method : 0.02468585968017578 length of segment : 196 time for calcul the mask position with numpy : 0.0030362606048583984 nb_pixel_total : 100576 time to create 1 rle with old method : 0.1145932674407959 length of segment : 648 time for calcul the mask position with numpy : 0.003510713577270508 nb_pixel_total : 145987 time to create 1 rle with old method : 0.16052579879760742 length of segment : 563 time for calcul the mask position with numpy : 0.0066339969635009766 nb_pixel_total : 209525 time to create 1 rle with new method : 0.02013111114501953 length of segment : 1245 time for calcul the mask position with numpy : 0.014496088027954102 nb_pixel_total : 448737 time to create 1 rle with new method : 0.23955011367797852 length of segment : 1598 time for calcul the mask position with numpy : 0.0015094280242919922 nb_pixel_total : 51339 time to create 1 rle with old method : 0.06915020942687988 length of segment : 276 time for calcul the mask position with numpy : 0.0006859302520751953 nb_pixel_total : 21509 time to create 1 rle with old method : 0.024241924285888672 length of segment : 285 time for calcul the mask position with numpy : 0.009160995483398438 nb_pixel_total : 434499 time to create 1 rle with new method : 0.023707151412963867 length of segment : 1239 time for calcul the mask position with numpy : 0.0019752979278564453 nb_pixel_total : 92224 time to create 1 rle with old method : 0.10264015197753906 length of segment : 440 time for calcul the mask position with numpy : 0.0017333030700683594 nb_pixel_total : 74307 time to create 1 rle with old method : 0.08467245101928711 length of segment : 303 time for calcul the mask position with numpy : 0.0007894039154052734 nb_pixel_total : 32034 time to create 1 rle with old method : 0.03947949409484863 length of segment : 266 time for calcul the mask position with numpy : 0.006926298141479492 nb_pixel_total : 194034 time to create 1 rle with new method : 0.01896524429321289 length of segment : 674 time for calcul the mask position with numpy : 0.002658843994140625 nb_pixel_total : 96905 time to create 1 rle with old method : 0.1151123046875 length of segment : 262 time for calcul the mask position with numpy : 0.0006539821624755859 nb_pixel_total : 26168 time to create 1 rle with old method : 0.029758453369140625 length of segment : 143 time for calcul the mask position with numpy : 0.006254911422729492 nb_pixel_total : 202513 time to create 1 rle with new method : 0.017006397247314453 length of segment : 1032 time for calcul the mask position with numpy : 0.0019555091857910156 nb_pixel_total : 91415 time to create 1 rle with old method : 0.12871170043945312 length of segment : 337 time for calcul the mask position with numpy : 0.002880573272705078 nb_pixel_total : 120490 time to create 1 rle with old method : 0.16247963905334473 length of segment : 482 time for calcul the mask position with numpy : 0.0015900135040283203 nb_pixel_total : 64720 time to create 1 rle with old method : 0.0735936164855957 length of segment : 353 time for calcul the mask position with numpy : 0.010355949401855469 nb_pixel_total : 399665 time to create 1 rle with new method : 0.029175996780395508 length of segment : 1165 time for calcul the mask position with numpy : 0.0021135807037353516 nb_pixel_total : 79389 time to create 1 rle with old method : 0.08863258361816406 length of segment : 329 time for calcul the mask position with numpy : 0.007021903991699219 nb_pixel_total : 376189 time to create 1 rle with new method : 0.017286300659179688 length of segment : 652 time for calcul the mask position with numpy : 0.0006666183471679688 nb_pixel_total : 22983 time to create 1 rle with old method : 0.026422739028930664 length of segment : 230 time for calcul the mask position with numpy : 0.0013058185577392578 nb_pixel_total : 10944 time to create 1 rle with old method : 0.012680530548095703 length of segment : 199 time for calcul the mask position with numpy : 0.0246429443359375 nb_pixel_total : 677031 time to create 1 rle with new method : 0.1419365406036377 length of segment : 1936 time for calcul the mask position with numpy : 0.011782407760620117 nb_pixel_total : 576405 time to create 1 rle with new method : 0.05829739570617676 length of segment : 1214 time for calcul the mask position with numpy : 0.006714582443237305 nb_pixel_total : 186001 time to create 1 rle with new method : 0.021315813064575195 length of segment : 836 time for calcul the mask position with numpy : 0.00483393669128418 nb_pixel_total : 220845 time to create 1 rle with new method : 0.011977910995483398 length of segment : 375 time for calcul the mask position with numpy : 0.002775430679321289 nb_pixel_total : 111980 time to create 1 rle with old method : 0.1276555061340332 length of segment : 651 time for calcul the mask position with numpy : 0.009517669677734375 nb_pixel_total : 397643 time to create 1 rle with new method : 0.030177831649780273 length of segment : 1084 time for calcul the mask position with numpy : 0.0024254322052001953 nb_pixel_total : 83918 time to create 1 rle with old method : 0.1222696304321289 length of segment : 485 time for calcul the mask position with numpy : 0.008552789688110352 nb_pixel_total : 354217 time to create 1 rle with new method : 0.017702817916870117 length of segment : 845 time for calcul the mask position with numpy : 0.005852460861206055 nb_pixel_total : 339997 time to create 1 rle with new method : 0.015590667724609375 length of segment : 825 time for calcul the mask position with numpy : 0.0013904571533203125 nb_pixel_total : 88183 time to create 1 rle with old method : 0.09946751594543457 length of segment : 389 time for calcul the mask position with numpy : 0.0031011104583740234 nb_pixel_total : 133154 time to create 1 rle with old method : 0.1503458023071289 length of segment : 564 time for calcul the mask position with numpy : 0.0017261505126953125 nb_pixel_total : 15615 time to create 1 rle with old method : 0.018537521362304688 length of segment : 170 time for calcul the mask position with numpy : 0.0011036396026611328 nb_pixel_total : 47780 time to create 1 rle with old method : 0.053605079650878906 length of segment : 296 time for calcul the mask position with numpy : 0.006003856658935547 nb_pixel_total : 294685 time to create 1 rle with new method : 0.017057418823242188 length of segment : 622 time for calcul the mask position with numpy : 0.0016870498657226562 nb_pixel_total : 88809 time to create 1 rle with old method : 0.0996699333190918 length of segment : 443 time for calcul the mask position with numpy : 0.0033965110778808594 nb_pixel_total : 165277 time to create 1 rle with new method : 0.010082721710205078 length of segment : 507 time for calcul the mask position with numpy : 0.0034105777740478516 nb_pixel_total : 152837 time to create 1 rle with new method : 0.00977468490600586 length of segment : 701 time for calcul the mask position with numpy : 0.0022695064544677734 nb_pixel_total : 77650 time to create 1 rle with old method : 0.0880594253540039 length of segment : 288 time for calcul the mask position with numpy : 0.003352642059326172 nb_pixel_total : 132973 time to create 1 rle with old method : 0.1577892303466797 length of segment : 497 time for calcul the mask position with numpy : 0.0011072158813476562 nb_pixel_total : 56444 time to create 1 rle with old method : 0.06357836723327637 length of segment : 245 time for calcul the mask position with numpy : 0.0012676715850830078 nb_pixel_total : 62667 time to create 1 rle with old method : 0.07058215141296387 length of segment : 261 time for calcul the mask position with numpy : 0.0012738704681396484 nb_pixel_total : 53269 time to create 1 rle with old method : 0.06092095375061035 length of segment : 277 time for calcul the mask position with numpy : 0.0063419342041015625 nb_pixel_total : 314416 time to create 1 rle with new method : 0.014276981353759766 length of segment : 520 time for calcul the mask position with numpy : 0.0027523040771484375 nb_pixel_total : 111830 time to create 1 rle with old method : 0.12457990646362305 length of segment : 554 time for calcul the mask position with numpy : 0.0019085407257080078 nb_pixel_total : 76858 time to create 1 rle with old method : 0.1015310287475586 length of segment : 330 time for calcul the mask position with numpy : 0.0030570030212402344 nb_pixel_total : 146180 time to create 1 rle with old method : 0.16486191749572754 length of segment : 566 time for calcul the mask position with numpy : 0.0008153915405273438 nb_pixel_total : 31012 time to create 1 rle with old method : 0.03697943687438965 length of segment : 192 time for calcul the mask position with numpy : 0.00554347038269043 nb_pixel_total : 222728 time to create 1 rle with new method : 0.012396574020385742 length of segment : 569 time for calcul the mask position with numpy : 0.0012912750244140625 nb_pixel_total : 64318 time to create 1 rle with old method : 0.07420659065246582 length of segment : 279 time for calcul the mask position with numpy : 0.006042003631591797 nb_pixel_total : 280199 time to create 1 rle with new method : 0.014476299285888672 length of segment : 732 time for calcul the mask position with numpy : 0.003988981246948242 nb_pixel_total : 239294 time to create 1 rle with new method : 0.009042739868164062 length of segment : 639 time for calcul the mask position with numpy : 0.0013697147369384766 nb_pixel_total : 62440 time to create 1 rle with old method : 0.06763195991516113 length of segment : 327 time for calcul the mask position with numpy : 0.001573801040649414 nb_pixel_total : 84370 time to create 1 rle with old method : 0.0994713306427002 length of segment : 253 time for calcul the mask position with numpy : 0.0008580684661865234 nb_pixel_total : 39368 time to create 1 rle with old method : 0.045400142669677734 length of segment : 229 time for calcul the mask position with numpy : 0.017877578735351562 nb_pixel_total : 393629 time to create 1 rle with new method : 0.1748034954071045 length of segment : 831 time for calcul the mask position with numpy : 0.0031435489654541016 nb_pixel_total : 116131 time to create 1 rle with old method : 0.14427495002746582 length of segment : 374 time for calcul the mask position with numpy : 0.008321762084960938 nb_pixel_total : 438645 time to create 1 rle with new method : 0.01912522315979004 length of segment : 904 time for calcul the mask position with numpy : 0.0007450580596923828 nb_pixel_total : 30203 time to create 1 rle with old method : 0.033879995346069336 length of segment : 224 time for calcul the mask position with numpy : 0.01187443733215332 nb_pixel_total : 506522 time to create 1 rle with new method : 0.3680078983306885 length of segment : 723 time for calcul the mask position with numpy : 0.0006477832794189453 nb_pixel_total : 16927 time to create 1 rle with old method : 0.04342293739318848 length of segment : 119 time for calcul the mask position with numpy : 0.011044025421142578 nb_pixel_total : 502854 time to create 1 rle with new method : 0.023225784301757812 length of segment : 708 time for calcul the mask position with numpy : 0.0045928955078125 nb_pixel_total : 186609 time to create 1 rle with new method : 0.013574838638305664 length of segment : 1003 time for calcul the mask position with numpy : 0.04259490966796875 nb_pixel_total : 1249519 time to create 1 rle with new method : 0.7636227607727051 length of segment : 2460 time for calcul the mask position with numpy : 0.007230997085571289 nb_pixel_total : 369554 time to create 1 rle with new method : 0.016524314880371094 length of segment : 1018 time for calcul the mask position with numpy : 0.002070903778076172 nb_pixel_total : 79811 time to create 1 rle with old method : 0.09075188636779785 length of segment : 394 time for calcul the mask position with numpy : 0.0010678768157958984 nb_pixel_total : 46652 time to create 1 rle with old method : 0.053209543228149414 length of segment : 346 time for calcul the mask position with numpy : 0.0008335113525390625 nb_pixel_total : 39889 time to create 1 rle with old method : 0.04733729362487793 length of segment : 357 time for calcul the mask position with numpy : 0.004369020462036133 nb_pixel_total : 221967 time to create 1 rle with new method : 0.011609554290771484 length of segment : 718 time for calcul the mask position with numpy : 0.0016450881958007812 nb_pixel_total : 74274 time to create 1 rle with old method : 0.08434104919433594 length of segment : 492 time for calcul the mask position with numpy : 0.018100500106811523 nb_pixel_total : 1019717 time to create 1 rle with new method : 0.23522329330444336 length of segment : 1310 time for calcul the mask position with numpy : 0.0045201778411865234 nb_pixel_total : 199350 time to create 1 rle with new method : 0.012970924377441406 length of segment : 585 time for calcul the mask position with numpy : 0.0009424686431884766 nb_pixel_total : 26315 time to create 1 rle with old method : 0.03035879135131836 length of segment : 285 time for calcul the mask position with numpy : 0.008083105087280273 nb_pixel_total : 426583 time to create 1 rle with new method : 0.054444074630737305 length of segment : 599 time for calcul the mask position with numpy : 0.002069711685180664 nb_pixel_total : 80195 time to create 1 rle with old method : 0.08897590637207031 length of segment : 327 time for calcul the mask position with numpy : 0.008994340896606445 nb_pixel_total : 351337 time to create 1 rle with new method : 0.034326791763305664 length of segment : 2042 time for calcul the mask position with numpy : 0.0005342960357666016 nb_pixel_total : 17372 time to create 1 rle with old method : 0.01998281478881836 length of segment : 166 time for calcul the mask position with numpy : 0.004023313522338867 nb_pixel_total : 218711 time to create 1 rle with new method : 0.011191129684448242 length of segment : 697 time for calcul the mask position with numpy : 0.0026085376739501953 nb_pixel_total : 131014 time to create 1 rle with old method : 0.14744949340820312 length of segment : 407 time for calcul the mask position with numpy : 0.0031943321228027344 nb_pixel_total : 174678 time to create 1 rle with new method : 0.008954286575317383 length of segment : 378 time for calcul the mask position with numpy : 0.0016374588012695312 nb_pixel_total : 68632 time to create 1 rle with old method : 0.10137319564819336 length of segment : 281 time for calcul the mask position with numpy : 0.0012226104736328125 nb_pixel_total : 44800 time to create 1 rle with old method : 0.05275321006774902 length of segment : 307 time for calcul the mask position with numpy : 0.001957416534423828 nb_pixel_total : 103435 time to create 1 rle with old method : 0.11399674415588379 length of segment : 675 time for calcul the mask position with numpy : 0.0017545223236083984 nb_pixel_total : 95237 time to create 1 rle with old method : 0.10646748542785645 length of segment : 583 time for calcul the mask position with numpy : 0.001644134521484375 nb_pixel_total : 79046 time to create 1 rle with old method : 0.09287190437316895 length of segment : 362 time for calcul the mask position with numpy : 0.002912282943725586 nb_pixel_total : 151429 time to create 1 rle with new method : 0.007995128631591797 length of segment : 350 time for calcul the mask position with numpy : 0.002131938934326172 nb_pixel_total : 81922 time to create 1 rle with old method : 0.0925607681274414 length of segment : 353 time for calcul the mask position with numpy : 0.0037033557891845703 nb_pixel_total : 172686 time to create 1 rle with new method : 0.01036834716796875 length of segment : 514 time for calcul the mask position with numpy : 0.016053199768066406 nb_pixel_total : 985535 time to create 1 rle with new method : 0.20885252952575684 length of segment : 1196 time for calcul the mask position with numpy : 0.003141641616821289 nb_pixel_total : 181906 time to create 1 rle with new method : 0.008379697799682617 length of segment : 342 time for calcul the mask position with numpy : 0.002521038055419922 nb_pixel_total : 128214 time to create 1 rle with old method : 0.14481163024902344 length of segment : 405 time for calcul the mask position with numpy : 0.0032188892364501953 nb_pixel_total : 93095 time to create 1 rle with old method : 0.10444068908691406 length of segment : 1386 time for calcul the mask position with numpy : 0.0018277168273925781 nb_pixel_total : 78019 time to create 1 rle with old method : 0.08789610862731934 length of segment : 365 time for calcul the mask position with numpy : 0.0009965896606445312 nb_pixel_total : 36471 time to create 1 rle with old method : 0.04233980178833008 length of segment : 176 time for calcul the mask position with numpy : 0.0024995803833007812 nb_pixel_total : 87900 time to create 1 rle with old method : 0.09816861152648926 length of segment : 565 time for calcul the mask position with numpy : 0.4510371685028076 nb_pixel_total : 9207380 time to create 1 rle with new method : 2.719076156616211 length of segment : 5317 time for calcul the mask position with numpy : 0.2190999984741211 nb_pixel_total : 2484670 time to create 1 rle with new method : 0.5666036605834961 length of segment : 2473 time for calcul the mask position with numpy : 0.026251554489135742 nb_pixel_total : 2247602 time to create 1 rle with new method : 0.19241666793823242 length of segment : 1326 time spent for convertir_results : 44.557913064956665 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 238 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 129.8220295906067 time spend to save output : 0.1682906150817871 total time spend for step 2 : 129.99032020568848 step3:blur_detection Sat May 10 19:44:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (4160, 3120) 1468.1180151174349 (4160, 3120) 1957.636341118384 (4160, 3120) 2419.718827301347 (4160, 3120) 229.45080935190535 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.01139068603515625 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.012027263641357422 save missing photos in datou_result : time spend for datou_step_exec : 2.6648011207580566 time spend to save output : 0.02797389030456543 total time spend for step 3 : 2.692775011062622 step4:brightness Sat May 10 19:44:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746898890_735745_1357512878_594a4ba92166cf72f957cc87e900bdbc.jpg treat image : temp/1746898890_735745_1357512874_36b12aef6d86c8266e944e1c858977c2.jpg treat image : temp/1746898890_735745_1357512854_00ae557d80138ed57e1117ba1390597f.jpg treat image : temp/1746898890_735745_1357512832_73b58f498cc55875edf23222922b9122.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.013811588287353516 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.01189112663269043 save missing photos in datou_result : time spend for datou_step_exec : 6.637698173522949 time spend to save output : 0.030551433563232422 total time spend for step 4 : 6.668249607086182 step5:rle_unique_nms_with_priority Sat May 10 19:44:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 238 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 89 nb_hashtags : 9 time to prepare the origin masks : 73.11820554733276 time for calcul the mask position with numpy : 1.0734212398529053 nb_pixel_total : 4785214 time to create 1 rle with new method : 1.4487504959106445 time for calcul the mask position with numpy : 0.05549120903015137 nb_pixel_total : 190784 time to create 1 rle with new method : 1.2767412662506104 time for calcul the mask position with numpy : 0.05404472351074219 nb_pixel_total : 77884 time to create 1 rle with old method : 0.08907151222229004 time for calcul the mask position with numpy : 0.054131507873535156 nb_pixel_total : 78086 time to create 1 rle with old method : 0.0867164134979248 time for calcul the mask position with numpy : 0.0545651912689209 nb_pixel_total : 105452 time to create 1 rle with old method : 0.11742901802062988 time for calcul the mask position with numpy : 0.1060636043548584 nb_pixel_total : 756407 time to create 1 rle with new method : 1.5400464534759521 time for calcul the mask position with numpy : 0.053620338439941406 nb_pixel_total : 9797 time to create 1 rle with old method : 0.011158466339111328 time for calcul the mask position with numpy : 0.05333709716796875 nb_pixel_total : 6364 time to create 1 rle with old method : 0.010397672653198242 time for calcul the mask position with numpy : 0.06700563430786133 nb_pixel_total : 169621 time to create 1 rle with new method : 1.1581318378448486 time for calcul the mask position with numpy : 0.0574190616607666 nb_pixel_total : 355797 time to create 1 rle with new method : 1.2085814476013184 time for calcul the mask position with numpy : 0.05361509323120117 nb_pixel_total : 342 time to create 1 rle with old method : 0.00042891502380371094 time for calcul the mask position with numpy : 0.05410432815551758 nb_pixel_total : 9403 time to create 1 rle with old method : 0.011300325393676758 time for calcul the mask position with numpy : 0.062020301818847656 nb_pixel_total : 373269 time to create 1 rle with new method : 1.5188860893249512 time for calcul the mask position with numpy : 0.05387711524963379 nb_pixel_total : 49045 time to create 1 rle with old method : 0.056485891342163086 time for calcul the mask position with numpy : 0.0538482666015625 nb_pixel_total : 9677 time to create 1 rle with old method : 0.011028289794921875 time for calcul the mask position with numpy : 0.05382394790649414 nb_pixel_total : 30738 time to create 1 rle with old method : 0.034430742263793945 time for calcul the mask position with numpy : 0.0538787841796875 nb_pixel_total : 50483 time to create 1 rle with old method : 0.05857539176940918 time for calcul the mask position with numpy : 0.053708553314208984 nb_pixel_total : 19887 time to create 1 rle with old method : 0.02250957489013672 time for calcul the mask position with numpy : 0.05416274070739746 nb_pixel_total : 114427 time to create 1 rle with old method : 0.12908196449279785 time for calcul the mask position with numpy : 0.05401945114135742 nb_pixel_total : 115629 time to create 1 rle with old method : 0.13809847831726074 time for calcul the mask position with numpy : 0.0547947883605957 nb_pixel_total : 61020 time to create 1 rle with old method : 0.06801915168762207 time for calcul the mask position with numpy : 0.05352330207824707 nb_pixel_total : 4790 time to create 1 rle with old method : 0.005712032318115234 time for calcul the mask position with numpy : 0.05374908447265625 nb_pixel_total : 61175 time to create 1 rle with old method : 0.06879496574401855 time for calcul the mask position with numpy : 0.05387067794799805 nb_pixel_total : 30734 time to create 1 rle with old method : 0.03466486930847168 time for calcul the mask position with numpy : 0.05373215675354004 nb_pixel_total : 46740 time to create 1 rle with old method : 0.0535883903503418 time for calcul the mask position with numpy : 0.05410027503967285 nb_pixel_total : 145114 time to create 1 rle with old method : 0.16253113746643066 time for calcul the mask position with numpy : 0.054926395416259766 nb_pixel_total : 272820 time to create 1 rle with new method : 0.8756742477416992 time for calcul the mask position with numpy : 0.055512428283691406 nb_pixel_total : 78303 time to create 1 rle with old method : 0.08702588081359863 time for calcul the mask position with numpy : 0.053663015365600586 nb_pixel_total : 20678 time to create 1 rle with old method : 0.023155927658081055 time for calcul the mask position with numpy : 0.05807137489318848 nb_pixel_total : 57756 time to create 1 rle with old method : 0.07661867141723633 time for calcul the mask position with numpy : 0.05603480339050293 nb_pixel_total : 71364 time to create 1 rle with old method : 0.08545613288879395 time for calcul the mask position with numpy : 0.06407356262207031 nb_pixel_total : 49680 time to create 1 rle with old method : 0.06880521774291992 time for calcul the mask position with numpy : 0.052893638610839844 nb_pixel_total : 101756 time to create 1 rle with old method : 0.11321067810058594 time for calcul the mask position with numpy : 0.055269479751586914 nb_pixel_total : 22036 time to create 1 rle with old method : 0.02465081214904785 time for calcul the mask position with numpy : 0.056397438049316406 nb_pixel_total : 463680 time to create 1 rle with new method : 1.210956335067749 time for calcul the mask position with numpy : 0.05379605293273926 nb_pixel_total : 9275 time to create 1 rle with old method : 0.010902166366577148 time for calcul the mask position with numpy : 0.053884029388427734 nb_pixel_total : 35427 time to create 1 rle with old method : 0.03967022895812988 time for calcul the mask position with numpy : 0.054877281188964844 nb_pixel_total : 90141 time to create 1 rle with old method : 0.10209822654724121 time for calcul the mask position with numpy : 0.0526883602142334 nb_pixel_total : 102243 time to create 1 rle with old method : 0.11381816864013672 time for calcul the mask position with numpy : 0.053394317626953125 nb_pixel_total : 237033 time to create 1 rle with new method : 1.1975929737091064 time for calcul the mask position with numpy : 0.051813364028930664 nb_pixel_total : 58191 time to create 1 rle with old method : 0.06183362007141113 time for calcul the mask position with numpy : 0.05086851119995117 nb_pixel_total : 43330 time to create 1 rle with old method : 0.048523902893066406 time for calcul the mask position with numpy : 0.052243709564208984 nb_pixel_total : 277264 time to create 1 rle with new method : 1.249112844467163 time for calcul the mask position with numpy : 0.05408310890197754 nb_pixel_total : 37690 time to create 1 rle with old method : 0.0425412654876709 time for calcul the mask position with numpy : 0.05382537841796875 nb_pixel_total : 126983 time to create 1 rle with old method : 0.1426224708557129 time for calcul the mask position with numpy : 0.05379319190979004 nb_pixel_total : 48890 time to create 1 rle with old method : 0.05654788017272949 time for calcul the mask position with numpy : 0.05456256866455078 nb_pixel_total : 75995 time to create 1 rle with old method : 0.08620572090148926 time for calcul the mask position with numpy : 0.05469822883605957 nb_pixel_total : 31249 time to create 1 rle with old method : 0.0367732048034668 time for calcul the mask position with numpy : 0.06423687934875488 nb_pixel_total : 1884 time to create 1 rle with old method : 0.0025446414947509766 time for calcul the mask position with numpy : 0.05756425857543945 nb_pixel_total : 29431 time to create 1 rle with old method : 0.04953765869140625 time for calcul the mask position with numpy : 0.05454897880554199 nb_pixel_total : 33579 time to create 1 rle with old method : 0.039781808853149414 time for calcul the mask position with numpy : 0.057128190994262695 nb_pixel_total : 14642 time to create 1 rle with old method : 0.0164186954498291 time for calcul the mask position with numpy : 0.05515265464782715 nb_pixel_total : 185897 time to create 1 rle with new method : 1.8263046741485596 time for calcul the mask position with numpy : 0.05823922157287598 nb_pixel_total : 188527 time to create 1 rle with new method : 1.3158700466156006 time for calcul the mask position with numpy : 0.054776906967163086 nb_pixel_total : 134284 time to create 1 rle with old method : 0.15028738975524902 time for calcul the mask position with numpy : 0.053533077239990234 nb_pixel_total : 907 time to create 1 rle with old method : 0.0014233589172363281 time for calcul the mask position with numpy : 0.054467201232910156 nb_pixel_total : 4322 time to create 1 rle with old method : 0.005384683609008789 time for calcul the mask position with numpy : 0.058483123779296875 nb_pixel_total : 203165 time to create 1 rle with new method : 1.4458286762237549 time for calcul the mask position with numpy : 0.05701255798339844 nb_pixel_total : 432562 time to create 1 rle with new method : 1.3938300609588623 time for calcul the mask position with numpy : 0.05353856086730957 nb_pixel_total : 1996 time to create 1 rle with old method : 0.0026128292083740234 time for calcul the mask position with numpy : 0.053391456604003906 nb_pixel_total : 615 time to create 1 rle with old method : 0.0011148452758789062 time for calcul the mask position with numpy : 0.0546727180480957 nb_pixel_total : 131197 time to create 1 rle with old method : 0.14853882789611816 time for calcul the mask position with numpy : 0.05351734161376953 nb_pixel_total : 4284 time to create 1 rle with old method : 0.004948616027832031 time for calcul the mask position with numpy : 0.05545163154602051 nb_pixel_total : 273072 time to create 1 rle with new method : 1.0783910751342773 time for calcul the mask position with numpy : 0.05439877510070801 nb_pixel_total : 1205 time to create 1 rle with old method : 0.0022630691528320312 time for calcul the mask position with numpy : 0.06185293197631836 nb_pixel_total : 78626 time to create 1 rle with old method : 0.09081387519836426 time for calcul the mask position with numpy : 0.05405116081237793 nb_pixel_total : 35740 time to create 1 rle with old method : 0.03999471664428711 time for calcul the mask position with numpy : 0.05411052703857422 nb_pixel_total : 53890 time to create 1 rle with old method : 0.06063365936279297 time for calcul the mask position with numpy : 0.0545954704284668 nb_pixel_total : 87185 time to create 1 rle with old method : 0.12231898307800293 time for calcul the mask position with numpy : 0.053971290588378906 nb_pixel_total : 6251 time to create 1 rle with old method : 0.007534980773925781 time for calcul the mask position with numpy : 0.054087162017822266 nb_pixel_total : 18803 time to create 1 rle with old method : 0.022961854934692383 time for calcul the mask position with numpy : 0.05646848678588867 nb_pixel_total : 640 time to create 1 rle with old method : 0.0009088516235351562 time for calcul the mask position with numpy : 0.056833505630493164 nb_pixel_total : 35275 time to create 1 rle with old method : 0.04067039489746094 time for calcul the mask position with numpy : 0.056905269622802734 nb_pixel_total : 335448 time to create 1 rle with new method : 2.1565823554992676 time for calcul the mask position with numpy : 0.05426478385925293 nb_pixel_total : 68276 time to create 1 rle with old method : 0.0798032283782959 time for calcul the mask position with numpy : 0.05432915687561035 nb_pixel_total : 2070 time to create 1 rle with old method : 0.0026710033416748047 time for calcul the mask position with numpy : 0.05469107627868652 nb_pixel_total : 112406 time to create 1 rle with old method : 0.12603282928466797 time for calcul the mask position with numpy : 0.05429720878601074 nb_pixel_total : 112047 time to create 1 rle with old method : 0.12505578994750977 time for calcul the mask position with numpy : 0.05498933792114258 nb_pixel_total : 17617 time to create 1 rle with old method : 0.021051645278930664 time for calcul the mask position with numpy : 0.0543215274810791 nb_pixel_total : 37660 time to create 1 rle with old method : 0.042044639587402344 time for calcul the mask position with numpy : 0.058850765228271484 nb_pixel_total : 1087 time to create 1 rle with old method : 0.0013682842254638672 time for calcul the mask position with numpy : 0.05446124076843262 nb_pixel_total : 30690 time to create 1 rle with old method : 0.03498435020446777 time for calcul the mask position with numpy : 0.05352282524108887 nb_pixel_total : 11641 time to create 1 rle with old method : 0.01318812370300293 time for calcul the mask position with numpy : 0.05451631546020508 nb_pixel_total : 111136 time to create 1 rle with old method : 0.1281585693359375 time for calcul the mask position with numpy : 0.054779052734375 nb_pixel_total : 70340 time to create 1 rle with old method : 0.07900834083557129 time for calcul the mask position with numpy : 0.0545198917388916 nb_pixel_total : 6517 time to create 1 rle with old method : 0.00760650634765625 time for calcul the mask position with numpy : 0.05621623992919922 nb_pixel_total : 1175 time to create 1 rle with old method : 0.0017385482788085938 time for calcul the mask position with numpy : 0.05384254455566406 nb_pixel_total : 77167 time to create 1 rle with old method : 0.08617234230041504 time for calcul the mask position with numpy : 0.053719520568847656 nb_pixel_total : 56050 time to create 1 rle with old method : 0.06375479698181152 time for calcul the mask position with numpy : 0.05360698699951172 nb_pixel_total : 231 time to create 1 rle with old method : 0.00041937828063964844 create new chi : 32.66166567802429 time to delete rle : 0.038262128829956055 batch 1 Loaded 90 chid ids of type : 4855 Number RLEs to save : 64853 TO DO : save crop sub photo not yet done ! save time : 3.872199773788452 nb_obj : 76 nb_hashtags : 8 time to prepare the origin masks : 69.11169195175171 time for calcul the mask position with numpy : 1.1081347465515137 nb_pixel_total : 4431298 time to create 1 rle with new method : 1.3378124237060547 time for calcul the mask position with numpy : 0.055768489837646484 nb_pixel_total : 91283 time to create 1 rle with old method : 0.09955501556396484 time for calcul the mask position with numpy : 0.054028987884521484 nb_pixel_total : 33751 time to create 1 rle with old method : 0.03900289535522461 time for calcul the mask position with numpy : 0.055166006088256836 nb_pixel_total : 243740 time to create 1 rle with new method : 1.4468719959259033 time for calcul the mask position with numpy : 0.05385923385620117 nb_pixel_total : 46099 time to create 1 rle with old method : 0.052037954330444336 time for calcul the mask position with numpy : 0.06676030158996582 nb_pixel_total : 642400 time to create 1 rle with new method : 1.274064064025879 time for calcul the mask position with numpy : 0.05444765090942383 nb_pixel_total : 154350 time to create 1 rle with new method : 2.115717649459839 time for calcul the mask position with numpy : 0.05518984794616699 nb_pixel_total : 176953 time to create 1 rle with new method : 1.197218656539917 time for calcul the mask position with numpy : 0.05436205863952637 nb_pixel_total : 188684 time to create 1 rle with new method : 1.409660816192627 time for calcul the mask position with numpy : 0.05213522911071777 nb_pixel_total : 7681 time to create 1 rle with old method : 0.008433103561401367 time for calcul the mask position with numpy : 0.053322792053222656 nb_pixel_total : 16352 time to create 1 rle with old method : 0.01809835433959961 time for calcul the mask position with numpy : 0.050959110260009766 nb_pixel_total : 43020 time to create 1 rle with old method : 0.047432661056518555 time for calcul the mask position with numpy : 0.053597450256347656 nb_pixel_total : 1326 time to create 1 rle with old method : 0.0016813278198242188 time for calcul the mask position with numpy : 0.055367469787597656 nb_pixel_total : 26200 time to create 1 rle with old method : 0.030794858932495117 time for calcul the mask position with numpy : 0.05347704887390137 nb_pixel_total : 360440 time to create 1 rle with new method : 1.5034902095794678 time for calcul the mask position with numpy : 0.0534515380859375 nb_pixel_total : 220069 time to create 1 rle with new method : 1.149604320526123 time for calcul the mask position with numpy : 0.05107760429382324 nb_pixel_total : 22949 time to create 1 rle with old method : 0.024443626403808594 time for calcul the mask position with numpy : 0.05161452293395996 nb_pixel_total : 21810 time to create 1 rle with old method : 0.024407148361206055 time for calcul the mask position with numpy : 0.05495810508728027 nb_pixel_total : 197492 time to create 1 rle with new method : 1.3207998275756836 time for calcul the mask position with numpy : 0.05442452430725098 nb_pixel_total : 109011 time to create 1 rle with old method : 0.12323212623596191 time for calcul the mask position with numpy : 0.05556058883666992 nb_pixel_total : 145932 time to create 1 rle with old method : 0.17017340660095215 time for calcul the mask position with numpy : 0.06120109558105469 nb_pixel_total : 26592 time to create 1 rle with old method : 0.04288315773010254 time for calcul the mask position with numpy : 0.05634260177612305 nb_pixel_total : 10910 time to create 1 rle with old method : 0.012475728988647461 time for calcul the mask position with numpy : 0.05785322189331055 nb_pixel_total : 64582 time to create 1 rle with old method : 0.08477425575256348 time for calcul the mask position with numpy : 0.05626940727233887 nb_pixel_total : 58066 time to create 1 rle with old method : 0.06517672538757324 time for calcul the mask position with numpy : 0.05454874038696289 nb_pixel_total : 246540 time to create 1 rle with new method : 1.3441493511199951 time for calcul the mask position with numpy : 0.05307149887084961 nb_pixel_total : 74159 time to create 1 rle with old method : 0.08142638206481934 time for calcul the mask position with numpy : 0.05093050003051758 nb_pixel_total : 48931 time to create 1 rle with old method : 0.052484750747680664 time for calcul the mask position with numpy : 0.051268815994262695 nb_pixel_total : 33341 time to create 1 rle with old method : 0.03648710250854492 time for calcul the mask position with numpy : 0.0516972541809082 nb_pixel_total : 208977 time to create 1 rle with new method : 0.8888163566589355 time for calcul the mask position with numpy : 0.0542447566986084 nb_pixel_total : 99822 time to create 1 rle with old method : 0.10858631134033203 time for calcul the mask position with numpy : 0.05316019058227539 nb_pixel_total : 132667 time to create 1 rle with old method : 0.14391613006591797 time for calcul the mask position with numpy : 0.05273866653442383 nb_pixel_total : 26291 time to create 1 rle with old method : 0.030095577239990234 time for calcul the mask position with numpy : 0.05437326431274414 nb_pixel_total : 88131 time to create 1 rle with old method : 0.11284041404724121 time for calcul the mask position with numpy : 0.06114697456359863 nb_pixel_total : 279869 time to create 1 rle with new method : 1.7883639335632324 time for calcul the mask position with numpy : 0.054717063903808594 nb_pixel_total : 87700 time to create 1 rle with old method : 0.09737539291381836 time for calcul the mask position with numpy : 0.05271458625793457 nb_pixel_total : 51178 time to create 1 rle with old method : 0.06083178520202637 time for calcul the mask position with numpy : 0.05355191230773926 nb_pixel_total : 12094 time to create 1 rle with old method : 0.013775825500488281 time for calcul the mask position with numpy : 0.05346059799194336 nb_pixel_total : 92142 time to create 1 rle with old method : 0.1057119369506836 time for calcul the mask position with numpy : 0.054013729095458984 nb_pixel_total : 165622 time to create 1 rle with new method : 1.3474340438842773 time for calcul the mask position with numpy : 0.053524017333984375 nb_pixel_total : 4047 time to create 1 rle with old method : 0.005001068115234375 time for calcul the mask position with numpy : 0.05518150329589844 nb_pixel_total : 106395 time to create 1 rle with old method : 0.11967229843139648 time for calcul the mask position with numpy : 0.05353713035583496 nb_pixel_total : 21464 time to create 1 rle with old method : 0.024278640747070312 time for calcul the mask position with numpy : 0.056986331939697266 nb_pixel_total : 446591 time to create 1 rle with new method : 1.122708797454834 time for calcul the mask position with numpy : 0.05336332321166992 nb_pixel_total : 41989 time to create 1 rle with old method : 0.047219038009643555 time for calcul the mask position with numpy : 0.05386638641357422 nb_pixel_total : 61948 time to create 1 rle with old method : 0.06972980499267578 time for calcul the mask position with numpy : 0.05384349822998047 nb_pixel_total : 56367 time to create 1 rle with old method : 0.06321573257446289 time for calcul the mask position with numpy : 0.05779743194580078 nb_pixel_total : 332386 time to create 1 rle with new method : 1.3322291374206543 time for calcul the mask position with numpy : 0.05428814888000488 nb_pixel_total : 83536 time to create 1 rle with old method : 0.0942995548248291 time for calcul the mask position with numpy : 0.053633928298950195 nb_pixel_total : 16308 time to create 1 rle with old method : 0.01996016502380371 time for calcul the mask position with numpy : 0.055680274963378906 nb_pixel_total : 403966 time to create 1 rle with new method : 2.1807963848114014 time for calcul the mask position with numpy : 0.053429603576660156 nb_pixel_total : 27858 time to create 1 rle with old method : 0.03217744827270508 time for calcul the mask position with numpy : 0.053783416748046875 nb_pixel_total : 54506 time to create 1 rle with old method : 0.06072258949279785 time for calcul the mask position with numpy : 0.05339407920837402 nb_pixel_total : 8953 time to create 1 rle with old method : 0.010223627090454102 time for calcul the mask position with numpy : 0.053525447845458984 nb_pixel_total : 102989 time to create 1 rle with old method : 0.11547589302062988 time for calcul the mask position with numpy : 0.0543675422668457 nb_pixel_total : 58489 time to create 1 rle with old method : 0.06534123420715332 time for calcul the mask position with numpy : 0.054843902587890625 nb_pixel_total : 240301 time to create 1 rle with new method : 1.1251912117004395 time for calcul the mask position with numpy : 0.05479717254638672 nb_pixel_total : 79835 time to create 1 rle with old method : 0.09469747543334961 time for calcul the mask position with numpy : 0.05616140365600586 nb_pixel_total : 39186 time to create 1 rle with old method : 0.04407215118408203 time for calcul the mask position with numpy : 0.056374311447143555 nb_pixel_total : 120381 time to create 1 rle with old method : 0.15246057510375977 time for calcul the mask position with numpy : 0.05629253387451172 nb_pixel_total : 333325 time to create 1 rle with new method : 1.1811590194702148 time for calcul the mask position with numpy : 0.051851511001586914 nb_pixel_total : 125144 time to create 1 rle with old method : 0.13578104972839355 time for calcul the mask position with numpy : 0.05330300331115723 nb_pixel_total : 339488 time to create 1 rle with new method : 1.1693665981292725 time for calcul the mask position with numpy : 0.05598735809326172 nb_pixel_total : 3338 time to create 1 rle with old method : 0.004228830337524414 time for calcul the mask position with numpy : 0.053467512130737305 nb_pixel_total : 67242 time to create 1 rle with old method : 0.07572245597839355 time for calcul the mask position with numpy : 0.05526614189147949 nb_pixel_total : 277062 time to create 1 rle with new method : 1.4173469543457031 time for calcul the mask position with numpy : 0.05229592323303223 nb_pixel_total : 87306 time to create 1 rle with old method : 0.09868478775024414 time for calcul the mask position with numpy : 0.05378866195678711 nb_pixel_total : 47593 time to create 1 rle with old method : 0.053038835525512695 time for calcul the mask position with numpy : 0.05514025688171387 nb_pixel_total : 161583 time to create 1 rle with new method : 1.27703857421875 time for calcul the mask position with numpy : 0.052480459213256836 nb_pixel_total : 26039 time to create 1 rle with old method : 0.02806401252746582 time for calcul the mask position with numpy : 0.05412101745605469 nb_pixel_total : 71818 time to create 1 rle with old method : 0.07738876342773438 time for calcul the mask position with numpy : 0.051522016525268555 nb_pixel_total : 60747 time to create 1 rle with old method : 0.06989574432373047 time for calcul the mask position with numpy : 0.053275346755981445 nb_pixel_total : 4573 time to create 1 rle with old method : 0.0052073001861572266 time for calcul the mask position with numpy : 0.05282020568847656 nb_pixel_total : 17802 time to create 1 rle with old method : 0.02044510841369629 time for calcul the mask position with numpy : 0.05469250679016113 nb_pixel_total : 39828 time to create 1 rle with old method : 0.04462838172912598 time for calcul the mask position with numpy : 0.0546717643737793 nb_pixel_total : 8798 time to create 1 rle with old method : 0.010077238082885742 time for calcul the mask position with numpy : 0.05280661582946777 nb_pixel_total : 11565 time to create 1 rle with old method : 0.013057470321655273 create new chi : 38.566059827804565 time to delete rle : 0.009412050247192383 batch 1 Loaded 77 chid ids of type : 4855 Number RLEs to save : 68881 TO DO : save crop sub photo not yet done ! save time : 3.776067018508911 nb_obj : 63 nb_hashtags : 8 time to prepare the origin masks : 63.18455505371094 time for calcul the mask position with numpy : 0.4299483299255371 nb_pixel_total : 4159567 time to create 1 rle with new method : 1.7049169540405273 time for calcul the mask position with numpy : 0.05237841606140137 nb_pixel_total : 3343 time to create 1 rle with old method : 0.004275321960449219 time for calcul the mask position with numpy : 0.35663533210754395 nb_pixel_total : 574237 time to create 1 rle with new method : 1.2677855491638184 time for calcul the mask position with numpy : 0.05609560012817383 nb_pixel_total : 384294 time to create 1 rle with new method : 1.7993817329406738 time for calcul the mask position with numpy : 0.0532684326171875 nb_pixel_total : 53085 time to create 1 rle with old method : 0.05917787551879883 time for calcul the mask position with numpy : 0.054398536682128906 nb_pixel_total : 369044 time to create 1 rle with new method : 1.2443506717681885 time for calcul the mask position with numpy : 0.05464005470275879 nb_pixel_total : 313144 time to create 1 rle with new method : 1.453789234161377 time for calcul the mask position with numpy : 0.0518345832824707 nb_pixel_total : 26147 time to create 1 rle with old method : 0.029325246810913086 time for calcul the mask position with numpy : 0.05422353744506836 nb_pixel_total : 79548 time to create 1 rle with old method : 0.08891916275024414 time for calcul the mask position with numpy : 0.05455303192138672 nb_pixel_total : 68468 time to create 1 rle with old method : 0.07635259628295898 time for calcul the mask position with numpy : 0.05351662635803223 nb_pixel_total : 47028 time to create 1 rle with old method : 0.05086517333984375 time for calcul the mask position with numpy : 0.0519256591796875 nb_pixel_total : 2928 time to create 1 rle with old method : 0.0036725997924804688 time for calcul the mask position with numpy : 0.05417180061340332 nb_pixel_total : 90170 time to create 1 rle with old method : 0.1082601547241211 time for calcul the mask position with numpy : 0.06165909767150879 nb_pixel_total : 28063 time to create 1 rle with old method : 0.031945228576660156 time for calcul the mask position with numpy : 0.054314374923706055 nb_pixel_total : 171476 time to create 1 rle with new method : 1.7030043601989746 time for calcul the mask position with numpy : 0.052596092224121094 nb_pixel_total : 53626 time to create 1 rle with old method : 0.061615705490112305 time for calcul the mask position with numpy : 0.05466580390930176 nb_pixel_total : 79163 time to create 1 rle with old method : 0.08830952644348145 time for calcul the mask position with numpy : 0.05456733703613281 nb_pixel_total : 76817 time to create 1 rle with old method : 0.08627533912658691 time for calcul the mask position with numpy : 0.05422091484069824 nb_pixel_total : 84088 time to create 1 rle with old method : 0.09457182884216309 time for calcul the mask position with numpy : 0.05404162406921387 nb_pixel_total : 126877 time to create 1 rle with old method : 0.13959836959838867 time for calcul the mask position with numpy : 0.06063485145568848 nb_pixel_total : 392286 time to create 1 rle with new method : 1.300037145614624 time for calcul the mask position with numpy : 0.05385875701904297 nb_pixel_total : 30738 time to create 1 rle with old method : 0.034253597259521484 time for calcul the mask position with numpy : 0.057195425033569336 nb_pixel_total : 448880 time to create 1 rle with new method : 0.993535041809082 time for calcul the mask position with numpy : 0.051778316497802734 nb_pixel_total : 15688 time to create 1 rle with old method : 0.01645803451538086 time for calcul the mask position with numpy : 0.05045890808105469 nb_pixel_total : 62535 time to create 1 rle with old method : 0.06684398651123047 time for calcul the mask position with numpy : 0.06911230087280273 nb_pixel_total : 1112929 time to create 1 rle with new method : 2.1461150646209717 time for calcul the mask position with numpy : 0.05257248878479004 nb_pixel_total : 30085 time to create 1 rle with old method : 0.03314995765686035 time for calcul the mask position with numpy : 0.054436445236206055 nb_pixel_total : 222458 time to create 1 rle with new method : 1.5766043663024902 time for calcul the mask position with numpy : 0.05364513397216797 nb_pixel_total : 62393 time to create 1 rle with old method : 0.07108736038208008 time for calcul the mask position with numpy : 0.061614990234375 nb_pixel_total : 988932 time to create 1 rle with new method : 2.256415605545044 time for calcul the mask position with numpy : 0.05692577362060547 nb_pixel_total : 469 time to create 1 rle with old method : 0.0007283687591552734 time for calcul the mask position with numpy : 0.0533902645111084 nb_pixel_total : 13173 time to create 1 rle with old method : 0.016037940979003906 time for calcul the mask position with numpy : 0.05675005912780762 nb_pixel_total : 103444 time to create 1 rle with old method : 0.13680768013000488 time for calcul the mask position with numpy : 0.05330085754394531 nb_pixel_total : 30811 time to create 1 rle with old method : 0.03452944755554199 time for calcul the mask position with numpy : 0.05353832244873047 nb_pixel_total : 111622 time to create 1 rle with old method : 0.16073369979858398 time for calcul the mask position with numpy : 0.05299067497253418 nb_pixel_total : 1278 time to create 1 rle with old method : 0.001998424530029297 time for calcul the mask position with numpy : 0.05329632759094238 nb_pixel_total : 21873 time to create 1 rle with old method : 0.024152278900146484 time for calcul the mask position with numpy : 0.052315711975097656 nb_pixel_total : 110368 time to create 1 rle with old method : 0.12270641326904297 time for calcul the mask position with numpy : 0.05184340476989746 nb_pixel_total : 850 time to create 1 rle with old method : 0.0010974407196044922 time for calcul the mask position with numpy : 0.05268549919128418 nb_pixel_total : 11841 time to create 1 rle with old method : 0.013548851013183594 time for calcul the mask position with numpy : 0.05251169204711914 nb_pixel_total : 80715 time to create 1 rle with old method : 0.10779762268066406 time for calcul the mask position with numpy : 0.05853533744812012 nb_pixel_total : 568 time to create 1 rle with old method : 0.000965118408203125 time for calcul the mask position with numpy : 0.052581787109375 nb_pixel_total : 78855 time to create 1 rle with old method : 0.08827447891235352 time for calcul the mask position with numpy : 0.05604219436645508 nb_pixel_total : 279978 time to create 1 rle with new method : 1.6488068103790283 time for calcul the mask position with numpy : 0.05674600601196289 nb_pixel_total : 35577 time to create 1 rle with old method : 0.043358564376831055 time for calcul the mask position with numpy : 0.05396556854248047 nb_pixel_total : 171213 time to create 1 rle with new method : 1.3583455085754395 time for calcul the mask position with numpy : 0.05300402641296387 nb_pixel_total : 17251 time to create 1 rle with old method : 0.019775867462158203 time for calcul the mask position with numpy : 0.05318760871887207 nb_pixel_total : 39860 time to create 1 rle with old method : 0.04284381866455078 time for calcul the mask position with numpy : 0.05336284637451172 nb_pixel_total : 239076 time to create 1 rle with new method : 1.0656931400299072 time for calcul the mask position with numpy : 0.05450034141540527 nb_pixel_total : 221409 time to create 1 rle with new method : 1.764622449874878 time for calcul the mask position with numpy : 0.058145999908447266 nb_pixel_total : 437124 time to create 1 rle with new method : 1.7975513935089111 time for calcul the mask position with numpy : 0.0525813102722168 nb_pixel_total : 2825 time to create 1 rle with old method : 0.0036139488220214844 time for calcul the mask position with numpy : 0.05872654914855957 nb_pixel_total : 9743 time to create 1 rle with old method : 0.010509252548217773 time for calcul the mask position with numpy : 0.0510716438293457 nb_pixel_total : 115808 time to create 1 rle with old method : 0.14003896713256836 time for calcul the mask position with numpy : 0.05585455894470215 nb_pixel_total : 24988 time to create 1 rle with old method : 0.028101444244384766 time for calcul the mask position with numpy : 0.05601310729980469 nb_pixel_total : 143110 time to create 1 rle with old method : 0.15953373908996582 time for calcul the mask position with numpy : 0.0548405647277832 nb_pixel_total : 146137 time to create 1 rle with old method : 0.16444087028503418 time for calcul the mask position with numpy : 0.06410455703735352 nb_pixel_total : 70196 time to create 1 rle with old method : 0.11037659645080566 time for calcul the mask position with numpy : 0.06206870079040527 nb_pixel_total : 4291 time to create 1 rle with old method : 0.005758047103881836 time for calcul the mask position with numpy : 0.05629229545593262 nb_pixel_total : 198612 time to create 1 rle with new method : 1.3479650020599365 time for calcul the mask position with numpy : 0.05821394920349121 nb_pixel_total : 64122 time to create 1 rle with old method : 0.1150059700012207 time for calcul the mask position with numpy : 0.06765127182006836 nb_pixel_total : 54 time to create 1 rle with old method : 0.00014209747314453125 time for calcul the mask position with numpy : 0.057613372802734375 nb_pixel_total : 33368 time to create 1 rle with old method : 0.044951438903808594 time for calcul the mask position with numpy : 0.054723501205444336 nb_pixel_total : 554 time to create 1 rle with old method : 0.0009157657623291016 create new chi : 34.17302370071411 time to delete rle : 0.008693218231201172 batch 1 Loaded 64 chid ids of type : 4855 Number RLEs to save : 53535 TO DO : save crop sub photo not yet done ! save time : 3.0349185466766357 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 26.960664987564087 time for calcul the mask position with numpy : 0.21122479438781738 nb_pixel_total : 1243841 time to create 1 rle with new method : 1.544196605682373 time for calcul the mask position with numpy : 0.06910920143127441 nb_pixel_total : 10242 time to create 1 rle with old method : 0.014109373092651367 time for calcul the mask position with numpy : 0.06798362731933594 nb_pixel_total : 71312 time to create 1 rle with old method : 0.08034300804138184 time for calcul the mask position with numpy : 0.06642508506774902 nb_pixel_total : 37559 time to create 1 rle with old method : 0.04201769828796387 time for calcul the mask position with numpy : 0.0679023265838623 nb_pixel_total : 81477 time to create 1 rle with old method : 0.09229874610900879 time for calcul the mask position with numpy : 0.0629127025604248 nb_pixel_total : 207234 time to create 1 rle with new method : 1.8030588626861572 time for calcul the mask position with numpy : 0.04286599159240723 nb_pixel_total : 3831 time to create 1 rle with old method : 0.005237102508544922 time for calcul the mask position with numpy : 0.054112911224365234 nb_pixel_total : 526239 time to create 1 rle with new method : 1.5357744693756104 time for calcul the mask position with numpy : 0.07470989227294922 nb_pixel_total : 1593041 time to create 1 rle with new method : 1.3346760272979736 time for calcul the mask position with numpy : 0.6642265319824219 nb_pixel_total : 9204424 time to create 1 rle with new method : 0.9112515449523926 create new chi : 9.003145694732666 time to delete rle : 0.0020666122436523438 batch 1 Loaded 10 chid ids of type : 4855 Number RLEs to save : 22792 TO DO : save crop sub photo not yet done ! save time : 1.2754185199737549 map_output_result : {1357512878: (0.5053897049279162, 'Should be the crop_list due to order', 0.4135253577595037), 1357512874: (0.5053897049279162, 'Should be the crop_list due to order', 0.36605188845746556), 1357512854: (0.5053897049279162, 'Should be the crop_list due to order', 0.24198157349469565), 1357512832: (0.5053897049279162, 'Should be the crop_list due to order', 1.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 4 /1357512878.Didn't retrieve data . /1357512874.Didn't retrieve data . /1357512854.Didn't retrieve data . /1357512832.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.012747049331665039 save_final save missing photos in datou_result : time spend for datou_step_exec : 360.77423763275146 time spend to save output : 0.21866679191589355 total time spend for step 5 : 360.99290442466736 step6:crop_condition Sat May 10 19:50:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 4 ! batch 1 Loaded 241 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3015255 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899461_735745 we have uploaded 13 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.205371856689453 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 59 About to insert : list_path_to_insert length 59 new photo from crops ! About to upload 59 photos upload in portfolio : 3015255 init cache_photo without model_param we have 59 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899506_735745 we have uploaded 59 photos in the portfolio 3015255 time of upload the photos Elapsed time : 13.827229976654053 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 27 About to insert : list_path_to_insert length 27 new photo from crops ! About to upload 27 photos upload in portfolio : 3015255 init cache_photo without model_param we have 27 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899528_735745 we have uploaded 27 photos in the portfolio 3015255 time of upload the photos Elapsed time : 6.479675769805908 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3015255 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899538_735745 we have uploaded 4 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.3397023677825928 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899543_735745 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.7797451019287109 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3015255 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899549_735745 we have uploaded 7 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.9933416843414307 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 3015255 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899579_735745 we have uploaded 31 photos in the portfolio 3015255 time of upload the photos Elapsed time : 8.79801321029663 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3015255 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899592_735745 we have uploaded 6 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.754786729812622 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 36 About to insert : list_path_to_insert length 36 new photo from crops ! About to upload 36 photos upload in portfolio : 3015255 init cache_photo without model_param we have 36 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746899708_735745 we have uploaded 36 photos in the portfolio 3015255 time of upload the photos Elapsed time : 12.57039213180542 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 185 /1357519328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519816Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357519849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357520413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521305Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1357521345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 559 time used for this insertion : 0.04641294479370117 save_final save missing photos in datou_result : time spend for datou_step_exec : 279.4532482624054 time spend to save output : 0.05054807662963867 total time spend for step 6 : 279.50379633903503 step7:ventilate_hashtags_in_portfolio Sat May 10 19:55:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22864312 get user id for portfolio 22864312 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22864312 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22864312 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22864312 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22864312 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22864881,22864882,22864883,22864884,22864885,22864890,22864892,22864893,22864894,22864895,22864896,22864897,22864899,22864900,22864901,22864902,22864903,22864904,22864905,22864906?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22864312 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 1 /22864312. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.012911081314086914 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.311418533325195 time spend to save output : 0.013117551803588867 total time spend for step 7 : 4.324536085128784 step8:final Sat May 10 19:55:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1357512878: ('0.3358447727137948',), 1357512874: ('0.3358447727137948',), 1357512854: ('0.3358447727137948',), 1357512832: ('0.3358447727137948',)} new output for save of step final : {1357512878: ('0.3358447727137948',), 1357512874: ('0.3358447727137948',), 1357512854: ('0.3358447727137948',), 1357512832: ('0.3358447727137948',)} [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 4 /1357512878.Didn't retrieve data . /1357512874.Didn't retrieve data . /1357512854.Didn't retrieve data . /1357512832.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.01269841194152832 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10400891304016113 time spend to save output : 0.013042211532592773 total time spend for step 8 : 0.1170511245727539 step9:velours_tree Sat May 10 19:55:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.173999309539795 time spend to save output : 4.00543212890625e-05 total time spend for step 9 : 1.174039363861084 step10:send_mail_cod Sat May 10 19:55:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22864312_10-05-2025_19_55_28.pdf 22864854 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648541746899728 22864855 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648551746899729 22864856 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648561746899730 22864857 change filename to text .change filename to text .change filename to text .change filename to text .imagette228648571746899732 22864858 change filename to text .change filename to text .imagette228648581746899732 22864859 imagette228648591746899732 22864860 imagette228648601746899732 22864861 imagette228648611746899732 22864862 imagette228648621746899732 22864863 imagette228648631746899732 22864864 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648641746899732 22864865 imagette228648651746899733 22864866 imagette228648661746899733 22864867 imagette228648671746899733 22864868 imagette228648681746899733 22864869 imagette228648691746899733 22864870 imagette228648701746899733 22864871 imagette228648711746899733 22864872 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648721746899733 22864874 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette228648741746899734 22864875 imagette228648751746899735 22864876 imagette228648761746899735 22864877 imagette228648771746899735 22864879 imagette228648791746899735 22864880 imagette228648801746899735 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22864312 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22864881,22864882,22864883,22864884,22864885,22864890,22864892,22864893,22864894,22864895,22864896,22864897,22864899,22864900,22864901,22864902,22864903,22864904,22864905,22864906?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22864312 your option no_mail is active, we will not send the real mail to your client args[1357512878] : ((1357512878, 1468.1180151174349, 2107751945), (1357512878, -0.4004769615597169, 496442774), '0.3358447727137948') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1357512874] : ((1357512874, 1957.636341118384, 2107751945), (1357512874, -0.2880904576892666, 496442774), '0.3358447727137948') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1357512854] : ((1357512854, 2419.718827301347, 492609224), (1357512854, -0.2056867218017021, 496442774), '0.3358447727137948') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1357512832] : ((1357512832, 229.45080935190535, 492688767), (1357512832, -0.7757684524136305, 501862349), '0.3358447727137948') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.3358447727137948 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22864312_10-05-2025_19_55_28.pdf results_Auto_P22864312_10-05-2025_19_55_28.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22864312_10-05-2025_19_55_28.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22864312','results_Auto_P22864312_10-05-2025_19_55_28.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22864312_10-05-2025_19_55_28.pdf','pdf','','2.01','0.3358447727137948') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.013350248336791992 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.455276250839233 time spend to save output : 0.013652324676513672 total time spend for step 10 : 10.468928575515747 step11:split_time_score Sat May 10 19:55:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('42', 1), ('43', 3)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10052025 22864312 Nombre de photos uploadées : 4 / 23040 (0%) 10052025 22864312 Nombre de photos taguées (types de déchets): 0 / 4 (0%) 10052025 22864312 Nombre de photos taguées (volume) : 0 / 4 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???? elapsed_time : fill_and_build_computed_from_old_data 0.0002694129943847656 elapsed_time : insert_dashboard_record_day_entry 0.02316570281982422 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.2280055167718577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22862238_10-05-2025_15_49_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22862238 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22862238 AND mptpi.`type`=4855 To do Qualite : 0.39773799115774777 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22862239_10-05-2025_16_04_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22862239 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22862239 AND mptpi.`type`=4855 To do Qualite : 0.3358447727137948 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22864312_10-05-2025_19_55_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22864312 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22864312 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10052025': {'nb_upload': 4, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1357512878, 1357512874, 1357512854, 1357512832] Looping around the photos to save general results len do output : 1 /22864312Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512878', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512874', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512854', None, None, None, None, None, '2818118') ('4741', None, None, None, None, None, None, None, '2818118') ('4741', '22864312', '1357512832', None, None, None, None, None, '2818118') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.012445211410522461 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3764829635620117 time spend to save output : 0.01262044906616211 total time spend for step 11 : 0.38910341262817383 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 4 set_done_treatment 501.46user 300.15system 14:13.44elapsed 93%CPU (0avgtext+0avgdata 5535812maxresident)k 3254960inputs+565928outputs (31570major+41450816minor)pagefaults 0swaps