python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2145787 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2799189'] with mtr_portfolio_ids : ['22731304'] and first list_photo_ids : [] new path : /proc/2145787/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 4 ; length of list_pids : 4 ; length of list_args : 4 time to download the photos : 0.8469288349151611 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue May 6 21:41:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 21:41:30.531474: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 21:41:30.563300: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 21:41:30.565794: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fb868000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 21:41:30.565854: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 21:41:30.570669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 21:41:30.826542: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x676c140 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 21:41:30.826595: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 21:41:30.828046: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:41:30.828524: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:41:30.831917: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:41:30.835064: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:41:30.835606: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:41:30.838811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:41:30.840224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:41:30.846070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:41:30.847586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:41:30.847666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:41:30.848444: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 21:41:30.848458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 21:41:30.848467: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 21:41:30.849765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 21:41:31.130354: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:41:31.130431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:41:31.130452: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:41:31.130471: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:41:31.130489: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:41:31.130506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:41:31.130523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:41:31.130543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:41:31.132167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:41:31.133427: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:41:31.133466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:41:31.133485: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:41:31.133502: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:41:31.133530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:41:31.133548: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:41:31.133565: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:41:31.133583: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:41:31.135188: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:41:31.135221: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 21:41:31.135232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 21:41:31.135241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 21:41:31.136823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 21:41:42.755903: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:41:42.939571: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 2146010 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.00435638427734375 nb_pixel_total : 111415 time to create 1 rle with old method : 0.1300950050354004 length of segment : 557 time for calcul the mask position with numpy : 0.001867532730102539 nb_pixel_total : 112735 time to create 1 rle with old method : 0.13125276565551758 length of segment : 506 time for calcul the mask position with numpy : 0.0007674694061279297 nb_pixel_total : 47671 time to create 1 rle with old method : 0.050299882888793945 length of segment : 286 time for calcul the mask position with numpy : 0.0007112026214599609 nb_pixel_total : 52861 time to create 1 rle with old method : 0.05650758743286133 length of segment : 220 time for calcul the mask position with numpy : 0.0010237693786621094 nb_pixel_total : 49915 time to create 1 rle with old method : 0.05429649353027344 length of segment : 310 time for calcul the mask position with numpy : 0.00046253204345703125 nb_pixel_total : 23249 time to create 1 rle with old method : 0.025764942169189453 length of segment : 209 time for calcul the mask position with numpy : 0.00014710426330566406 nb_pixel_total : 5673 time to create 1 rle with old method : 0.0064640045166015625 length of segment : 99 time for calcul the mask position with numpy : 0.0011928081512451172 nb_pixel_total : 66381 time to create 1 rle with old method : 0.06961774826049805 length of segment : 343 time for calcul the mask position with numpy : 0.0015685558319091797 nb_pixel_total : 83906 time to create 1 rle with old method : 0.08653664588928223 length of segment : 387 time for calcul the mask position with numpy : 0.0003044605255126953 nb_pixel_total : 15989 time to create 1 rle with old method : 0.017949581146240234 length of segment : 139 time for calcul the mask position with numpy : 0.0007112026214599609 nb_pixel_total : 50882 time to create 1 rle with old method : 0.053324222564697266 length of segment : 234 time for calcul the mask position with numpy : 0.0003437995910644531 nb_pixel_total : 16510 time to create 1 rle with old method : 0.017833709716796875 length of segment : 144 time for calcul the mask position with numpy : 0.0004124641418457031 nb_pixel_total : 26498 time to create 1 rle with old method : 0.02905893325805664 length of segment : 211 time for calcul the mask position with numpy : 0.0004220008850097656 nb_pixel_total : 20787 time to create 1 rle with old method : 0.022371292114257812 length of segment : 206 time for calcul the mask position with numpy : 0.00042700767517089844 nb_pixel_total : 28072 time to create 1 rle with old method : 0.030354022979736328 length of segment : 185 time for calcul the mask position with numpy : 0.0010313987731933594 nb_pixel_total : 65598 time to create 1 rle with old method : 0.07202625274658203 length of segment : 284 time for calcul the mask position with numpy : 0.0003514289855957031 nb_pixel_total : 21566 time to create 1 rle with old method : 0.023021697998046875 length of segment : 149 time for calcul the mask position with numpy : 0.0003304481506347656 nb_pixel_total : 22924 time to create 1 rle with old method : 0.024550199508666992 length of segment : 180 time for calcul the mask position with numpy : 0.00021195411682128906 nb_pixel_total : 11008 time to create 1 rle with old method : 0.012634515762329102 length of segment : 161 time for calcul the mask position with numpy : 0.00030541419982910156 nb_pixel_total : 17221 time to create 1 rle with old method : 0.018740415573120117 length of segment : 135 time for calcul the mask position with numpy : 0.0009870529174804688 nb_pixel_total : 69381 time to create 1 rle with old method : 0.0773932933807373 length of segment : 260 time for calcul the mask position with numpy : 0.00047969818115234375 nb_pixel_total : 12493 time to create 1 rle with old method : 0.014408588409423828 length of segment : 137 time for calcul the mask position with numpy : 0.0004024505615234375 nb_pixel_total : 27889 time to create 1 rle with old method : 0.030338525772094727 length of segment : 199 time for calcul the mask position with numpy : 0.0038034915924072266 nb_pixel_total : 173330 time to create 1 rle with new method : 0.014203071594238281 length of segment : 558 time for calcul the mask position with numpy : 0.0008792877197265625 nb_pixel_total : 54224 time to create 1 rle with old method : 0.05760335922241211 length of segment : 289 time for calcul the mask position with numpy : 0.00030422210693359375 nb_pixel_total : 19622 time to create 1 rle with old method : 0.02065443992614746 length of segment : 149 time for calcul the mask position with numpy : 0.21423697471618652 nb_pixel_total : 940449 time to create 1 rle with new method : 0.10426545143127441 length of segment : 795 time for calcul the mask position with numpy : 0.001956939697265625 nb_pixel_total : 82492 time to create 1 rle with old method : 0.0947411060333252 length of segment : 380 time for calcul the mask position with numpy : 0.0004870891571044922 nb_pixel_total : 25708 time to create 1 rle with old method : 0.03504610061645508 length of segment : 147 time for calcul the mask position with numpy : 0.0014426708221435547 nb_pixel_total : 47026 time to create 1 rle with old method : 0.060207366943359375 length of segment : 320 time for calcul the mask position with numpy : 0.0003223419189453125 nb_pixel_total : 12146 time to create 1 rle with old method : 0.015237569808959961 length of segment : 111 time for calcul the mask position with numpy : 0.0015804767608642578 nb_pixel_total : 61302 time to create 1 rle with old method : 0.07691597938537598 length of segment : 310 time for calcul the mask position with numpy : 0.0005922317504882812 nb_pixel_total : 22149 time to create 1 rle with old method : 0.025542259216308594 length of segment : 156 time for calcul the mask position with numpy : 0.0019795894622802734 nb_pixel_total : 71693 time to create 1 rle with old method : 0.08208346366882324 length of segment : 450 time for calcul the mask position with numpy : 0.0006146430969238281 nb_pixel_total : 26727 time to create 1 rle with old method : 0.030695676803588867 length of segment : 186 time for calcul the mask position with numpy : 0.0012598037719726562 nb_pixel_total : 72034 time to create 1 rle with old method : 0.0914156436920166 length of segment : 454 time for calcul the mask position with numpy : 0.06791400909423828 nb_pixel_total : 1305305 time to create 1 rle with new method : 0.3460710048675537 length of segment : 1664 time for calcul the mask position with numpy : 0.11257171630859375 nb_pixel_total : 2563923 time to create 1 rle with new method : 0.6165525913238525 length of segment : 2846 time for calcul the mask position with numpy : 0.15367698669433594 nb_pixel_total : 2919054 time to create 1 rle with new method : 0.3786814212799072 length of segment : 1823 time spent for convertir_results : 8.889822244644165 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 39 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 41.476078033447266 time spend to save output : 0.0867772102355957 total time spend for step 1 : 41.56285524368286 step2:mask_detect Tue May 6 21:42:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 21:42:12.052369: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 21:42:12.083248: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 21:42:12.084919: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fb870000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 21:42:12.084942: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 21:42:12.087905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 21:42:12.317725: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x75af190 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 21:42:12.317781: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 21:42:12.318589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:42:12.319026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:42:12.324844: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:42:12.327465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:42:12.327891: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:42:12.331090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:42:12.332743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:42:12.338453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:42:12.340100: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:42:12.340203: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:42:12.341020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 21:42:12.341041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 21:42:12.341053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 21:42:12.342525: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 21:42:12.463153: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:42:12.463311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:42:12.463338: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:42:12.463362: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:42:12.463385: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:42:12.463410: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:42:12.463434: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:42:12.463459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:42:12.465078: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:42:12.466572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 21:42:12.466636: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 21:42:12.466663: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:42:12.466688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 21:42:12.466711: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 21:42:12.466739: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 21:42:12.466763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 21:42:12.466801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 21:42:12.468410: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 21:42:12.468454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 21:42:12.468465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 21:42:12.468476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 21:42:12.470040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 21:42:23.402550: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 21:42:23.579759: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 56 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 54 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 58 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 2147840 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.005008220672607422 nb_pixel_total : 86757 time to create 1 rle with old method : 0.09874176979064941 length of segment : 443 time for calcul the mask position with numpy : 0.004327535629272461 nb_pixel_total : 179622 time to create 1 rle with new method : 0.012686014175415039 length of segment : 472 time for calcul the mask position with numpy : 0.0005316734313964844 nb_pixel_total : 16727 time to create 1 rle with old method : 0.03754854202270508 length of segment : 121 time for calcul the mask position with numpy : 0.016791582107543945 nb_pixel_total : 715020 time to create 1 rle with new method : 0.09880614280700684 length of segment : 1082 time for calcul the mask position with numpy : 0.0037004947662353516 nb_pixel_total : 202929 time to create 1 rle with new method : 0.009843826293945312 length of segment : 738 time for calcul the mask position with numpy : 0.0057294368743896484 nb_pixel_total : 302662 time to create 1 rle with new method : 0.016064882278442383 length of segment : 648 time for calcul the mask position with numpy : 0.002828836441040039 nb_pixel_total : 126578 time to create 1 rle with old method : 0.13892245292663574 length of segment : 347 time for calcul the mask position with numpy : 0.0022072792053222656 nb_pixel_total : 103923 time to create 1 rle with old method : 0.11322689056396484 length of segment : 364 time for calcul the mask position with numpy : 0.0009250640869140625 nb_pixel_total : 37378 time to create 1 rle with old method : 0.04529929161071777 length of segment : 244 time for calcul the mask position with numpy : 0.0040740966796875 nb_pixel_total : 176107 time to create 1 rle with new method : 0.011161327362060547 length of segment : 476 time for calcul the mask position with numpy : 0.0006022453308105469 nb_pixel_total : 32588 time to create 1 rle with old method : 0.03515052795410156 length of segment : 181 time for calcul the mask position with numpy : 0.006099700927734375 nb_pixel_total : 298824 time to create 1 rle with new method : 0.02287888526916504 length of segment : 442 time for calcul the mask position with numpy : 0.0038003921508789062 nb_pixel_total : 231452 time to create 1 rle with new method : 0.009521484375 length of segment : 788 time for calcul the mask position with numpy : 0.0007941722869873047 nb_pixel_total : 31113 time to create 1 rle with old method : 0.03628182411193848 length of segment : 188 time for calcul the mask position with numpy : 0.0012350082397460938 nb_pixel_total : 53334 time to create 1 rle with old method : 0.06044578552246094 length of segment : 275 time for calcul the mask position with numpy : 0.0011205673217773438 nb_pixel_total : 58601 time to create 1 rle with old method : 0.06474947929382324 length of segment : 234 time for calcul the mask position with numpy : 0.0012776851654052734 nb_pixel_total : 83882 time to create 1 rle with old method : 0.09220170974731445 length of segment : 334 time for calcul the mask position with numpy : 0.00047135353088378906 nb_pixel_total : 23364 time to create 1 rle with old method : 0.02636432647705078 length of segment : 142 time for calcul the mask position with numpy : 0.006073474884033203 nb_pixel_total : 315746 time to create 1 rle with new method : 0.0156252384185791 length of segment : 886 time for calcul the mask position with numpy : 0.0012383460998535156 nb_pixel_total : 58693 time to create 1 rle with old method : 0.0848093032836914 length of segment : 283 time for calcul the mask position with numpy : 0.006909370422363281 nb_pixel_total : 329607 time to create 1 rle with new method : 0.013671636581420898 length of segment : 568 time for calcul the mask position with numpy : 0.0051479339599609375 nb_pixel_total : 254484 time to create 1 rle with new method : 0.013753890991210938 length of segment : 756 time for calcul the mask position with numpy : 0.0005023479461669922 nb_pixel_total : 21133 time to create 1 rle with old method : 0.026110410690307617 length of segment : 183 time for calcul the mask position with numpy : 0.019946813583374023 nb_pixel_total : 1071803 time to create 1 rle with new method : 0.08279275894165039 length of segment : 885 time for calcul the mask position with numpy : 0.0016109943389892578 nb_pixel_total : 88187 time to create 1 rle with old method : 0.12270569801330566 length of segment : 247 time for calcul the mask position with numpy : 0.001226663589477539 nb_pixel_total : 53035 time to create 1 rle with old method : 0.06095409393310547 length of segment : 376 time for calcul the mask position with numpy : 0.0016329288482666016 nb_pixel_total : 74152 time to create 1 rle with old method : 0.09407687187194824 length of segment : 306 time for calcul the mask position with numpy : 0.0005953311920166016 nb_pixel_total : 20203 time to create 1 rle with old method : 0.02354907989501953 length of segment : 194 time for calcul the mask position with numpy : 0.0007646083831787109 nb_pixel_total : 33662 time to create 1 rle with old method : 0.0382082462310791 length of segment : 228 time for calcul the mask position with numpy : 0.0023975372314453125 nb_pixel_total : 122715 time to create 1 rle with old method : 0.1355142593383789 length of segment : 421 time for calcul the mask position with numpy : 0.002140045166015625 nb_pixel_total : 121672 time to create 1 rle with old method : 0.13605737686157227 length of segment : 422 time for calcul the mask position with numpy : 0.0009355545043945312 nb_pixel_total : 29235 time to create 1 rle with old method : 0.03285360336303711 length of segment : 234 time for calcul the mask position with numpy : 0.009848833084106445 nb_pixel_total : 326897 time to create 1 rle with new method : 0.17253947257995605 length of segment : 979 time for calcul the mask position with numpy : 0.008814573287963867 nb_pixel_total : 262086 time to create 1 rle with new method : 0.03637552261352539 length of segment : 1082 time for calcul the mask position with numpy : 0.0004570484161376953 nb_pixel_total : 17069 time to create 1 rle with old method : 0.018260955810546875 length of segment : 134 time for calcul the mask position with numpy : 0.0019314289093017578 nb_pixel_total : 118829 time to create 1 rle with old method : 0.12598681449890137 length of segment : 568 time for calcul the mask position with numpy : 0.002535104751586914 nb_pixel_total : 116128 time to create 1 rle with old method : 0.13705873489379883 length of segment : 502 time for calcul the mask position with numpy : 0.0014231204986572266 nb_pixel_total : 65307 time to create 1 rle with old method : 0.0754544734954834 length of segment : 225 time for calcul the mask position with numpy : 0.0008623600006103516 nb_pixel_total : 38748 time to create 1 rle with old method : 0.0433504581451416 length of segment : 259 time for calcul the mask position with numpy : 0.004192829132080078 nb_pixel_total : 222902 time to create 1 rle with new method : 0.011850595474243164 length of segment : 533 time for calcul the mask position with numpy : 0.0007245540618896484 nb_pixel_total : 31101 time to create 1 rle with old method : 0.03581500053405762 length of segment : 186 time for calcul the mask position with numpy : 0.0006268024444580078 nb_pixel_total : 26064 time to create 1 rle with old method : 0.02947854995727539 length of segment : 196 time for calcul the mask position with numpy : 0.003567934036254883 nb_pixel_total : 186767 time to create 1 rle with new method : 0.010677576065063477 length of segment : 821 time for calcul the mask position with numpy : 0.012457132339477539 nb_pixel_total : 586195 time to create 1 rle with new method : 0.047232627868652344 length of segment : 846 time for calcul the mask position with numpy : 0.0022668838500976562 nb_pixel_total : 92792 time to create 1 rle with old method : 0.10410594940185547 length of segment : 383 time for calcul the mask position with numpy : 0.0005400180816650391 nb_pixel_total : 18862 time to create 1 rle with old method : 0.021266460418701172 length of segment : 170 time for calcul the mask position with numpy : 0.0018498897552490234 nb_pixel_total : 91802 time to create 1 rle with old method : 0.10607647895812988 length of segment : 300 time for calcul the mask position with numpy : 0.0037021636962890625 nb_pixel_total : 163397 time to create 1 rle with new method : 0.010949134826660156 length of segment : 492 time for calcul the mask position with numpy : 0.0023941993713378906 nb_pixel_total : 95093 time to create 1 rle with old method : 0.10402584075927734 length of segment : 576 time for calcul the mask position with numpy : 0.0015721321105957031 nb_pixel_total : 82491 time to create 1 rle with old method : 0.08934402465820312 length of segment : 263 time for calcul the mask position with numpy : 0.010550975799560547 nb_pixel_total : 500523 time to create 1 rle with new method : 0.16186285018920898 length of segment : 920 time for calcul the mask position with numpy : 0.00475764274597168 nb_pixel_total : 237109 time to create 1 rle with new method : 0.019786834716796875 length of segment : 654 time for calcul the mask position with numpy : 0.0015876293182373047 nb_pixel_total : 93312 time to create 1 rle with old method : 0.1301136016845703 length of segment : 297 time for calcul the mask position with numpy : 0.004846334457397461 nb_pixel_total : 260459 time to create 1 rle with new method : 0.01593494415283203 length of segment : 899 time for calcul the mask position with numpy : 0.014085531234741211 nb_pixel_total : 1010326 time to create 1 rle with new method : 0.06525659561157227 length of segment : 902 time for calcul the mask position with numpy : 0.0051915645599365234 nb_pixel_total : 241598 time to create 1 rle with new method : 0.020931005477905273 length of segment : 651 time for calcul the mask position with numpy : 0.0008456707000732422 nb_pixel_total : 37903 time to create 1 rle with old method : 0.043129920959472656 length of segment : 224 time for calcul the mask position with numpy : 0.002187013626098633 nb_pixel_total : 111672 time to create 1 rle with old method : 0.12619614601135254 length of segment : 389 time for calcul the mask position with numpy : 0.0009069442749023438 nb_pixel_total : 49407 time to create 1 rle with old method : 0.0559232234954834 length of segment : 244 time for calcul the mask position with numpy : 0.002640247344970703 nb_pixel_total : 100169 time to create 1 rle with old method : 0.11298799514770508 length of segment : 384 time for calcul the mask position with numpy : 0.0020470619201660156 nb_pixel_total : 83391 time to create 1 rle with old method : 0.09266185760498047 length of segment : 414 time for calcul the mask position with numpy : 0.0010652542114257812 nb_pixel_total : 39701 time to create 1 rle with old method : 0.060625314712524414 length of segment : 322 time for calcul the mask position with numpy : 0.007428407669067383 nb_pixel_total : 347197 time to create 1 rle with new method : 0.03958272933959961 length of segment : 846 time for calcul the mask position with numpy : 0.0013937950134277344 nb_pixel_total : 49331 time to create 1 rle with old method : 0.05376863479614258 length of segment : 232 time for calcul the mask position with numpy : 0.0017986297607421875 nb_pixel_total : 70347 time to create 1 rle with old method : 0.104766845703125 length of segment : 416 time for calcul the mask position with numpy : 0.0058596134185791016 nb_pixel_total : 381565 time to create 1 rle with new method : 0.015633821487426758 length of segment : 1332 time for calcul the mask position with numpy : 0.010523080825805664 nb_pixel_total : 501998 time to create 1 rle with new method : 0.14298081398010254 length of segment : 757 time for calcul the mask position with numpy : 0.0002582073211669922 nb_pixel_total : 11762 time to create 1 rle with old method : 0.013384580612182617 length of segment : 70 time for calcul the mask position with numpy : 0.0008521080017089844 nb_pixel_total : 35330 time to create 1 rle with old method : 0.039595842361450195 length of segment : 198 time for calcul the mask position with numpy : 0.0019130706787109375 nb_pixel_total : 101978 time to create 1 rle with old method : 0.11353325843811035 length of segment : 320 time for calcul the mask position with numpy : 0.001775503158569336 nb_pixel_total : 69993 time to create 1 rle with old method : 0.08504629135131836 length of segment : 399 time for calcul the mask position with numpy : 0.0015997886657714844 nb_pixel_total : 62004 time to create 1 rle with old method : 0.06828641891479492 length of segment : 357 time for calcul the mask position with numpy : 0.002004861831665039 nb_pixel_total : 114216 time to create 1 rle with old method : 0.12554049491882324 length of segment : 513 time for calcul the mask position with numpy : 0.0009827613830566406 nb_pixel_total : 52506 time to create 1 rle with old method : 0.057793617248535156 length of segment : 240 time for calcul the mask position with numpy : 0.004265308380126953 nb_pixel_total : 238927 time to create 1 rle with new method : 0.013046741485595703 length of segment : 629 time for calcul the mask position with numpy : 0.16265106201171875 nb_pixel_total : 1765731 time to create 1 rle with new method : 0.24022817611694336 length of segment : 2343 time for calcul the mask position with numpy : 0.001294851303100586 nb_pixel_total : 60037 time to create 1 rle with old method : 0.06862568855285645 length of segment : 275 time for calcul the mask position with numpy : 0.008432865142822266 nb_pixel_total : 400030 time to create 1 rle with new method : 0.028615474700927734 length of segment : 1761 time for calcul the mask position with numpy : 0.0017054080963134766 nb_pixel_total : 83513 time to create 1 rle with old method : 0.09053587913513184 length of segment : 237 time for calcul the mask position with numpy : 0.025041580200195312 nb_pixel_total : 1199624 time to create 1 rle with new method : 0.20877766609191895 length of segment : 1619 time for calcul the mask position with numpy : 0.0038695335388183594 nb_pixel_total : 271241 time to create 1 rle with new method : 0.011016368865966797 length of segment : 974 time for calcul the mask position with numpy : 0.003180980682373047 nb_pixel_total : 125238 time to create 1 rle with old method : 0.13482666015625 length of segment : 657 time for calcul the mask position with numpy : 0.0034644603729248047 nb_pixel_total : 233276 time to create 1 rle with new method : 0.00888967514038086 length of segment : 531 time for calcul the mask position with numpy : 0.0035505294799804688 nb_pixel_total : 150793 time to create 1 rle with new method : 0.00994110107421875 length of segment : 437 time for calcul the mask position with numpy : 0.03457450866699219 nb_pixel_total : 1748134 time to create 1 rle with new method : 0.30571794509887695 length of segment : 2663 time for calcul the mask position with numpy : 0.005883216857910156 nb_pixel_total : 256256 time to create 1 rle with new method : 0.01762557029724121 length of segment : 567 time for calcul the mask position with numpy : 0.003412485122680664 nb_pixel_total : 146078 time to create 1 rle with old method : 0.16429829597473145 length of segment : 651 time for calcul the mask position with numpy : 0.0007359981536865234 nb_pixel_total : 31720 time to create 1 rle with old method : 0.034377098083496094 length of segment : 150 time for calcul the mask position with numpy : 0.02208733558654785 nb_pixel_total : 900283 time to create 1 rle with new method : 0.16565370559692383 length of segment : 1663 time for calcul the mask position with numpy : 0.00442957878112793 nb_pixel_total : 171459 time to create 1 rle with new method : 0.012927055358886719 length of segment : 451 time for calcul the mask position with numpy : 0.0005025863647460938 nb_pixel_total : 24677 time to create 1 rle with old method : 0.02781534194946289 length of segment : 204 time for calcul the mask position with numpy : 0.0007162094116210938 nb_pixel_total : 33676 time to create 1 rle with old method : 0.03741097450256348 length of segment : 162 time for calcul the mask position with numpy : 0.0005741119384765625 nb_pixel_total : 28230 time to create 1 rle with old method : 0.0320436954498291 length of segment : 186 time for calcul the mask position with numpy : 0.001544952392578125 nb_pixel_total : 71270 time to create 1 rle with old method : 0.08377385139465332 length of segment : 303 time for calcul the mask position with numpy : 0.00051116943359375 nb_pixel_total : 20007 time to create 1 rle with old method : 0.02436351776123047 length of segment : 171 time for calcul the mask position with numpy : 0.007180213928222656 nb_pixel_total : 364076 time to create 1 rle with new method : 0.021542787551879883 length of segment : 1222 time for calcul the mask position with numpy : 0.004095315933227539 nb_pixel_total : 230375 time to create 1 rle with new method : 0.011145830154418945 length of segment : 975 time for calcul the mask position with numpy : 0.003831148147583008 nb_pixel_total : 162226 time to create 1 rle with new method : 0.010661125183105469 length of segment : 456 time for calcul the mask position with numpy : 0.0047609806060791016 nb_pixel_total : 239180 time to create 1 rle with new method : 0.013681650161743164 length of segment : 626 time for calcul the mask position with numpy : 0.0010368824005126953 nb_pixel_total : 50449 time to create 1 rle with old method : 0.05502796173095703 length of segment : 264 time for calcul the mask position with numpy : 0.0008533000946044922 nb_pixel_total : 14606 time to create 1 rle with old method : 0.01691412925720215 length of segment : 211 time for calcul the mask position with numpy : 0.002555370330810547 nb_pixel_total : 148728 time to create 1 rle with old method : 0.16252708435058594 length of segment : 414 time for calcul the mask position with numpy : 0.0017240047454833984 nb_pixel_total : 75220 time to create 1 rle with old method : 0.08164596557617188 length of segment : 638 time for calcul the mask position with numpy : 0.0009462833404541016 nb_pixel_total : 56110 time to create 1 rle with old method : 0.06139349937438965 length of segment : 261 time for calcul the mask position with numpy : 0.0036487579345703125 nb_pixel_total : 143984 time to create 1 rle with old method : 0.16199755668640137 length of segment : 410 time for calcul the mask position with numpy : 0.0012476444244384766 nb_pixel_total : 64868 time to create 1 rle with old method : 0.07034921646118164 length of segment : 315 time for calcul the mask position with numpy : 0.0005371570587158203 nb_pixel_total : 20143 time to create 1 rle with old method : 0.02325916290283203 length of segment : 214 time for calcul the mask position with numpy : 0.0016934871673583984 nb_pixel_total : 93973 time to create 1 rle with old method : 0.09902358055114746 length of segment : 494 time for calcul the mask position with numpy : 0.0007495880126953125 nb_pixel_total : 46838 time to create 1 rle with old method : 0.05394625663757324 length of segment : 226 time for calcul the mask position with numpy : 0.006781339645385742 nb_pixel_total : 378948 time to create 1 rle with new method : 0.019837617874145508 length of segment : 1199 time for calcul the mask position with numpy : 0.0006670951843261719 nb_pixel_total : 30072 time to create 1 rle with old method : 0.04723668098449707 length of segment : 372 time for calcul the mask position with numpy : 0.002771615982055664 nb_pixel_total : 151938 time to create 1 rle with new method : 0.007184505462646484 length of segment : 458 time for calcul the mask position with numpy : 0.0007088184356689453 nb_pixel_total : 30467 time to create 1 rle with old method : 0.032343149185180664 length of segment : 237 time for calcul the mask position with numpy : 0.0032579898834228516 nb_pixel_total : 179386 time to create 1 rle with new method : 0.008283138275146484 length of segment : 928 time for calcul the mask position with numpy : 0.0018486976623535156 nb_pixel_total : 73322 time to create 1 rle with old method : 0.08002543449401855 length of segment : 577 time for calcul the mask position with numpy : 0.0007698535919189453 nb_pixel_total : 40472 time to create 1 rle with old method : 0.04503893852233887 length of segment : 165 time for calcul the mask position with numpy : 0.002002716064453125 nb_pixel_total : 75485 time to create 1 rle with old method : 0.08474588394165039 length of segment : 419 time for calcul the mask position with numpy : 0.0008406639099121094 nb_pixel_total : 38232 time to create 1 rle with old method : 0.04240560531616211 length of segment : 237 time for calcul the mask position with numpy : 0.00122833251953125 nb_pixel_total : 41223 time to create 1 rle with old method : 0.04661107063293457 length of segment : 314 time for calcul the mask position with numpy : 0.0030219554901123047 nb_pixel_total : 99139 time to create 1 rle with old method : 0.10945868492126465 length of segment : 503 time for calcul the mask position with numpy : 0.0019404888153076172 nb_pixel_total : 91792 time to create 1 rle with old method : 0.13371896743774414 length of segment : 276 time for calcul the mask position with numpy : 0.0008931159973144531 nb_pixel_total : 28971 time to create 1 rle with old method : 0.03498435020446777 length of segment : 281 time for calcul the mask position with numpy : 0.0019659996032714844 nb_pixel_total : 64720 time to create 1 rle with old method : 0.09510040283203125 length of segment : 322 time for calcul the mask position with numpy : 0.0007231235504150391 nb_pixel_total : 38216 time to create 1 rle with old method : 0.04424405097961426 length of segment : 221 time for calcul the mask position with numpy : 0.008757829666137695 nb_pixel_total : 324865 time to create 1 rle with new method : 0.02473306655883789 length of segment : 815 time for calcul the mask position with numpy : 0.0018053054809570312 nb_pixel_total : 64816 time to create 1 rle with old method : 0.07216572761535645 length of segment : 427 time for calcul the mask position with numpy : 0.01198577880859375 nb_pixel_total : 445033 time to create 1 rle with new method : 0.23506879806518555 length of segment : 1060 time for calcul the mask position with numpy : 0.0032269954681396484 nb_pixel_total : 178593 time to create 1 rle with new method : 0.008857011795043945 length of segment : 479 time for calcul the mask position with numpy : 0.0007076263427734375 nb_pixel_total : 27617 time to create 1 rle with old method : 0.03132939338684082 length of segment : 182 time for calcul the mask position with numpy : 0.002844572067260742 nb_pixel_total : 169989 time to create 1 rle with new method : 0.007780313491821289 length of segment : 452 time for calcul the mask position with numpy : 0.00569605827331543 nb_pixel_total : 227445 time to create 1 rle with new method : 0.015520572662353516 length of segment : 885 time for calcul the mask position with numpy : 0.028362751007080078 nb_pixel_total : 1183779 time to create 1 rle with new method : 0.3485696315765381 length of segment : 1586 time for calcul the mask position with numpy : 0.0016012191772460938 nb_pixel_total : 73882 time to create 1 rle with old method : 0.07980966567993164 length of segment : 460 time for calcul the mask position with numpy : 0.011688470840454102 nb_pixel_total : 545427 time to create 1 rle with new method : 0.04646801948547363 length of segment : 978 time for calcul the mask position with numpy : 0.0069506168365478516 nb_pixel_total : 377389 time to create 1 rle with new method : 0.02230238914489746 length of segment : 1485 time for calcul the mask position with numpy : 0.00047016143798828125 nb_pixel_total : 19626 time to create 1 rle with old method : 0.021687984466552734 length of segment : 226 time for calcul the mask position with numpy : 0.016438007354736328 nb_pixel_total : 724373 time to create 1 rle with new method : 0.17173242568969727 length of segment : 1015 time for calcul the mask position with numpy : 0.0008003711700439453 nb_pixel_total : 31109 time to create 1 rle with old method : 0.034287214279174805 length of segment : 351 time for calcul the mask position with numpy : 0.0034983158111572266 nb_pixel_total : 137140 time to create 1 rle with old method : 0.15292644500732422 length of segment : 699 time for calcul the mask position with numpy : 0.0007915496826171875 nb_pixel_total : 40188 time to create 1 rle with old method : 0.046309471130371094 length of segment : 240 time for calcul the mask position with numpy : 0.004934787750244141 nb_pixel_total : 229198 time to create 1 rle with new method : 0.015561342239379883 length of segment : 666 time for calcul the mask position with numpy : 0.0040056705474853516 nb_pixel_total : 254722 time to create 1 rle with new method : 0.010296821594238281 length of segment : 411 time for calcul the mask position with numpy : 0.0043201446533203125 nb_pixel_total : 200492 time to create 1 rle with new method : 0.01141357421875 length of segment : 598 time for calcul the mask position with numpy : 0.0006017684936523438 nb_pixel_total : 15861 time to create 1 rle with old method : 0.018140077590942383 length of segment : 167 time for calcul the mask position with numpy : 0.008997440338134766 nb_pixel_total : 437415 time to create 1 rle with new method : 0.11708903312683105 length of segment : 1401 time for calcul the mask position with numpy : 0.0008108615875244141 nb_pixel_total : 12971 time to create 1 rle with old method : 0.015197038650512695 length of segment : 206 time for calcul the mask position with numpy : 0.00041937828063964844 nb_pixel_total : 15305 time to create 1 rle with old method : 0.018054962158203125 length of segment : 131 time for calcul the mask position with numpy : 0.0016818046569824219 nb_pixel_total : 76295 time to create 1 rle with old method : 0.08461260795593262 length of segment : 319 time for calcul the mask position with numpy : 0.0026721954345703125 nb_pixel_total : 175204 time to create 1 rle with new method : 0.008008956909179688 length of segment : 767 time for calcul the mask position with numpy : 0.0013899803161621094 nb_pixel_total : 47411 time to create 1 rle with old method : 0.052910804748535156 length of segment : 218 time for calcul the mask position with numpy : 0.0036249160766601562 nb_pixel_total : 129980 time to create 1 rle with old method : 0.14394569396972656 length of segment : 515 time for calcul the mask position with numpy : 0.003092050552368164 nb_pixel_total : 163499 time to create 1 rle with new method : 0.008402109146118164 length of segment : 522 time for calcul the mask position with numpy : 0.0012676715850830078 nb_pixel_total : 59294 time to create 1 rle with old method : 0.06824851036071777 length of segment : 375 time for calcul the mask position with numpy : 0.0027456283569335938 nb_pixel_total : 130251 time to create 1 rle with old method : 0.14795589447021484 length of segment : 607 time for calcul the mask position with numpy : 0.0021588802337646484 nb_pixel_total : 76825 time to create 1 rle with old method : 0.08647918701171875 length of segment : 442 time for calcul the mask position with numpy : 0.02612614631652832 nb_pixel_total : 1191173 time to create 1 rle with new method : 0.2021780014038086 length of segment : 1688 time for calcul the mask position with numpy : 0.0017993450164794922 nb_pixel_total : 82576 time to create 1 rle with old method : 0.08882713317871094 length of segment : 307 time for calcul the mask position with numpy : 0.0040781497955322266 nb_pixel_total : 200009 time to create 1 rle with new method : 0.012399673461914062 length of segment : 917 time for calcul the mask position with numpy : 0.00546574592590332 nb_pixel_total : 370107 time to create 1 rle with new method : 0.014973640441894531 length of segment : 653 time for calcul the mask position with numpy : 0.003993511199951172 nb_pixel_total : 171113 time to create 1 rle with new method : 0.013198375701904297 length of segment : 630 time for calcul the mask position with numpy : 0.00622248649597168 nb_pixel_total : 206971 time to create 1 rle with new method : 0.020794391632080078 length of segment : 735 time for calcul the mask position with numpy : 0.001971006393432617 nb_pixel_total : 111416 time to create 1 rle with old method : 0.12013506889343262 length of segment : 327 time for calcul the mask position with numpy : 0.0014650821685791016 nb_pixel_total : 76755 time to create 1 rle with old method : 0.08310294151306152 length of segment : 391 time for calcul the mask position with numpy : 0.0014846324920654297 nb_pixel_total : 71696 time to create 1 rle with old method : 0.07688164710998535 length of segment : 417 time for calcul the mask position with numpy : 0.0023484230041503906 nb_pixel_total : 143888 time to create 1 rle with old method : 0.18184638023376465 length of segment : 486 time for calcul the mask position with numpy : 0.001299142837524414 nb_pixel_total : 67939 time to create 1 rle with old method : 0.0791616439819336 length of segment : 261 time for calcul the mask position with numpy : 0.0016467571258544922 nb_pixel_total : 59266 time to create 1 rle with old method : 0.06607222557067871 length of segment : 378 time for calcul the mask position with numpy : 0.0018353462219238281 nb_pixel_total : 91168 time to create 1 rle with old method : 0.10727953910827637 length of segment : 291 time for calcul the mask position with numpy : 0.20822405815124512 nb_pixel_total : 2548338 time to create 1 rle with new method : 1.2220487594604492 length of segment : 6382 time for calcul the mask position with numpy : 0.0012807846069335938 nb_pixel_total : 100620 time to create 1 rle with old method : 0.11449408531188965 length of segment : 336 time for calcul the mask position with numpy : 0.017878055572509766 nb_pixel_total : 1015897 time to create 1 rle with new method : 0.13477277755737305 length of segment : 1071 time for calcul the mask position with numpy : 0.0017862319946289062 nb_pixel_total : 173452 time to create 1 rle with new method : 0.003747701644897461 length of segment : 268 time for calcul the mask position with numpy : 0.002869129180908203 nb_pixel_total : 199523 time to create 1 rle with new method : 0.007139682769775391 length of segment : 321 time spent for convertir_results : 36.267983198165894 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 212 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 91.28620886802673 time spend to save output : 0.1407942771911621 total time spend for step 2 : 91.4270031452179 step3:blur_detection Tue May 6 21:43:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (4160, 3120) 3986.6782363103684 (4160, 3120) 1610.5734421053278 (4160, 3120) 1513.1497577275027 (4160, 3120) 105.71229194791607 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.013376712799072266 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.012769699096679688 save missing photos in datou_result : time spend for datou_step_exec : 2.4074018001556396 time spend to save output : 0.030939817428588867 total time spend for step 3 : 2.4383416175842285 step4:brightness Tue May 6 21:43:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746560486_2145787_1356446286_efad209f25d2d7fef981365c7755df67.jpg treat image : temp/1746560486_2145787_1356446283_d2c1ae0f12b72f0c8ef98836ab79f3f0.jpg treat image : temp/1746560486_2145787_1356446281_e808adda9cb7ac38903f58764b85f374.jpg treat image : temp/1746560486_2145787_1356446276_a4f1b1530382563c91a9bf9c48f41ae2.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.01395273208618164 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.013816595077514648 save missing photos in datou_result : time spend for datou_step_exec : 6.7007927894592285 time spend to save output : 0.0320887565612793 total time spend for step 4 : 6.732881546020508 step5:rle_unique_nms_with_priority Tue May 6 21:43:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 212 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 68 nb_hashtags : 10 time to prepare the origin masks : 53.9887421131134 time for calcul the mask position with numpy : 0.5928781032562256 nb_pixel_total : 5653353 time to create 1 rle with new method : 1.1134061813354492 time for calcul the mask position with numpy : 0.05098724365234375 nb_pixel_total : 126055 time to create 1 rle with old method : 0.1337902545928955 time for calcul the mask position with numpy : 0.05351376533508301 nb_pixel_total : 10386 time to create 1 rle with old method : 0.012148380279541016 time for calcul the mask position with numpy : 0.052720069885253906 nb_pixel_total : 52716 time to create 1 rle with old method : 0.061105966567993164 time for calcul the mask position with numpy : 0.05096840858459473 nb_pixel_total : 38689 time to create 1 rle with old method : 0.04282879829406738 time for calcul the mask position with numpy : 0.05287742614746094 nb_pixel_total : 53150 time to create 1 rle with old method : 0.05936145782470703 time for calcul the mask position with numpy : 0.05193591117858887 nb_pixel_total : 125705 time to create 1 rle with old method : 0.13474559783935547 time for calcul the mask position with numpy : 0.05248665809631348 nb_pixel_total : 111144 time to create 1 rle with old method : 0.1235044002532959 time for calcul the mask position with numpy : 0.0520014762878418 nb_pixel_total : 25975 time to create 1 rle with old method : 0.028548479080200195 time for calcul the mask position with numpy : 0.05222940444946289 nb_pixel_total : 195 time to create 1 rle with old method : 0.00026226043701171875 time for calcul the mask position with numpy : 0.05311846733093262 nb_pixel_total : 563 time to create 1 rle with old method : 0.0008971691131591797 time for calcul the mask position with numpy : 0.05250978469848633 nb_pixel_total : 16453 time to create 1 rle with old method : 0.018268108367919922 time for calcul the mask position with numpy : 0.05233359336853027 nb_pixel_total : 121477 time to create 1 rle with old method : 0.13282489776611328 time for calcul the mask position with numpy : 0.05141115188598633 nb_pixel_total : 53021 time to create 1 rle with old method : 0.057573556900024414 time for calcul the mask position with numpy : 0.0526885986328125 nb_pixel_total : 175777 time to create 1 rle with new method : 0.7737047672271729 time for calcul the mask position with numpy : 0.05025148391723633 nb_pixel_total : 1381 time to create 1 rle with old method : 0.0015857219696044922 time for calcul the mask position with numpy : 0.04931306838989258 nb_pixel_total : 31061 time to create 1 rle with old method : 0.03258800506591797 time for calcul the mask position with numpy : 0.050411224365234375 nb_pixel_total : 202519 time to create 1 rle with new method : 1.294663429260254 time for calcul the mask position with numpy : 0.05196118354797363 nb_pixel_total : 76371 time to create 1 rle with old method : 0.10674238204956055 time for calcul the mask position with numpy : 0.052104949951171875 nb_pixel_total : 179070 time to create 1 rle with new method : 0.971477746963501 time for calcul the mask position with numpy : 0.05183839797973633 nb_pixel_total : 23081 time to create 1 rle with old method : 0.02571249008178711 time for calcul the mask position with numpy : 0.23071670532226562 nb_pixel_total : 578568 time to create 1 rle with new method : 0.9300448894500732 time for calcul the mask position with numpy : 0.05070757865905762 nb_pixel_total : 5664 time to create 1 rle with old method : 0.006086826324462891 time for calcul the mask position with numpy : 0.05180954933166504 nb_pixel_total : 10177 time to create 1 rle with old method : 0.011157989501953125 time for calcul the mask position with numpy : 0.05356454849243164 nb_pixel_total : 203691 time to create 1 rle with new method : 1.2807109355926514 time for calcul the mask position with numpy : 0.05313825607299805 nb_pixel_total : 162683 time to create 1 rle with new method : 0.9197964668273926 time for calcul the mask position with numpy : 0.05163431167602539 nb_pixel_total : 37262 time to create 1 rle with old method : 0.03992009162902832 time for calcul the mask position with numpy : 0.05268096923828125 nb_pixel_total : 1300 time to create 1 rle with old method : 0.0016357898712158203 time for calcul the mask position with numpy : 0.052503347396850586 nb_pixel_total : 23210 time to create 1 rle with old method : 0.025389671325683594 time for calcul the mask position with numpy : 0.05353379249572754 nb_pixel_total : 92791 time to create 1 rle with old method : 0.1018362045288086 time for calcul the mask position with numpy : 0.05144906044006348 nb_pixel_total : 33552 time to create 1 rle with old method : 0.03618788719177246 time for calcul the mask position with numpy : 0.052454233169555664 nb_pixel_total : 18765 time to create 1 rle with old method : 0.02064061164855957 time for calcul the mask position with numpy : 0.05194830894470215 nb_pixel_total : 86682 time to create 1 rle with old method : 0.09411787986755371 time for calcul the mask position with numpy : 0.05286526679992676 nb_pixel_total : 312901 time to create 1 rle with new method : 1.9923923015594482 time for calcul the mask position with numpy : 0.04899787902832031 nb_pixel_total : 8590 time to create 1 rle with old method : 0.00931692123413086 time for calcul the mask position with numpy : 0.05021929740905762 nb_pixel_total : 29157 time to create 1 rle with old method : 0.030800580978393555 time for calcul the mask position with numpy : 0.05127716064453125 nb_pixel_total : 58586 time to create 1 rle with old method : 0.06150650978088379 time for calcul the mask position with numpy : 0.050217390060424805 nb_pixel_total : 18583 time to create 1 rle with old method : 0.019580364227294922 time for calcul the mask position with numpy : 0.05034995079040527 nb_pixel_total : 122441 time to create 1 rle with old method : 0.1294393539428711 time for calcul the mask position with numpy : 0.050194740295410156 nb_pixel_total : 65484 time to create 1 rle with old method : 0.07145333290100098 time for calcul the mask position with numpy : 0.05035591125488281 nb_pixel_total : 65135 time to create 1 rle with old method : 0.06997370719909668 time for calcul the mask position with numpy : 0.0514986515045166 nb_pixel_total : 112509 time to create 1 rle with old method : 0.11630558967590332 time for calcul the mask position with numpy : 0.051137685775756836 nb_pixel_total : 47599 time to create 1 rle with old method : 0.05141401290893555 time for calcul the mask position with numpy : 0.05206894874572754 nb_pixel_total : 3682 time to create 1 rle with old method : 0.0040776729583740234 time for calcul the mask position with numpy : 0.05183124542236328 nb_pixel_total : 16999 time to create 1 rle with old method : 0.018875598907470703 time for calcul the mask position with numpy : 0.052545785903930664 nb_pixel_total : 20142 time to create 1 rle with old method : 0.02231431007385254 time for calcul the mask position with numpy : 0.05413985252380371 nb_pixel_total : 301955 time to create 1 rle with new method : 0.8564050197601318 time for calcul the mask position with numpy : 0.051752567291259766 nb_pixel_total : 35600 time to create 1 rle with old method : 0.039009809494018555 time for calcul the mask position with numpy : 0.05197644233703613 nb_pixel_total : 315263 time to create 1 rle with new method : 0.9379682540893555 time for calcul the mask position with numpy : 0.05009579658508301 nb_pixel_total : 206603 time to create 1 rle with new method : 1.4685609340667725 time for calcul the mask position with numpy : 0.05139589309692383 nb_pixel_total : 43932 time to create 1 rle with old method : 0.04791665077209473 time for calcul the mask position with numpy : 0.052222490310668945 nb_pixel_total : 1458 time to create 1 rle with old method : 0.0020592212677001953 time for calcul the mask position with numpy : 0.051798343658447266 nb_pixel_total : 49831 time to create 1 rle with old method : 0.05372786521911621 time for calcul the mask position with numpy : 0.05195283889770508 nb_pixel_total : 4074 time to create 1 rle with old method : 0.00484466552734375 time for calcul the mask position with numpy : 0.052429914474487305 nb_pixel_total : 32486 time to create 1 rle with old method : 0.034947872161865234 time for calcul the mask position with numpy : 0.05096292495727539 nb_pixel_total : 87893 time to create 1 rle with old method : 0.09492635726928711 time for calcul the mask position with numpy : 0.05408000946044922 nb_pixel_total : 711082 time to create 1 rle with new method : 1.154679775238037 time for calcul the mask position with numpy : 0.05187726020812988 nb_pixel_total : 103093 time to create 1 rle with old method : 0.11162567138671875 time for calcul the mask position with numpy : 0.05191206932067871 nb_pixel_total : 8655 time to create 1 rle with old method : 0.010577678680419922 time for calcul the mask position with numpy : 0.055869102478027344 nb_pixel_total : 734055 time to create 1 rle with new method : 0.855079174041748 time for calcul the mask position with numpy : 0.052633047103881836 nb_pixel_total : 186290 time to create 1 rle with new method : 1.0581552982330322 time for calcul the mask position with numpy : 0.051251888275146484 nb_pixel_total : 28199 time to create 1 rle with old method : 0.03204703330993652 time for calcul the mask position with numpy : 0.05357503890991211 nb_pixel_total : 231309 time to create 1 rle with new method : 0.9684844017028809 time for calcul the mask position with numpy : 0.051287174224853516 nb_pixel_total : 81911 time to create 1 rle with old method : 0.09007573127746582 time for calcul the mask position with numpy : 0.049936532974243164 nb_pixel_total : 210352 time to create 1 rle with new method : 1.0921027660369873 time for calcul the mask position with numpy : 0.05315136909484863 nb_pixel_total : 16634 time to create 1 rle with old method : 0.018454790115356445 time for calcul the mask position with numpy : 0.0553593635559082 nb_pixel_total : 298048 time to create 1 rle with new method : 0.7982757091522217 time for calcul the mask position with numpy : 0.05016922950744629 nb_pixel_total : 2135 time to create 1 rle with old method : 0.002518177032470703 time for calcul the mask position with numpy : 0.05571293830871582 nb_pixel_total : 74047 time to create 1 rle with old method : 0.09892392158508301 create new chi : 26.052682161331177 time to delete rle : 0.03616023063659668 batch 1 Loaded 69 chid ids of type : 4855 Number RLEs to save : 49947 TO DO : save crop sub photo not yet done ! save time : 2.714324951171875 nb_obj : 65 nb_hashtags : 8 time to prepare the origin masks : 64.03870820999146 time for calcul the mask position with numpy : 0.36444568634033203 nb_pixel_total : 3838678 time to create 1 rle with new method : 2.464487075805664 time for calcul the mask position with numpy : 0.05548357963562012 nb_pixel_total : 237807 time to create 1 rle with new method : 1.6013216972351074 time for calcul the mask position with numpy : 0.053145647048950195 nb_pixel_total : 11688 time to create 1 rle with old method : 0.012984275817871094 time for calcul the mask position with numpy : 0.09150838851928711 nb_pixel_total : 1272165 time to create 1 rle with new method : 1.159754753112793 time for calcul the mask position with numpy : 0.07560491561889648 nb_pixel_total : 1680376 time to create 1 rle with new method : 2.694457530975342 time for calcul the mask position with numpy : 0.05379986763000488 nb_pixel_total : 26405 time to create 1 rle with old method : 0.029779911041259766 time for calcul the mask position with numpy : 0.05361151695251465 nb_pixel_total : 22088 time to create 1 rle with old method : 0.024998188018798828 time for calcul the mask position with numpy : 0.05383610725402832 nb_pixel_total : 21507 time to create 1 rle with old method : 0.0244443416595459 time for calcul the mask position with numpy : 0.05904507637023926 nb_pixel_total : 64874 time to create 1 rle with old method : 0.09046506881713867 time for calcul the mask position with numpy : 0.05419635772705078 nb_pixel_total : 83088 time to create 1 rle with old method : 0.0927431583404541 time for calcul the mask position with numpy : 0.05344963073730469 nb_pixel_total : 43572 time to create 1 rle with old method : 0.0488889217376709 time for calcul the mask position with numpy : 0.05194878578186035 nb_pixel_total : 27959 time to create 1 rle with old method : 0.030905485153198242 time for calcul the mask position with numpy : 0.05332040786743164 nb_pixel_total : 10988 time to create 1 rle with old method : 0.012125968933105469 time for calcul the mask position with numpy : 0.05283331871032715 nb_pixel_total : 6787 time to create 1 rle with old method : 0.007831811904907227 time for calcul the mask position with numpy : 0.05279707908630371 nb_pixel_total : 886 time to create 1 rle with old method : 0.0010483264923095703 time for calcul the mask position with numpy : 0.054227590560913086 nb_pixel_total : 20763 time to create 1 rle with old method : 0.035330772399902344 time for calcul the mask position with numpy : 0.05885744094848633 nb_pixel_total : 15079 time to create 1 rle with old method : 0.016878843307495117 time for calcul the mask position with numpy : 0.05776500701904297 nb_pixel_total : 499943 time to create 1 rle with new method : 1.2192676067352295 time for calcul the mask position with numpy : 0.05402731895446777 nb_pixel_total : 39659 time to create 1 rle with old method : 0.044059038162231445 time for calcul the mask position with numpy : 0.055005788803100586 nb_pixel_total : 235542 time to create 1 rle with new method : 1.0440552234649658 time for calcul the mask position with numpy : 0.05345869064331055 nb_pixel_total : 72870 time to create 1 rle with old method : 0.0802302360534668 time for calcul the mask position with numpy : 0.0569918155670166 nb_pixel_total : 1171216 time to create 1 rle with new method : 1.1726808547973633 time for calcul the mask position with numpy : 0.05274200439453125 nb_pixel_total : 111477 time to create 1 rle with old method : 0.12308835983276367 time for calcul the mask position with numpy : 0.05573272705078125 nb_pixel_total : 39 time to create 1 rle with old method : 9.036064147949219e-05 time for calcul the mask position with numpy : 0.05116748809814453 nb_pixel_total : 14429 time to create 1 rle with old method : 0.016022205352783203 time for calcul the mask position with numpy : 0.05258035659790039 nb_pixel_total : 150312 time to create 1 rle with new method : 0.9968054294586182 time for calcul the mask position with numpy : 0.05054521560668945 nb_pixel_total : 31432 time to create 1 rle with old method : 0.03388833999633789 time for calcul the mask position with numpy : 0.051317691802978516 nb_pixel_total : 8751 time to create 1 rle with old method : 0.009888172149658203 time for calcul the mask position with numpy : 0.05106711387634277 nb_pixel_total : 19848 time to create 1 rle with old method : 0.021420717239379883 time for calcul the mask position with numpy : 0.05198359489440918 nb_pixel_total : 22881 time to create 1 rle with old method : 0.02615642547607422 time for calcul the mask position with numpy : 0.052283287048339844 nb_pixel_total : 69888 time to create 1 rle with old method : 0.07597923278808594 time for calcul the mask position with numpy : 0.05317044258117676 nb_pixel_total : 9218 time to create 1 rle with old method : 0.0106353759765625 time for calcul the mask position with numpy : 0.05359244346618652 nb_pixel_total : 19898 time to create 1 rle with old method : 0.022470474243164062 time for calcul the mask position with numpy : 0.05582475662231445 nb_pixel_total : 52359 time to create 1 rle with old method : 0.059969425201416016 time for calcul the mask position with numpy : 0.05330944061279297 nb_pixel_total : 100072 time to create 1 rle with old method : 0.11157751083374023 time for calcul the mask position with numpy : 0.05293464660644531 nb_pixel_total : 112106 time to create 1 rle with old method : 0.12354683876037598 time for calcul the mask position with numpy : 0.05240345001220703 nb_pixel_total : 70145 time to create 1 rle with old method : 0.07620549201965332 time for calcul the mask position with numpy : 0.050750017166137695 nb_pixel_total : 59003 time to create 1 rle with old method : 0.06470775604248047 time for calcul the mask position with numpy : 0.051575660705566406 nb_pixel_total : 49111 time to create 1 rle with old method : 0.060659170150756836 time for calcul the mask position with numpy : 0.0606074333190918 nb_pixel_total : 33511 time to create 1 rle with old method : 0.037897586822509766 time for calcul the mask position with numpy : 0.05260777473449707 nb_pixel_total : 148126 time to create 1 rle with old method : 0.16413426399230957 time for calcul the mask position with numpy : 0.05362296104431152 nb_pixel_total : 101639 time to create 1 rle with old method : 0.11093664169311523 time for calcul the mask position with numpy : 0.051784515380859375 nb_pixel_total : 28142 time to create 1 rle with old method : 0.033319711685180664 time for calcul the mask position with numpy : 0.05201983451843262 nb_pixel_total : 61683 time to create 1 rle with old method : 0.06661391258239746 time for calcul the mask position with numpy : 0.051908254623413086 nb_pixel_total : 39076 time to create 1 rle with old method : 0.04186272621154785 time for calcul the mask position with numpy : 0.051526546478271484 nb_pixel_total : 800 time to create 1 rle with old method : 0.0012116432189941406 time for calcul the mask position with numpy : 0.05195808410644531 nb_pixel_total : 37747 time to create 1 rle with old method : 0.04004955291748047 time for calcul the mask position with numpy : 0.05176711082458496 nb_pixel_total : 408678 time to create 1 rle with new method : 1.0551671981811523 time for calcul the mask position with numpy : 0.0523526668548584 nb_pixel_total : 346864 time to create 1 rle with new method : 1.1846866607666016 time for calcul the mask position with numpy : 0.05299711227416992 nb_pixel_total : 5232 time to create 1 rle with old method : 0.0063593387603759766 time for calcul the mask position with numpy : 0.05378580093383789 nb_pixel_total : 35164 time to create 1 rle with old method : 0.039138078689575195 time for calcul the mask position with numpy : 0.05423760414123535 nb_pixel_total : 155455 time to create 1 rle with new method : 1.0675067901611328 time for calcul the mask position with numpy : 0.05156302452087402 nb_pixel_total : 173995 time to create 1 rle with new method : 1.0410106182098389 time for calcul the mask position with numpy : 0.0518949031829834 nb_pixel_total : 37529 time to create 1 rle with old method : 0.053900718688964844 time for calcul the mask position with numpy : 0.06160902976989746 nb_pixel_total : 233062 time to create 1 rle with new method : 1.4557459354400635 time for calcul the mask position with numpy : 0.05639338493347168 nb_pixel_total : 369935 time to create 1 rle with new method : 1.2781093120574951 time for calcul the mask position with numpy : 0.056313276290893555 nb_pixel_total : 7946 time to create 1 rle with old method : 0.00910496711730957 time for calcul the mask position with numpy : 0.052907705307006836 nb_pixel_total : 65408 time to create 1 rle with old method : 0.0707557201385498 time for calcul the mask position with numpy : 0.05345606803894043 nb_pixel_total : 7683 time to create 1 rle with old method : 0.009045124053955078 time for calcul the mask position with numpy : 0.054178714752197266 nb_pixel_total : 114595 time to create 1 rle with old method : 0.15030312538146973 time for calcul the mask position with numpy : 0.0545651912689209 nb_pixel_total : 20364 time to create 1 rle with old method : 0.023768901824951172 time for calcul the mask position with numpy : 0.05418038368225098 nb_pixel_total : 97927 time to create 1 rle with old method : 0.11061978340148926 time for calcul the mask position with numpy : 0.05468463897705078 nb_pixel_total : 49351 time to create 1 rle with old method : 0.05512285232543945 time for calcul the mask position with numpy : 0.054550886154174805 nb_pixel_total : 66167 time to create 1 rle with old method : 0.09594607353210449 time for calcul the mask position with numpy : 0.054927825927734375 nb_pixel_total : 16214 time to create 1 rle with old method : 0.018511056900024414 time for calcul the mask position with numpy : 0.05217766761779785 nb_pixel_total : 11998 time to create 1 rle with old method : 0.013258218765258789 create new chi : 26.51333212852478 time to delete rle : 0.005589008331298828 batch 1 Loaded 66 chid ids of type : 4855 Number RLEs to save : 52117 TO DO : save crop sub photo not yet done ! save time : 2.8844616413116455 nb_obj : 66 nb_hashtags : 9 time to prepare the origin masks : 54.209261417388916 time for calcul the mask position with numpy : 0.5730514526367188 nb_pixel_total : 4875230 time to create 1 rle with new method : 1.4240949153900146 time for calcul the mask position with numpy : 0.07850480079650879 nb_pixel_total : 938435 time to create 1 rle with new method : 1.2417726516723633 time for calcul the mask position with numpy : 0.05070924758911133 nb_pixel_total : 171181 time to create 1 rle with new method : 0.9573402404785156 time for calcul the mask position with numpy : 0.0497126579284668 nb_pixel_total : 12124 time to create 1 rle with old method : 0.01232457160949707 time for calcul the mask position with numpy : 0.05089139938354492 nb_pixel_total : 178108 time to create 1 rle with new method : 1.0161044597625732 time for calcul the mask position with numpy : 0.04872941970825195 nb_pixel_total : 2462 time to create 1 rle with old method : 0.002779245376586914 time for calcul the mask position with numpy : 0.058031558990478516 nb_pixel_total : 433858 time to create 1 rle with new method : 1.206488847732544 time for calcul the mask position with numpy : 0.05992889404296875 nb_pixel_total : 64300 time to create 1 rle with old method : 0.07023310661315918 time for calcul the mask position with numpy : 0.0527796745300293 nb_pixel_total : 41091 time to create 1 rle with old method : 0.04410290718078613 time for calcul the mask position with numpy : 0.05224752426147461 nb_pixel_total : 25597 time to create 1 rle with old method : 0.027347087860107422 time for calcul the mask position with numpy : 0.05275392532348633 nb_pixel_total : 40242 time to create 1 rle with old method : 0.04446053504943848 time for calcul the mask position with numpy : 0.052608489990234375 nb_pixel_total : 26691 time to create 1 rle with old method : 0.02882528305053711 time for calcul the mask position with numpy : 0.052510976791381836 nb_pixel_total : 1311 time to create 1 rle with old method : 0.0017216205596923828 time for calcul the mask position with numpy : 0.05318188667297363 nb_pixel_total : 9123 time to create 1 rle with old method : 0.010345935821533203 time for calcul the mask position with numpy : 0.05458259582519531 nb_pixel_total : 323469 time to create 1 rle with new method : 1.1832191944122314 time for calcul the mask position with numpy : 0.05132007598876953 nb_pixel_total : 64754 time to create 1 rle with old method : 0.06654739379882812 time for calcul the mask position with numpy : 0.0527195930480957 nb_pixel_total : 91482 time to create 1 rle with old method : 0.09430861473083496 time for calcul the mask position with numpy : 0.051055908203125 nb_pixel_total : 23440 time to create 1 rle with old method : 0.02408146858215332 time for calcul the mask position with numpy : 0.05080270767211914 nb_pixel_total : 38148 time to create 1 rle with old method : 0.04055190086364746 time for calcul the mask position with numpy : 0.049204111099243164 nb_pixel_total : 15816 time to create 1 rle with old method : 0.017052412033081055 time for calcul the mask position with numpy : 0.049356937408447266 nb_pixel_total : 16924 time to create 1 rle with old method : 0.017611980438232422 time for calcul the mask position with numpy : 0.05077338218688965 nb_pixel_total : 196438 time to create 1 rle with new method : 1.0151820182800293 time for calcul the mask position with numpy : 0.05119514465332031 nb_pixel_total : 31009 time to create 1 rle with old method : 0.033690690994262695 time for calcul the mask position with numpy : 0.05111193656921387 nb_pixel_total : 29992 time to create 1 rle with old method : 0.03206014633178711 time for calcul the mask position with numpy : 0.05113649368286133 nb_pixel_total : 46946 time to create 1 rle with old method : 0.05062723159790039 time for calcul the mask position with numpy : 0.051590919494628906 nb_pixel_total : 19494 time to create 1 rle with old method : 0.020878076553344727 time for calcul the mask position with numpy : 0.050080060958862305 nb_pixel_total : 2342 time to create 1 rle with old method : 0.0026450157165527344 time for calcul the mask position with numpy : 0.05212664604187012 nb_pixel_total : 15229 time to create 1 rle with old method : 0.016594409942626953 time for calcul the mask position with numpy : 0.05397486686706543 nb_pixel_total : 75096 time to create 1 rle with old method : 0.07906866073608398 time for calcul the mask position with numpy : 0.05327916145324707 nb_pixel_total : 28765 time to create 1 rle with old method : 0.04632568359375 time for calcul the mask position with numpy : 0.06794428825378418 nb_pixel_total : 1181393 time to create 1 rle with new method : 1.2593491077423096 time for calcul the mask position with numpy : 0.052730560302734375 nb_pixel_total : 443212 time to create 1 rle with new method : 1.0333831310272217 time for calcul the mask position with numpy : 0.05101442337036133 nb_pixel_total : 67731 time to create 1 rle with old method : 0.07341647148132324 time for calcul the mask position with numpy : 0.05169248580932617 nb_pixel_total : 27570 time to create 1 rle with old method : 0.028598308563232422 time for calcul the mask position with numpy : 0.05184626579284668 nb_pixel_total : 82406 time to create 1 rle with old method : 0.08723139762878418 time for calcul the mask position with numpy : 0.050744056701660156 nb_pixel_total : 38849 time to create 1 rle with old method : 0.04041767120361328 time for calcul the mask position with numpy : 0.04888176918029785 nb_pixel_total : 23442 time to create 1 rle with old method : 0.024914264678955078 time for calcul the mask position with numpy : 0.05001521110534668 nb_pixel_total : 43354 time to create 1 rle with old method : 0.04520750045776367 time for calcul the mask position with numpy : 0.05249500274658203 nb_pixel_total : 91405 time to create 1 rle with old method : 0.11964154243469238 time for calcul the mask position with numpy : 0.04933285713195801 nb_pixel_total : 31231 time to create 1 rle with old method : 0.03330230712890625 time for calcul the mask position with numpy : 0.05023527145385742 nb_pixel_total : 73853 time to create 1 rle with old method : 0.07814216613769531 time for calcul the mask position with numpy : 0.052037954330444336 nb_pixel_total : 163007 time to create 1 rle with new method : 1.2815947532653809 time for calcul the mask position with numpy : 0.05212712287902832 nb_pixel_total : 151562 time to create 1 rle with new method : 1.0619192123413086 time for calcul the mask position with numpy : 0.04985523223876953 nb_pixel_total : 125392 time to create 1 rle with old method : 0.1309828758239746 time for calcul the mask position with numpy : 0.04981637001037598 nb_pixel_total : 73256 time to create 1 rle with old method : 0.08068466186523438 time for calcul the mask position with numpy : 0.05214428901672363 nb_pixel_total : 98882 time to create 1 rle with old method : 0.10727477073669434 time for calcul the mask position with numpy : 0.05153679847717285 nb_pixel_total : 1774 time to create 1 rle with old method : 0.0022580623626708984 time for calcul the mask position with numpy : 0.05046534538269043 nb_pixel_total : 9483 time to create 1 rle with old method : 0.010298967361450195 time for calcul the mask position with numpy : 0.05346226692199707 nb_pixel_total : 543321 time to create 1 rle with new method : 1.1182982921600342 time for calcul the mask position with numpy : 0.05295538902282715 nb_pixel_total : 375533 time to create 1 rle with new method : 0.9995038509368896 time for calcul the mask position with numpy : 0.05157971382141113 nb_pixel_total : 12904 time to create 1 rle with old method : 0.016864776611328125 time for calcul the mask position with numpy : 0.05297350883483887 nb_pixel_total : 1128 time to create 1 rle with old method : 0.0015940666198730469 time for calcul the mask position with numpy : 0.05311775207519531 nb_pixel_total : 59169 time to create 1 rle with old method : 0.09607291221618652 time for calcul the mask position with numpy : 0.071258544921875 nb_pixel_total : 178883 time to create 1 rle with new method : 1.350198745727539 time for calcul the mask position with numpy : 0.05367445945739746 nb_pixel_total : 63474 time to create 1 rle with old method : 0.07025575637817383 time for calcul the mask position with numpy : 0.06162071228027344 nb_pixel_total : 606676 time to create 1 rle with new method : 1.3726487159729004 time for calcul the mask position with numpy : 0.05560111999511719 nb_pixel_total : 231128 time to create 1 rle with new method : 1.0784664154052734 time for calcul the mask position with numpy : 0.05272555351257324 nb_pixel_total : 4954 time to create 1 rle with old method : 0.005752086639404297 time for calcul the mask position with numpy : 0.0569305419921875 nb_pixel_total : 81436 time to create 1 rle with old method : 0.09711885452270508 time for calcul the mask position with numpy : 0.0545191764831543 nb_pixel_total : 75730 time to create 1 rle with old method : 0.08592653274536133 time for calcul the mask position with numpy : 0.052706241607666016 nb_pixel_total : 76704 time to create 1 rle with old method : 0.08726143836975098 time for calcul the mask position with numpy : 0.053434133529663086 nb_pixel_total : 3416 time to create 1 rle with old method : 0.004003763198852539 time for calcul the mask position with numpy : 0.0587923526763916 nb_pixel_total : 22 time to create 1 rle with old method : 0.00010895729064941406 time for calcul the mask position with numpy : 0.05237984657287598 nb_pixel_total : 30388 time to create 1 rle with old method : 0.03410530090332031 time for calcul the mask position with numpy : 0.05574440956115723 nb_pixel_total : 37760 time to create 1 rle with old method : 0.04192066192626953 time for calcul the mask position with numpy : 0.053082942962646484 nb_pixel_total : 2861 time to create 1 rle with old method : 0.0038423538208007812 time for calcul the mask position with numpy : 0.05227923393249512 nb_pixel_total : 26814 time to create 1 rle with old method : 0.02996516227722168 create new chi : 25.621456146240234 time to delete rle : 0.005975961685180664 batch 1 Loaded 67 chid ids of type : 4855 Number RLEs to save : 54142 TO DO : save crop sub photo not yet done ! save time : 3.079500675201416 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 12.476695537567139 time for calcul the mask position with numpy : 0.2947702407836914 nb_pixel_total : 6689454 time to create 1 rle with new method : 1.5489509105682373 time for calcul the mask position with numpy : 0.06536102294921875 nb_pixel_total : 674539 time to create 1 rle with new method : 1.4973225593566895 time for calcul the mask position with numpy : 0.09219598770141602 nb_pixel_total : 92252 time to create 1 rle with old method : 0.1647493839263916 time for calcul the mask position with numpy : 0.10407447814941406 nb_pixel_total : 223377 time to create 1 rle with new method : 2.020603895187378 time for calcul the mask position with numpy : 0.06804442405700684 nb_pixel_total : 26937 time to create 1 rle with old method : 0.029669523239135742 time for calcul the mask position with numpy : 0.10656189918518066 nb_pixel_total : 1404088 time to create 1 rle with new method : 1.1622846126556396 time for calcul the mask position with numpy : 0.0791923999786377 nb_pixel_total : 1421904 time to create 1 rle with new method : 1.1025192737579346 time for calcul the mask position with numpy : 0.3261563777923584 nb_pixel_total : 2446649 time to create 1 rle with new method : 1.0961942672729492 create new chi : 10.094006299972534 time to delete rle : 0.002430438995361328 batch 1 Loaded 8 chid ids of type : 4855 Number RLEs to save : 22318 TO DO : save crop sub photo not yet done ! save time : 1.3292648792266846 map_output_result : {1356446286: (0.6294471201932699, 'Should be the crop_list due to order', 0.3327075118787817), 1356446283: (0.6294471201932699, 'Should be the crop_list due to order', 0.7759292679397878), 1356446281: (0.6294471201932699, 'Should be the crop_list due to order', 0.5294608855954358), 1356446276: (0.6294471201932699, 'Should be the crop_list due to order', 0.8796908153590743)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 4 /1356446286.Didn't retrieve data . /1356446283.Didn't retrieve data . /1356446281.Didn't retrieve data . /1356446276.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.012491226196289062 save_final save missing photos in datou_result : time spend for datou_step_exec : 284.4805037975311 time spend to save output : 0.017071962356567383 total time spend for step 5 : 284.4975757598877 step6:crop_condition Tue May 6 21:48:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 4 ! batch 1 Loaded 210 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3015255 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746560934_2145787 we have uploaded 21 photos in the portfolio 3015255 time of upload the photos Elapsed time : 15.433103799819946 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 42 About to insert : list_path_to_insert length 42 new photo from crops ! About to upload 42 photos upload in portfolio : 3015255 init cache_photo without model_param we have 42 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746560972_2145787 we have uploaded 42 photos in the portfolio 3015255 time of upload the photos Elapsed time : 17.622120141983032 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3015255 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746560994_2145787 we have uploaded 14 photos in the portfolio 3015255 time of upload the photos Elapsed time : 6.288509368896484 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 3015255 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561005_2145787 we have uploaded 6 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.8014166355133057 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 3015255 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561013_2145787 we have uploaded 16 photos in the portfolio 3015255 time of upload the photos Elapsed time : 4.657711982727051 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561019_2145787 we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.4638996124267578 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 3015255 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561074_2145787 we have uploaded 31 photos in the portfolio 3015255 time of upload the photos Elapsed time : 8.737775802612305 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3015255 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561088_2145787 we have uploaded 8 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.1817920207977295 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 22 About to insert : list_path_to_insert length 22 new photo from crops ! About to upload 22 photos upload in portfolio : 3015255 init cache_photo without model_param we have 22 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746561200_2145787 we have uploaded 22 photos in the portfolio 3015255 time of upload the photos Elapsed time : 11.088809728622437 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 161 /1356454525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454729Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454730Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454731Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454735Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454740Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454871Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454887Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356454968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455539Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356455967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 487 time used for this insertion : 0.05209493637084961 save_final save missing photos in datou_result : time spend for datou_step_exec : 298.3155176639557 time spend to save output : 0.056665897369384766 total time spend for step 6 : 298.3721835613251 step7:ventilate_hashtags_in_portfolio Tue May 6 21:53:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22731304 get user id for portfolio 22731304 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22732741,22732742,22732743,22732744,22732745,22732750,22732752,22732753,22732754,22732755,22732756,22732757,22732759,22732760,22732761,22732762,22732763,22732764,22732765,22732766?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22731304 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 1 /22731304. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.015010595321655273 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.252891778945923 time spend to save output : 0.015288591384887695 total time spend for step 7 : 4.2681803703308105 step8:final Tue May 6 21:53:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356446286: ('0.5753207720446533',), 1356446283: ('0.5753207720446533',), 1356446281: ('0.5753207720446533',), 1356446276: ('0.5753207720446533',)} new output for save of step final : {1356446286: ('0.5753207720446533',), 1356446283: ('0.5753207720446533',), 1356446281: ('0.5753207720446533',), 1356446276: ('0.5753207720446533',)} [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 4 /1356446286.Didn't retrieve data . /1356446283.Didn't retrieve data . /1356446281.Didn't retrieve data . /1356446276.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.01241445541381836 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1086263656616211 time spend to save output : 0.012949705123901367 total time spend for step 8 : 0.12157607078552246 step9:velours_tree Tue May 6 21:53:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.378676176071167 time spend to save output : 3.647804260253906e-05 total time spend for step 9 : 0.37871265411376953 step10:send_mail_cod Tue May 6 21:53:37 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22731304_06-05-2025_21_53_37.pdf 22732715 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327151746561217 22732716 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327161746561219 22732717 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327171746561221 22732718 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327181746561222 22732719 imagette227327191746561222 22732720 imagette227327201746561222 22732721 imagette227327211746561222 22732722 imagette227327221746561222 22732723 imagette227327231746561222 22732724 imagette227327241746561222 22732725 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327251746561222 22732726 change filename to text .imagette227327261746561223 22732727 imagette227327271746561223 22732728 imagette227327281746561223 22732729 imagette227327291746561223 22732730 imagette227327301746561223 22732731 imagette227327311746561223 22732732 imagette227327321746561223 22732733 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327331746561223 22732735 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227327351746561224 22732736 imagette227327361746561226 22732737 imagette227327371746561226 22732738 imagette227327381746561226 22732739 imagette227327391746561226 22732740 imagette227327401746561226 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22731304 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22732741,22732742,22732743,22732744,22732745,22732750,22732752,22732753,22732754,22732755,22732756,22732757,22732759,22732760,22732761,22732762,22732763,22732764,22732765,22732766?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22731304 your option no_mail is active, we will not send the real mail to your client args[1356446286] : ((1356446286, 3986.6782363103684, 492609224), (1356446286, -0.12292033608357766, 496442774), '0.5753207720446533') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446283] : ((1356446283, 1610.5734421053278, 2107751945), (1356446283, -0.3113270428472655, 496442774), '0.5753207720446533') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446281] : ((1356446281, 1513.1497577275027, 2107751945), (1356446281, -0.4424915098618584, 496442774), '0.5753207720446533') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446276] : ((1356446276, 105.71229194791607, 492688767), (1356446276, -0.16279136555265697, 496442774), '0.5753207720446533') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.5753207720446533 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731304_06-05-2025_21_53_37.pdf results_Auto_P22731304_06-05-2025_21_53_37.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731304_06-05-2025_21_53_37.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22731304','results_Auto_P22731304_06-05-2025_21_53_37.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731304_06-05-2025_21_53_37.pdf','pdf','','2.43','0.5753207720446533') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.012535333633422852 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.087057113647461 time spend to save output : 0.012766599655151367 total time spend for step 10 : 13.099823713302612 step11:split_time_score Tue May 6 21:53:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('22', 4),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06052025 22731304 Nombre de photos uploadées : 4 / 23040 (0%) 06052025 22731304 Nombre de photos taguées (types de déchets): 0 / 4 (0%) 06052025 22731304 Nombre de photos taguées (volume) : 0 / 4 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 1.049041748046875e-05 ???? elapsed_time : fill_and_build_computed_from_old_data 0.0002884864807128906 elapsed_time : insert_dashboard_record_day_entry 0.02960991859436035 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731300 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731301 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731302 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731303 order by id desc limit 1 Qualite : 0.5753207720446533 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731304_06-05-2025_21_53_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731304 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06052025': {'nb_upload': 4, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356446286, 1356446283, 1356446281, 1356446276] Looping around the photos to save general results len do output : 1 /22731304Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446286', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446283', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446281', None, None, None, None, None, '2799189') ('4741', None, None, None, None, None, None, None, '2799189') ('4741', '22731304', '1356446276', None, None, None, None, None, '2799189') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.014230728149414062 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.487511396408081 time spend to save output : 0.014429330825805664 total time spend for step 11 : 3.5019407272338867 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 4 set_done_treatment 451.40user 227.58system 12:29.23elapsed 90%CPU (0avgtext+0avgdata 6103764maxresident)k 1267256inputs+560712outputs (25317major+35098941minor)pagefaults 0swaps