python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 829539 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2797335'] with mtr_portfolio_ids : ['22713553'] and first list_photo_ids : [] new path : /proc/829539/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 6 ; length of list_pids : 6 ; length of list_args : 6 time to download the photos : 1.7253620624542236 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue May 6 13:01:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3831 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 13:01:33.802264: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 13:01:33.827266: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 13:01:33.828896: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd710000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 13:01:33.828947: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 13:01:33.832015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 13:01:34.093711: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x99c9cb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 13:01:34.093785: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 13:01:34.095052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:01:34.095658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:01:34.100397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:01:34.104082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:01:34.104450: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:01:34.107350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:01:34.108804: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:01:34.115250: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:01:34.116506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:01:34.116613: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:01:34.117376: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 13:01:34.117400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 13:01:34.117414: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 13:01:34.118687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 13:01:34.421493: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:01:34.421596: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:01:34.421627: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:01:34.421652: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:01:34.421677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:01:34.421706: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:01:34.421737: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:01:34.421770: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:01:34.423137: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:01:34.424658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:01:34.424697: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:01:34.424724: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:01:34.424750: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:01:34.424781: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:01:34.424799: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:01:34.424816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:01:34.424833: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:01:34.426107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:01:34.426151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 13:01:34.426162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 13:01:34.426172: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 13:01:34.427460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 13:01:47.586671: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:01:47.832618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:01:49.839324: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.605072: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-06 13:01:50.670374: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.670451: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.671683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.671738: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.679001: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.679046: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.685178: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.685199: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.685772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.685787: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.713849: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.713913: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.713931: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-06 13:01:50.715038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.715070: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.716155: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.716183: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.723216: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.723242: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:01:50.724023: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.724807: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.725577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.733989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.734998: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.756344: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.757321: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.758291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.759252: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.763703: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.764664: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.765619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.766572: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.768973: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.778117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.779082: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.788522: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.789489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.790460: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.791431: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.792440: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:01:50.793401: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 6 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 829820 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2638 tf kernel not reseted sub process len(results) : 6 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 6 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3831 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.03909921646118164 nb_pixel_total : 840419 time to create 1 rle with new method : 0.07515454292297363 length of segment : 807 time for calcul the mask position with numpy : 0.004788637161254883 nb_pixel_total : 248106 time to create 1 rle with new method : 0.016949892044067383 length of segment : 421 time for calcul the mask position with numpy : 0.0027666091918945312 nb_pixel_total : 158217 time to create 1 rle with new method : 0.006856679916381836 length of segment : 596 time for calcul the mask position with numpy : 0.007283210754394531 nb_pixel_total : 369982 time to create 1 rle with new method : 0.02964639663696289 length of segment : 494 time for calcul the mask position with numpy : 0.0017292499542236328 nb_pixel_total : 74451 time to create 1 rle with old method : 0.0909111499786377 length of segment : 324 time for calcul the mask position with numpy : 0.004387378692626953 nb_pixel_total : 201621 time to create 1 rle with new method : 0.012255191802978516 length of segment : 452 time for calcul the mask position with numpy : 0.002393960952758789 nb_pixel_total : 114406 time to create 1 rle with old method : 0.1270298957824707 length of segment : 518 time for calcul the mask position with numpy : 0.0013055801391601562 nb_pixel_total : 58521 time to create 1 rle with old method : 0.0718226432800293 length of segment : 202 time for calcul the mask position with numpy : 0.0007207393646240234 nb_pixel_total : 34877 time to create 1 rle with old method : 0.03930497169494629 length of segment : 247 time for calcul the mask position with numpy : 0.00035834312438964844 nb_pixel_total : 1752 time to create 1 rle with old method : 0.0025663375854492188 length of segment : 47 time for calcul the mask position with numpy : 0.0019290447235107422 nb_pixel_total : 84810 time to create 1 rle with old method : 0.09735417366027832 length of segment : 577 time for calcul the mask position with numpy : 0.0005764961242675781 nb_pixel_total : 27474 time to create 1 rle with old method : 0.051116228103637695 length of segment : 155 time for calcul the mask position with numpy : 0.0004925727844238281 nb_pixel_total : 12251 time to create 1 rle with old method : 0.024140596389770508 length of segment : 115 time for calcul the mask position with numpy : 0.0016484260559082031 nb_pixel_total : 89774 time to create 1 rle with old method : 0.14740300178527832 length of segment : 359 time for calcul the mask position with numpy : 0.00034689903259277344 nb_pixel_total : 11014 time to create 1 rle with old method : 0.012509584426879883 length of segment : 122 time for calcul the mask position with numpy : 0.0006911754608154297 nb_pixel_total : 31250 time to create 1 rle with old method : 0.03629112243652344 length of segment : 192 time for calcul the mask position with numpy : 0.0008668899536132812 nb_pixel_total : 18915 time to create 1 rle with old method : 0.02136993408203125 length of segment : 231 time for calcul the mask position with numpy : 0.0002875328063964844 nb_pixel_total : 4320 time to create 1 rle with old method : 0.007660627365112305 length of segment : 52 time for calcul the mask position with numpy : 0.0004284381866455078 nb_pixel_total : 15291 time to create 1 rle with old method : 0.02593517303466797 length of segment : 92 time for calcul the mask position with numpy : 0.0005280971527099609 nb_pixel_total : 17521 time to create 1 rle with old method : 0.02046370506286621 length of segment : 165 time for calcul the mask position with numpy : 0.002234220504760742 nb_pixel_total : 101986 time to create 1 rle with old method : 0.12172532081604004 length of segment : 358 time for calcul the mask position with numpy : 0.023809432983398438 nb_pixel_total : 809305 time to create 1 rle with new method : 0.27332043647766113 length of segment : 835 time for calcul the mask position with numpy : 0.0018122196197509766 nb_pixel_total : 68142 time to create 1 rle with old method : 0.07770490646362305 length of segment : 339 time for calcul the mask position with numpy : 0.0008947849273681641 nb_pixel_total : 37878 time to create 1 rle with old method : 0.04340314865112305 length of segment : 230 time for calcul the mask position with numpy : 0.0017237663269042969 nb_pixel_total : 46120 time to create 1 rle with old method : 0.09398031234741211 length of segment : 255 time for calcul the mask position with numpy : 0.004738330841064453 nb_pixel_total : 62310 time to create 1 rle with old method : 0.11225485801696777 length of segment : 358 time for calcul the mask position with numpy : 0.005959749221801758 nb_pixel_total : 77470 time to create 1 rle with old method : 0.09952020645141602 length of segment : 443 time for calcul the mask position with numpy : 0.009777545928955078 nb_pixel_total : 218442 time to create 1 rle with new method : 0.013309717178344727 length of segment : 655 time for calcul the mask position with numpy : 0.0014252662658691406 nb_pixel_total : 22327 time to create 1 rle with old method : 0.029067039489746094 length of segment : 314 time for calcul the mask position with numpy : 0.002164602279663086 nb_pixel_total : 67283 time to create 1 rle with old method : 0.10789799690246582 length of segment : 295 time for calcul the mask position with numpy : 0.010255098342895508 nb_pixel_total : 215615 time to create 1 rle with new method : 0.02242112159729004 length of segment : 591 time for calcul the mask position with numpy : 0.003128528594970703 nb_pixel_total : 103927 time to create 1 rle with old method : 0.13743281364440918 length of segment : 608 time for calcul the mask position with numpy : 0.001399993896484375 nb_pixel_total : 43149 time to create 1 rle with old method : 0.04980778694152832 length of segment : 225 time for calcul the mask position with numpy : 0.008791685104370117 nb_pixel_total : 225733 time to create 1 rle with new method : 0.013106346130371094 length of segment : 555 time for calcul the mask position with numpy : 0.0008094310760498047 nb_pixel_total : 18440 time to create 1 rle with old method : 0.023966312408447266 length of segment : 200 time for calcul the mask position with numpy : 0.0007741451263427734 nb_pixel_total : 14705 time to create 1 rle with old method : 0.017451763153076172 length of segment : 204 time for calcul the mask position with numpy : 0.0038411617279052734 nb_pixel_total : 105549 time to create 1 rle with old method : 0.11950850486755371 length of segment : 556 time for calcul the mask position with numpy : 0.0039789676666259766 nb_pixel_total : 116399 time to create 1 rle with old method : 0.13081026077270508 length of segment : 634 time for calcul the mask position with numpy : 0.0018362998962402344 nb_pixel_total : 19708 time to create 1 rle with old method : 0.02326798439025879 length of segment : 245 time for calcul the mask position with numpy : 0.000926971435546875 nb_pixel_total : 19725 time to create 1 rle with old method : 0.022920846939086914 length of segment : 176 time for calcul the mask position with numpy : 0.00066375732421875 nb_pixel_total : 1888 time to create 1 rle with old method : 0.0024530887603759766 length of segment : 71 time for calcul the mask position with numpy : 0.0026488304138183594 nb_pixel_total : 89578 time to create 1 rle with old method : 0.10444045066833496 length of segment : 206 time for calcul the mask position with numpy : 0.0016124248504638672 nb_pixel_total : 53404 time to create 1 rle with old method : 0.060350656509399414 length of segment : 268 time for calcul the mask position with numpy : 0.0011758804321289062 nb_pixel_total : 22900 time to create 1 rle with old method : 0.03554892539978027 length of segment : 208 time for calcul the mask position with numpy : 0.005964994430541992 nb_pixel_total : 188839 time to create 1 rle with new method : 0.011579513549804688 length of segment : 573 time for calcul the mask position with numpy : 0.0007193088531494141 nb_pixel_total : 21155 time to create 1 rle with old method : 0.024578094482421875 length of segment : 202 time for calcul the mask position with numpy : 0.007160186767578125 nb_pixel_total : 313530 time to create 1 rle with new method : 0.011801958084106445 length of segment : 497 time for calcul the mask position with numpy : 0.006898403167724609 nb_pixel_total : 263674 time to create 1 rle with new method : 0.01026773452758789 length of segment : 561 time for calcul the mask position with numpy : 0.007311105728149414 nb_pixel_total : 241827 time to create 1 rle with new method : 0.010964155197143555 length of segment : 500 time for calcul the mask position with numpy : 0.00795292854309082 nb_pixel_total : 246691 time to create 1 rle with new method : 0.015167713165283203 length of segment : 685 time for calcul the mask position with numpy : 0.00684809684753418 nb_pixel_total : 168667 time to create 1 rle with new method : 0.010229825973510742 length of segment : 450 time for calcul the mask position with numpy : 0.023981571197509766 nb_pixel_total : 878963 time to create 1 rle with new method : 0.07947516441345215 length of segment : 2154 time for calcul the mask position with numpy : 0.005924224853515625 nb_pixel_total : 138024 time to create 1 rle with old method : 0.20458149909973145 length of segment : 565 time for calcul the mask position with numpy : 0.0008227825164794922 nb_pixel_total : 19986 time to create 1 rle with old method : 0.023081302642822266 length of segment : 162 time for calcul the mask position with numpy : 0.0006854534149169922 nb_pixel_total : 17490 time to create 1 rle with old method : 0.02041006088256836 length of segment : 185 time for calcul the mask position with numpy : 0.0022399425506591797 nb_pixel_total : 7338 time to create 1 rle with old method : 0.012351512908935547 length of segment : 155 time for calcul the mask position with numpy : 0.0008118152618408203 nb_pixel_total : 20745 time to create 1 rle with old method : 0.024026155471801758 length of segment : 194 time for calcul the mask position with numpy : 0.0030562877655029297 nb_pixel_total : 122044 time to create 1 rle with old method : 0.14544391632080078 length of segment : 347 time for calcul the mask position with numpy : 0.0010399818420410156 nb_pixel_total : 51049 time to create 1 rle with old method : 0.05863618850708008 length of segment : 114 time for calcul the mask position with numpy : 0.0007572174072265625 nb_pixel_total : 11430 time to create 1 rle with old method : 0.01387476921081543 length of segment : 171 time for calcul the mask position with numpy : 0.0008776187896728516 nb_pixel_total : 38660 time to create 1 rle with old method : 0.05746102333068848 length of segment : 257 time for calcul the mask position with numpy : 0.02654886245727539 nb_pixel_total : 1065704 time to create 1 rle with new method : 0.06624102592468262 length of segment : 1650 time for calcul the mask position with numpy : 0.0002892017364501953 nb_pixel_total : 2453 time to create 1 rle with old method : 0.004231929779052734 length of segment : 92 time for calcul the mask position with numpy : 0.001119375228881836 nb_pixel_total : 18813 time to create 1 rle with old method : 0.029663562774658203 length of segment : 131 time for calcul the mask position with numpy : 0.0004990100860595703 nb_pixel_total : 12299 time to create 1 rle with old method : 0.014209508895874023 length of segment : 149 time for calcul the mask position with numpy : 0.0005464553833007812 nb_pixel_total : 12815 time to create 1 rle with old method : 0.01516413688659668 length of segment : 210 time for calcul the mask position with numpy : 0.007334470748901367 nb_pixel_total : 226046 time to create 1 rle with new method : 0.010534524917602539 length of segment : 605 time for calcul the mask position with numpy : 0.002107381820678711 nb_pixel_total : 122894 time to create 1 rle with old method : 0.14005041122436523 length of segment : 308 time for calcul the mask position with numpy : 0.008245468139648438 nb_pixel_total : 315931 time to create 1 rle with new method : 0.020151138305664062 length of segment : 677 time for calcul the mask position with numpy : 0.016185283660888672 nb_pixel_total : 708858 time to create 1 rle with new method : 0.044646501541137695 length of segment : 796 time for calcul the mask position with numpy : 0.005658388137817383 nb_pixel_total : 150211 time to create 1 rle with new method : 0.011057138442993164 length of segment : 416 time for calcul the mask position with numpy : 0.002269744873046875 nb_pixel_total : 86645 time to create 1 rle with old method : 0.0978550910949707 length of segment : 360 time for calcul the mask position with numpy : 0.005604743957519531 nb_pixel_total : 287306 time to create 1 rle with new method : 0.008751153945922852 length of segment : 704 time for calcul the mask position with numpy : 0.0024900436401367188 nb_pixel_total : 96144 time to create 1 rle with old method : 0.10732746124267578 length of segment : 422 time for calcul the mask position with numpy : 0.0008897781372070312 nb_pixel_total : 26441 time to create 1 rle with old method : 0.03076028823852539 length of segment : 240 time for calcul the mask position with numpy : 0.0012483596801757812 nb_pixel_total : 48946 time to create 1 rle with old method : 0.05731081962585449 length of segment : 230 time for calcul the mask position with numpy : 0.0013451576232910156 nb_pixel_total : 38258 time to create 1 rle with old method : 0.04381513595581055 length of segment : 300 time for calcul the mask position with numpy : 0.017978191375732422 nb_pixel_total : 841086 time to create 1 rle with new method : 0.0704796314239502 length of segment : 1303 time for calcul the mask position with numpy : 0.0002956390380859375 nb_pixel_total : 6077 time to create 1 rle with old method : 0.007285356521606445 length of segment : 74 time for calcul the mask position with numpy : 0.0005815029144287109 nb_pixel_total : 19652 time to create 1 rle with old method : 0.023133516311645508 length of segment : 177 time for calcul the mask position with numpy : 0.0004444122314453125 nb_pixel_total : 11097 time to create 1 rle with old method : 0.013989448547363281 length of segment : 286 time for calcul the mask position with numpy : 0.0006489753723144531 nb_pixel_total : 39043 time to create 1 rle with old method : 0.04747343063354492 length of segment : 225 time for calcul the mask position with numpy : 0.0005452632904052734 nb_pixel_total : 17223 time to create 1 rle with old method : 0.021027088165283203 length of segment : 135 time for calcul the mask position with numpy : 0.00043964385986328125 nb_pixel_total : 7900 time to create 1 rle with old method : 0.009309530258178711 length of segment : 142 time for calcul the mask position with numpy : 0.0018563270568847656 nb_pixel_total : 42791 time to create 1 rle with old method : 0.053612709045410156 length of segment : 603 time for calcul the mask position with numpy : 0.00015854835510253906 nb_pixel_total : 3463 time to create 1 rle with old method : 0.004274606704711914 length of segment : 51 time for calcul the mask position with numpy : 0.0010294914245605469 nb_pixel_total : 36776 time to create 1 rle with old method : 0.041792869567871094 length of segment : 528 time for calcul the mask position with numpy : 0.008034944534301758 nb_pixel_total : 466449 time to create 1 rle with new method : 0.024703025817871094 length of segment : 544 time for calcul the mask position with numpy : 0.17781925201416016 nb_pixel_total : 3955861 time to create 1 rle with new method : 0.8681864738464355 length of segment : 4667 time for calcul the mask position with numpy : 0.00025844573974609375 nb_pixel_total : 11221 time to create 1 rle with old method : 0.012686491012573242 length of segment : 161 time for calcul the mask position with numpy : 0.0032176971435546875 nb_pixel_total : 257228 time to create 1 rle with new method : 0.012265443801879883 length of segment : 518 time for calcul the mask position with numpy : 0.7094945907592773 nb_pixel_total : 4183143 time to create 1 rle with new method : 0.591193675994873 length of segment : 3364 time for calcul the mask position with numpy : 0.0017056465148925781 nb_pixel_total : 105846 time to create 1 rle with old method : 0.11739420890808105 length of segment : 375 time for calcul the mask position with numpy : 0.03655529022216797 nb_pixel_total : 1159297 time to create 1 rle with new method : 0.2150571346282959 length of segment : 1015 time for calcul the mask position with numpy : 0.009999513626098633 nb_pixel_total : 583469 time to create 1 rle with new method : 0.017060518264770508 length of segment : 1136 time for calcul the mask position with numpy : 0.0049021244049072266 nb_pixel_total : 261520 time to create 1 rle with new method : 0.013601541519165039 length of segment : 1653 time for calcul the mask position with numpy : 0.006825923919677734 nb_pixel_total : 464750 time to create 1 rle with new method : 0.013486862182617188 length of segment : 1280 time for calcul the mask position with numpy : 0.007965326309204102 nb_pixel_total : 466240 time to create 1 rle with new method : 0.020868539810180664 length of segment : 1354 time spent for convertir_results : 18.146586656570435 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 98 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 68.88136863708496 time spend to save output : 0.11795377731323242 total time spend for step 1 : 68.9993224143982 step2:mask_detect Tue May 6 13:02:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3831 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 13:02:43.472112: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 13:02:43.480751: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 13:02:43.482394: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd714000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 13:02:43.482451: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 13:02:43.485529: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 13:02:43.743338: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa953110 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 13:02:43.743451: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 13:02:43.744197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:02:43.744731: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:02:43.748256: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:02:43.751397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:02:43.755415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:02:43.759180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:02:43.760679: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:02:43.772058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:02:43.773302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:02:43.773388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:02:43.773993: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 13:02:43.774015: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 13:02:43.774025: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 13:02:43.775189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 13:02:43.908224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:02:43.908379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:02:43.908400: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:02:43.908418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:02:43.908435: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:02:43.908452: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:02:43.908470: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:02:43.908492: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:02:43.909554: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:02:43.910893: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 13:02:43.910959: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 13:02:43.910986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:02:43.911012: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 13:02:43.911035: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 13:02:43.911060: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 13:02:43.911084: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 13:02:43.911132: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:02:43.912243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 13:02:43.912299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 13:02:43.912313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 13:02:43.912324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 13:02:43.913510: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 13:02:56.349334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 13:02:56.504006: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 13:02:57.921554: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.658536: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-06 13:02:58.718115: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.718199: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.719319: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.719351: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.726255: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.726285: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.732504: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.732523: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.733093: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.733107: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.753812: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.753888: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.753905: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-06 13:02:58.754844: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.754864: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.755865: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.755889: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.762712: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.762733: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 13:02:58.763324: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.763921: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.764499: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.772273: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.772887: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.790500: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.791362: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.792154: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.792862: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.796898: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.797483: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.798094: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.798722: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.799796: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.809880: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.810511: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.819588: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.820222: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.820858: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.821484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.822117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 13:02:58.822743: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 6 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 65 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 60 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 54 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 40 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 7 Detection mask done ! Trying to reset tf kernel 834612 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2638 tf kernel not reseted sub process len(results) : 6 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 6 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3831 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.014188051223754883 nb_pixel_total : 309170 time to create 1 rle with new method : 0.027321815490722656 length of segment : 395 time for calcul the mask position with numpy : 0.0033402442932128906 nb_pixel_total : 143172 time to create 1 rle with old method : 0.2042374610900879 length of segment : 549 time for calcul the mask position with numpy : 0.0008368492126464844 nb_pixel_total : 39205 time to create 1 rle with old method : 0.045944929122924805 length of segment : 163 time for calcul the mask position with numpy : 0.0017044544219970703 nb_pixel_total : 86333 time to create 1 rle with old method : 0.09793663024902344 length of segment : 397 time for calcul the mask position with numpy : 0.0013356208801269531 nb_pixel_total : 44994 time to create 1 rle with old method : 0.051721811294555664 length of segment : 242 time for calcul the mask position with numpy : 0.0022919178009033203 nb_pixel_total : 92612 time to create 1 rle with old method : 0.1300797462463379 length of segment : 291 time for calcul the mask position with numpy : 0.004631519317626953 nb_pixel_total : 289881 time to create 1 rle with new method : 0.011233806610107422 length of segment : 609 time for calcul the mask position with numpy : 0.0016357898712158203 nb_pixel_total : 60747 time to create 1 rle with old method : 0.0679924488067627 length of segment : 371 time for calcul the mask position with numpy : 0.005011320114135742 nb_pixel_total : 222022 time to create 1 rle with new method : 0.015663862228393555 length of segment : 944 time for calcul the mask position with numpy : 0.0037539005279541016 nb_pixel_total : 173721 time to create 1 rle with new method : 0.010898590087890625 length of segment : 518 time for calcul the mask position with numpy : 0.002873659133911133 nb_pixel_total : 159483 time to create 1 rle with new method : 0.00847482681274414 length of segment : 426 time for calcul the mask position with numpy : 0.0015223026275634766 nb_pixel_total : 94823 time to create 1 rle with old method : 0.1044471263885498 length of segment : 291 time for calcul the mask position with numpy : 0.0022323131561279297 nb_pixel_total : 76494 time to create 1 rle with old method : 0.08481717109680176 length of segment : 714 time for calcul the mask position with numpy : 0.0005466938018798828 nb_pixel_total : 23682 time to create 1 rle with old method : 0.02755117416381836 length of segment : 177 time for calcul the mask position with numpy : 0.0050122737884521484 nb_pixel_total : 251564 time to create 1 rle with new method : 0.015667438507080078 length of segment : 569 time for calcul the mask position with numpy : 0.013415098190307617 nb_pixel_total : 339805 time to create 1 rle with new method : 0.3968634605407715 length of segment : 765 time for calcul the mask position with numpy : 0.0065310001373291016 nb_pixel_total : 340884 time to create 1 rle with new method : 0.016156673431396484 length of segment : 626 time for calcul the mask position with numpy : 0.001241922378540039 nb_pixel_total : 57833 time to create 1 rle with old method : 0.06643939018249512 length of segment : 269 time for calcul the mask position with numpy : 0.001435995101928711 nb_pixel_total : 52340 time to create 1 rle with old method : 0.06748723983764648 length of segment : 269 time for calcul the mask position with numpy : 0.0004756450653076172 nb_pixel_total : 11681 time to create 1 rle with old method : 0.013358354568481445 length of segment : 149 time for calcul the mask position with numpy : 0.004632234573364258 nb_pixel_total : 245176 time to create 1 rle with new method : 0.013325214385986328 length of segment : 556 time for calcul the mask position with numpy : 0.0005464553833007812 nb_pixel_total : 14293 time to create 1 rle with old method : 0.016138315200805664 length of segment : 203 time for calcul the mask position with numpy : 0.0004227161407470703 nb_pixel_total : 14980 time to create 1 rle with old method : 0.017114877700805664 length of segment : 171 time for calcul the mask position with numpy : 0.001394510269165039 nb_pixel_total : 53817 time to create 1 rle with old method : 0.06067204475402832 length of segment : 444 time for calcul the mask position with numpy : 0.014127254486083984 nb_pixel_total : 529022 time to create 1 rle with new method : 0.04283881187438965 length of segment : 1107 time for calcul the mask position with numpy : 0.0027670860290527344 nb_pixel_total : 144581 time to create 1 rle with old method : 0.16044974327087402 length of segment : 526 time for calcul the mask position with numpy : 0.0007963180541992188 nb_pixel_total : 37146 time to create 1 rle with old method : 0.04024195671081543 length of segment : 233 time for calcul the mask position with numpy : 0.002590179443359375 nb_pixel_total : 139632 time to create 1 rle with old method : 0.1548633575439453 length of segment : 683 time for calcul the mask position with numpy : 0.004580497741699219 nb_pixel_total : 228029 time to create 1 rle with new method : 0.011833667755126953 length of segment : 527 time for calcul the mask position with numpy : 0.0018124580383300781 nb_pixel_total : 44147 time to create 1 rle with old method : 0.0503544807434082 length of segment : 738 time for calcul the mask position with numpy : 0.013702630996704102 nb_pixel_total : 646590 time to create 1 rle with new method : 0.03757786750793457 length of segment : 1015 time for calcul the mask position with numpy : 0.0004405975341796875 nb_pixel_total : 1150 time to create 1 rle with old method : 0.002040863037109375 length of segment : 76 time for calcul the mask position with numpy : 0.0014355182647705078 nb_pixel_total : 60891 time to create 1 rle with old method : 0.06942367553710938 length of segment : 384 time for calcul the mask position with numpy : 0.0008573532104492188 nb_pixel_total : 35188 time to create 1 rle with old method : 0.039144277572631836 length of segment : 233 time for calcul the mask position with numpy : 0.0012869834899902344 nb_pixel_total : 46743 time to create 1 rle with old method : 0.06163930892944336 length of segment : 268 time for calcul the mask position with numpy : 0.002543210983276367 nb_pixel_total : 103971 time to create 1 rle with old method : 0.11560893058776855 length of segment : 551 time for calcul the mask position with numpy : 0.0012128353118896484 nb_pixel_total : 56157 time to create 1 rle with old method : 0.08061480522155762 length of segment : 231 time for calcul the mask position with numpy : 0.0029785633087158203 nb_pixel_total : 157659 time to create 1 rle with new method : 0.008337974548339844 length of segment : 333 time for calcul the mask position with numpy : 0.005779266357421875 nb_pixel_total : 148173 time to create 1 rle with old method : 0.16877055168151855 length of segment : 450 time for calcul the mask position with numpy : 0.004247426986694336 nb_pixel_total : 126880 time to create 1 rle with old method : 0.16892528533935547 length of segment : 521 time for calcul the mask position with numpy : 0.0035462379455566406 nb_pixel_total : 158266 time to create 1 rle with new method : 0.008985757827758789 length of segment : 474 time for calcul the mask position with numpy : 0.002872943878173828 nb_pixel_total : 107173 time to create 1 rle with old method : 0.12044405937194824 length of segment : 297 time for calcul the mask position with numpy : 0.0004181861877441406 nb_pixel_total : 2627 time to create 1 rle with old method : 0.00321197509765625 length of segment : 82 time for calcul the mask position with numpy : 0.014483451843261719 nb_pixel_total : 472187 time to create 1 rle with new method : 0.05822420120239258 length of segment : 1398 time for calcul the mask position with numpy : 0.018592119216918945 nb_pixel_total : 705254 time to create 1 rle with new method : 0.1324326992034912 length of segment : 931 time for calcul the mask position with numpy : 0.0030608177185058594 nb_pixel_total : 129569 time to create 1 rle with old method : 0.14316129684448242 length of segment : 443 time for calcul the mask position with numpy : 0.0020246505737304688 nb_pixel_total : 86029 time to create 1 rle with old method : 0.09563374519348145 length of segment : 304 time for calcul the mask position with numpy : 0.002424478530883789 nb_pixel_total : 124255 time to create 1 rle with old method : 0.13996243476867676 length of segment : 607 time for calcul the mask position with numpy : 0.002742767333984375 nb_pixel_total : 107541 time to create 1 rle with old method : 0.12133455276489258 length of segment : 323 time for calcul the mask position with numpy : 0.002727985382080078 nb_pixel_total : 142096 time to create 1 rle with old method : 0.15945935249328613 length of segment : 404 time for calcul the mask position with numpy : 0.0035529136657714844 nb_pixel_total : 158252 time to create 1 rle with new method : 0.010854721069335938 length of segment : 445 time for calcul the mask position with numpy : 0.0015840530395507812 nb_pixel_total : 58245 time to create 1 rle with old method : 0.07908415794372559 length of segment : 253 time for calcul the mask position with numpy : 0.012097597122192383 nb_pixel_total : 494247 time to create 1 rle with new method : 0.3007495403289795 length of segment : 748 time for calcul the mask position with numpy : 0.0014660358428955078 nb_pixel_total : 16985 time to create 1 rle with old method : 0.02074456214904785 length of segment : 192 time for calcul the mask position with numpy : 0.0012831687927246094 nb_pixel_total : 36903 time to create 1 rle with old method : 0.04294896125793457 length of segment : 228 time for calcul the mask position with numpy : 0.0021207332611083984 nb_pixel_total : 104165 time to create 1 rle with old method : 0.11767244338989258 length of segment : 435 time for calcul the mask position with numpy : 0.0021936893463134766 nb_pixel_total : 103294 time to create 1 rle with old method : 0.1181786060333252 length of segment : 261 time for calcul the mask position with numpy : 0.008680582046508789 nb_pixel_total : 301321 time to create 1 rle with new method : 0.026384592056274414 length of segment : 1103 time for calcul the mask position with numpy : 0.0010933876037597656 nb_pixel_total : 59977 time to create 1 rle with old method : 0.06876134872436523 length of segment : 246 time for calcul the mask position with numpy : 0.0012629032135009766 nb_pixel_total : 73857 time to create 1 rle with old method : 0.09363746643066406 length of segment : 339 time for calcul the mask position with numpy : 0.0032422542572021484 nb_pixel_total : 154042 time to create 1 rle with new method : 0.009238243103027344 length of segment : 437 time for calcul the mask position with numpy : 0.0034635066986083984 nb_pixel_total : 171119 time to create 1 rle with new method : 0.010488510131835938 length of segment : 470 time for calcul the mask position with numpy : 0.0016162395477294922 nb_pixel_total : 83763 time to create 1 rle with old method : 0.09355902671813965 length of segment : 309 time for calcul the mask position with numpy : 0.00229644775390625 nb_pixel_total : 94518 time to create 1 rle with old method : 0.10827040672302246 length of segment : 306 time for calcul the mask position with numpy : 0.003032684326171875 nb_pixel_total : 160709 time to create 1 rle with new method : 0.008225440979003906 length of segment : 413 time for calcul the mask position with numpy : 0.0017020702362060547 nb_pixel_total : 81337 time to create 1 rle with old method : 0.11611437797546387 length of segment : 232 time for calcul the mask position with numpy : 0.0020694732666015625 nb_pixel_total : 91578 time to create 1 rle with old method : 0.10998201370239258 length of segment : 387 time for calcul the mask position with numpy : 0.0011699199676513672 nb_pixel_total : 44325 time to create 1 rle with old method : 0.049924373626708984 length of segment : 327 time for calcul the mask position with numpy : 0.0006778240203857422 nb_pixel_total : 35966 time to create 1 rle with old method : 0.04120683670043945 length of segment : 183 time for calcul the mask position with numpy : 0.002149343490600586 nb_pixel_total : 122006 time to create 1 rle with old method : 0.13511061668395996 length of segment : 623 time for calcul the mask position with numpy : 0.0006530284881591797 nb_pixel_total : 22029 time to create 1 rle with old method : 0.0254971981048584 length of segment : 156 time for calcul the mask position with numpy : 0.00504302978515625 nb_pixel_total : 178954 time to create 1 rle with new method : 0.013936996459960938 length of segment : 662 time for calcul the mask position with numpy : 0.0034265518188476562 nb_pixel_total : 167572 time to create 1 rle with new method : 0.009810686111450195 length of segment : 261 time for calcul the mask position with numpy : 0.001844167709350586 nb_pixel_total : 46811 time to create 1 rle with old method : 0.06215047836303711 length of segment : 391 time for calcul the mask position with numpy : 0.0016968250274658203 nb_pixel_total : 75052 time to create 1 rle with old method : 0.08583259582519531 length of segment : 324 time for calcul the mask position with numpy : 0.005684852600097656 nb_pixel_total : 312417 time to create 1 rle with new method : 0.0157015323638916 length of segment : 715 time for calcul the mask position with numpy : 0.0009531974792480469 nb_pixel_total : 36536 time to create 1 rle with old method : 0.041704654693603516 length of segment : 284 time for calcul the mask position with numpy : 0.002496480941772461 nb_pixel_total : 97795 time to create 1 rle with old method : 0.10932469367980957 length of segment : 309 time for calcul the mask position with numpy : 0.004840850830078125 nb_pixel_total : 220749 time to create 1 rle with new method : 0.013653278350830078 length of segment : 439 time for calcul the mask position with numpy : 0.0006029605865478516 nb_pixel_total : 18233 time to create 1 rle with old method : 0.021032094955444336 length of segment : 182 time for calcul the mask position with numpy : 0.0046041011810302734 nb_pixel_total : 168651 time to create 1 rle with new method : 0.0173952579498291 length of segment : 486 time for calcul the mask position with numpy : 0.002704620361328125 nb_pixel_total : 86135 time to create 1 rle with old method : 0.15624761581420898 length of segment : 375 time for calcul the mask position with numpy : 0.0031614303588867188 nb_pixel_total : 81658 time to create 1 rle with old method : 0.16142582893371582 length of segment : 341 time for calcul the mask position with numpy : 0.00048542022705078125 nb_pixel_total : 14171 time to create 1 rle with old method : 0.026204347610473633 length of segment : 157 time for calcul the mask position with numpy : 0.001004934310913086 nb_pixel_total : 35206 time to create 1 rle with old method : 0.04887986183166504 length of segment : 192 time for calcul the mask position with numpy : 0.003595113754272461 nb_pixel_total : 99763 time to create 1 rle with old method : 0.15423297882080078 length of segment : 433 time for calcul the mask position with numpy : 0.0005557537078857422 nb_pixel_total : 24601 time to create 1 rle with old method : 0.027999162673950195 length of segment : 124 time for calcul the mask position with numpy : 0.017029523849487305 nb_pixel_total : 577026 time to create 1 rle with new method : 0.0462336540222168 length of segment : 998 time for calcul the mask position with numpy : 0.0009679794311523438 nb_pixel_total : 55017 time to create 1 rle with old method : 0.06446981430053711 length of segment : 233 time for calcul the mask position with numpy : 0.0013849735260009766 nb_pixel_total : 66553 time to create 1 rle with old method : 0.08193182945251465 length of segment : 293 time for calcul the mask position with numpy : 0.0036928653717041016 nb_pixel_total : 199736 time to create 1 rle with new method : 0.009629011154174805 length of segment : 435 time for calcul the mask position with numpy : 0.001608133316040039 nb_pixel_total : 75075 time to create 1 rle with old method : 0.08467245101928711 length of segment : 260 time for calcul the mask position with numpy : 0.006753206253051758 nb_pixel_total : 237688 time to create 1 rle with new method : 0.018046140670776367 length of segment : 707 time for calcul the mask position with numpy : 0.009157896041870117 nb_pixel_total : 407040 time to create 1 rle with new method : 0.02441716194152832 length of segment : 1208 time for calcul the mask position with numpy : 0.007058143615722656 nb_pixel_total : 376590 time to create 1 rle with new method : 0.01650261878967285 length of segment : 607 time for calcul the mask position with numpy : 0.001760721206665039 nb_pixel_total : 78774 time to create 1 rle with old method : 0.08852839469909668 length of segment : 399 time for calcul the mask position with numpy : 0.0006377696990966797 nb_pixel_total : 22862 time to create 1 rle with old method : 0.031781911849975586 length of segment : 120 time for calcul the mask position with numpy : 0.005646228790283203 nb_pixel_total : 59010 time to create 1 rle with old method : 0.11463785171508789 length of segment : 432 time for calcul the mask position with numpy : 0.011056900024414062 nb_pixel_total : 518006 time to create 1 rle with new method : 0.03044581413269043 length of segment : 941 time for calcul the mask position with numpy : 0.001547098159790039 nb_pixel_total : 62285 time to create 1 rle with old method : 0.07080936431884766 length of segment : 380 time for calcul the mask position with numpy : 0.003335237503051758 nb_pixel_total : 191465 time to create 1 rle with new method : 0.008779287338256836 length of segment : 427 time for calcul the mask position with numpy : 0.0026929378509521484 nb_pixel_total : 172200 time to create 1 rle with new method : 0.007235288619995117 length of segment : 444 time for calcul the mask position with numpy : 0.0006318092346191406 nb_pixel_total : 20944 time to create 1 rle with old method : 0.02521681785583496 length of segment : 178 time for calcul the mask position with numpy : 0.002809286117553711 nb_pixel_total : 111422 time to create 1 rle with old method : 0.14141273498535156 length of segment : 607 time for calcul the mask position with numpy : 0.031165599822998047 nb_pixel_total : 726023 time to create 1 rle with new method : 0.5371472835540771 length of segment : 1359 time for calcul the mask position with numpy : 0.0011966228485107422 nb_pixel_total : 53060 time to create 1 rle with old method : 0.06132674217224121 length of segment : 247 time for calcul the mask position with numpy : 0.007531166076660156 nb_pixel_total : 288006 time to create 1 rle with new method : 0.02379894256591797 length of segment : 805 time for calcul the mask position with numpy : 0.013797760009765625 nb_pixel_total : 759672 time to create 1 rle with new method : 0.0608518123626709 length of segment : 811 time for calcul the mask position with numpy : 0.0008220672607421875 nb_pixel_total : 17517 time to create 1 rle with old method : 0.020071983337402344 length of segment : 308 time for calcul the mask position with numpy : 0.003311634063720703 nb_pixel_total : 164390 time to create 1 rle with new method : 0.009515762329101562 length of segment : 692 time for calcul the mask position with numpy : 0.0007178783416748047 nb_pixel_total : 34012 time to create 1 rle with old method : 0.03834056854248047 length of segment : 173 time for calcul the mask position with numpy : 0.0046651363372802734 nb_pixel_total : 150785 time to create 1 rle with new method : 0.015569448471069336 length of segment : 814 time for calcul the mask position with numpy : 0.002605915069580078 nb_pixel_total : 122309 time to create 1 rle with old method : 0.13590693473815918 length of segment : 292 time for calcul the mask position with numpy : 0.0020139217376708984 nb_pixel_total : 65466 time to create 1 rle with old method : 0.0729377269744873 length of segment : 569 time for calcul the mask position with numpy : 0.001500844955444336 nb_pixel_total : 60680 time to create 1 rle with old method : 0.06669116020202637 length of segment : 428 time for calcul the mask position with numpy : 0.001430511474609375 nb_pixel_total : 60661 time to create 1 rle with old method : 0.06760287284851074 length of segment : 372 time for calcul the mask position with numpy : 0.0007758140563964844 nb_pixel_total : 31637 time to create 1 rle with old method : 0.034860849380493164 length of segment : 227 time for calcul the mask position with numpy : 0.001955270767211914 nb_pixel_total : 108198 time to create 1 rle with old method : 0.11908197402954102 length of segment : 586 time for calcul the mask position with numpy : 0.0025391578674316406 nb_pixel_total : 134263 time to create 1 rle with old method : 0.1451091766357422 length of segment : 361 time for calcul the mask position with numpy : 0.002557516098022461 nb_pixel_total : 121613 time to create 1 rle with old method : 0.13813495635986328 length of segment : 607 time for calcul the mask position with numpy : 0.006802082061767578 nb_pixel_total : 376482 time to create 1 rle with new method : 0.019888877868652344 length of segment : 600 time for calcul the mask position with numpy : 0.0007369518280029297 nb_pixel_total : 32377 time to create 1 rle with old method : 0.03724265098571777 length of segment : 185 time for calcul the mask position with numpy : 0.0009093284606933594 nb_pixel_total : 40031 time to create 1 rle with old method : 0.04779481887817383 length of segment : 237 time for calcul the mask position with numpy : 0.0034325122833251953 nb_pixel_total : 146664 time to create 1 rle with old method : 0.16078782081604004 length of segment : 388 time for calcul the mask position with numpy : 0.00410771369934082 nb_pixel_total : 184150 time to create 1 rle with new method : 0.012046098709106445 length of segment : 529 time for calcul the mask position with numpy : 0.0013413429260253906 nb_pixel_total : 56019 time to create 1 rle with old method : 0.07405304908752441 length of segment : 244 time for calcul the mask position with numpy : 0.025554895401000977 nb_pixel_total : 1403048 time to create 1 rle with new method : 0.1288890838623047 length of segment : 1487 time for calcul the mask position with numpy : 0.002621889114379883 nb_pixel_total : 130529 time to create 1 rle with old method : 0.14092016220092773 length of segment : 463 time for calcul the mask position with numpy : 0.0010671615600585938 nb_pixel_total : 61720 time to create 1 rle with old method : 0.06887030601501465 length of segment : 279 time for calcul the mask position with numpy : 0.005872249603271484 nb_pixel_total : 269311 time to create 1 rle with new method : 0.016075849533081055 length of segment : 845 time for calcul the mask position with numpy : 0.0016050338745117188 nb_pixel_total : 99503 time to create 1 rle with old method : 0.10798120498657227 length of segment : 358 time for calcul the mask position with numpy : 0.0005934238433837891 nb_pixel_total : 19850 time to create 1 rle with old method : 0.024047136306762695 length of segment : 178 time for calcul the mask position with numpy : 0.002162456512451172 nb_pixel_total : 140790 time to create 1 rle with old method : 0.1555490493774414 length of segment : 532 time for calcul the mask position with numpy : 0.001567840576171875 nb_pixel_total : 84855 time to create 1 rle with old method : 0.09537887573242188 length of segment : 359 time for calcul the mask position with numpy : 0.0010955333709716797 nb_pixel_total : 49217 time to create 1 rle with old method : 0.05557680130004883 length of segment : 250 time for calcul the mask position with numpy : 0.001194000244140625 nb_pixel_total : 74587 time to create 1 rle with old method : 0.08359503746032715 length of segment : 365 time for calcul the mask position with numpy : 0.01386260986328125 nb_pixel_total : 480338 time to create 1 rle with new method : 0.06560564041137695 length of segment : 1098 time for calcul the mask position with numpy : 0.0005393028259277344 nb_pixel_total : 20066 time to create 1 rle with old method : 0.022986173629760742 length of segment : 192 time for calcul the mask position with numpy : 0.005150794982910156 nb_pixel_total : 331203 time to create 1 rle with new method : 0.011859655380249023 length of segment : 445 time for calcul the mask position with numpy : 0.0013003349304199219 nb_pixel_total : 54511 time to create 1 rle with old method : 0.06046009063720703 length of segment : 300 time for calcul the mask position with numpy : 0.0011355876922607422 nb_pixel_total : 55475 time to create 1 rle with old method : 0.060291290283203125 length of segment : 223 time for calcul the mask position with numpy : 0.0019497871398925781 nb_pixel_total : 86495 time to create 1 rle with old method : 0.0953826904296875 length of segment : 449 time for calcul the mask position with numpy : 0.005809307098388672 nb_pixel_total : 269354 time to create 1 rle with new method : 0.016927480697631836 length of segment : 713 time for calcul the mask position with numpy : 0.0010733604431152344 nb_pixel_total : 48684 time to create 1 rle with old method : 0.052568912506103516 length of segment : 271 time for calcul the mask position with numpy : 0.0014493465423583984 nb_pixel_total : 54608 time to create 1 rle with old method : 0.05951547622680664 length of segment : 378 time for calcul the mask position with numpy : 0.005920886993408203 nb_pixel_total : 332589 time to create 1 rle with new method : 0.014419794082641602 length of segment : 493 time for calcul the mask position with numpy : 0.0022475719451904297 nb_pixel_total : 96245 time to create 1 rle with old method : 0.10398173332214355 length of segment : 471 time for calcul the mask position with numpy : 0.0013413429260253906 nb_pixel_total : 61327 time to create 1 rle with old method : 0.06967520713806152 length of segment : 349 time for calcul the mask position with numpy : 0.0056264400482177734 nb_pixel_total : 280213 time to create 1 rle with new method : 0.014343500137329102 length of segment : 704 time for calcul the mask position with numpy : 0.007787466049194336 nb_pixel_total : 442247 time to create 1 rle with new method : 0.01871490478515625 length of segment : 739 time for calcul the mask position with numpy : 0.004315614700317383 nb_pixel_total : 226670 time to create 1 rle with new method : 0.011111736297607422 length of segment : 839 time for calcul the mask position with numpy : 0.0008714199066162109 nb_pixel_total : 28777 time to create 1 rle with old method : 0.040650367736816406 length of segment : 215 time for calcul the mask position with numpy : 0.004625082015991211 nb_pixel_total : 211464 time to create 1 rle with new method : 0.012341976165771484 length of segment : 481 time for calcul the mask position with numpy : 0.009867191314697266 nb_pixel_total : 515247 time to create 1 rle with new method : 0.01876688003540039 length of segment : 666 time for calcul the mask position with numpy : 0.0005638599395751953 nb_pixel_total : 17657 time to create 1 rle with old method : 0.02590322494506836 length of segment : 148 time for calcul the mask position with numpy : 0.000576019287109375 nb_pixel_total : 20334 time to create 1 rle with old method : 0.024812936782836914 length of segment : 169 time for calcul the mask position with numpy : 0.0010662078857421875 nb_pixel_total : 39854 time to create 1 rle with old method : 0.0508112907409668 length of segment : 248 time for calcul the mask position with numpy : 0.001131296157836914 nb_pixel_total : 41041 time to create 1 rle with old method : 0.0467677116394043 length of segment : 328 time for calcul the mask position with numpy : 0.0007343292236328125 nb_pixel_total : 35843 time to create 1 rle with old method : 0.057698965072631836 length of segment : 173 time for calcul the mask position with numpy : 0.0005929470062255859 nb_pixel_total : 28586 time to create 1 rle with old method : 0.03237581253051758 length of segment : 235 time for calcul the mask position with numpy : 0.007183074951171875 nb_pixel_total : 390086 time to create 1 rle with new method : 0.017862319946289062 length of segment : 882 time for calcul the mask position with numpy : 0.0012738704681396484 nb_pixel_total : 57338 time to create 1 rle with old method : 0.06372714042663574 length of segment : 313 time for calcul the mask position with numpy : 0.005927085876464844 nb_pixel_total : 261832 time to create 1 rle with new method : 0.01965951919555664 length of segment : 933 time for calcul the mask position with numpy : 0.0004992485046386719 nb_pixel_total : 6420 time to create 1 rle with old method : 0.007891178131103516 length of segment : 158 time for calcul the mask position with numpy : 0.0008072853088378906 nb_pixel_total : 39771 time to create 1 rle with old method : 0.045495033264160156 length of segment : 183 time for calcul the mask position with numpy : 0.006290912628173828 nb_pixel_total : 307907 time to create 1 rle with new method : 0.018771886825561523 length of segment : 1143 time for calcul the mask position with numpy : 0.0006666183471679688 nb_pixel_total : 33731 time to create 1 rle with old method : 0.03853249549865723 length of segment : 157 time for calcul the mask position with numpy : 0.0032927989959716797 nb_pixel_total : 175107 time to create 1 rle with new method : 0.008321285247802734 length of segment : 656 time for calcul the mask position with numpy : 0.0022001266479492188 nb_pixel_total : 139314 time to create 1 rle with old method : 0.14890384674072266 length of segment : 305 time for calcul the mask position with numpy : 0.0022411346435546875 nb_pixel_total : 118848 time to create 1 rle with old method : 0.13419628143310547 length of segment : 445 time for calcul the mask position with numpy : 0.0016813278198242188 nb_pixel_total : 99242 time to create 1 rle with old method : 0.11205887794494629 length of segment : 270 time for calcul the mask position with numpy : 0.0008373260498046875 nb_pixel_total : 35049 time to create 1 rle with old method : 0.04099297523498535 length of segment : 211 time for calcul the mask position with numpy : 0.002351999282836914 nb_pixel_total : 122945 time to create 1 rle with old method : 0.13907480239868164 length of segment : 419 time for calcul the mask position with numpy : 0.0011749267578125 nb_pixel_total : 58509 time to create 1 rle with old method : 0.06660199165344238 length of segment : 131 time for calcul the mask position with numpy : 0.001165628433227539 nb_pixel_total : 60087 time to create 1 rle with old method : 0.06711459159851074 length of segment : 322 time for calcul the mask position with numpy : 0.0015888214111328125 nb_pixel_total : 79582 time to create 1 rle with old method : 0.08830928802490234 length of segment : 533 time for calcul the mask position with numpy : 0.001962423324584961 nb_pixel_total : 108800 time to create 1 rle with old method : 0.1184227466583252 length of segment : 410 time for calcul the mask position with numpy : 0.0031092166900634766 nb_pixel_total : 135045 time to create 1 rle with old method : 0.14660000801086426 length of segment : 512 time for calcul the mask position with numpy : 0.0015139579772949219 nb_pixel_total : 56838 time to create 1 rle with old method : 0.06422734260559082 length of segment : 375 time for calcul the mask position with numpy : 0.0019497871398925781 nb_pixel_total : 109134 time to create 1 rle with old method : 0.12259554862976074 length of segment : 300 time for calcul the mask position with numpy : 0.003412008285522461 nb_pixel_total : 185752 time to create 1 rle with new method : 0.00996541976928711 length of segment : 632 time for calcul the mask position with numpy : 0.004216194152832031 nb_pixel_total : 161473 time to create 1 rle with new method : 0.013604164123535156 length of segment : 607 time for calcul the mask position with numpy : 0.0022840499877929688 nb_pixel_total : 123558 time to create 1 rle with old method : 0.13579249382019043 length of segment : 453 time for calcul the mask position with numpy : 0.0011131763458251953 nb_pixel_total : 42072 time to create 1 rle with old method : 0.06743383407592773 length of segment : 281 time for calcul the mask position with numpy : 0.0011208057403564453 nb_pixel_total : 53829 time to create 1 rle with old method : 0.06639671325683594 length of segment : 217 time for calcul the mask position with numpy : 0.0005691051483154297 nb_pixel_total : 16344 time to create 1 rle with old method : 0.022429943084716797 length of segment : 124 time for calcul the mask position with numpy : 0.008546590805053711 nb_pixel_total : 299957 time to create 1 rle with new method : 0.024094343185424805 length of segment : 895 time for calcul the mask position with numpy : 0.0025272369384765625 nb_pixel_total : 153086 time to create 1 rle with new method : 0.00574803352355957 length of segment : 361 time for calcul the mask position with numpy : 0.0010116100311279297 nb_pixel_total : 28597 time to create 1 rle with old method : 0.03465771675109863 length of segment : 326 time for calcul the mask position with numpy : 0.0009965896606445312 nb_pixel_total : 43980 time to create 1 rle with old method : 0.0506291389465332 length of segment : 259 time for calcul the mask position with numpy : 0.0012662410736083984 nb_pixel_total : 55751 time to create 1 rle with old method : 0.06410932540893555 length of segment : 322 time for calcul the mask position with numpy : 0.001203298568725586 nb_pixel_total : 54593 time to create 1 rle with old method : 0.0660104751586914 length of segment : 286 time for calcul the mask position with numpy : 0.0004975795745849609 nb_pixel_total : 22457 time to create 1 rle with old method : 0.025600194931030273 length of segment : 180 time for calcul the mask position with numpy : 0.005560874938964844 nb_pixel_total : 319917 time to create 1 rle with new method : 0.013521671295166016 length of segment : 685 time for calcul the mask position with numpy : 0.0020987987518310547 nb_pixel_total : 108322 time to create 1 rle with old method : 0.11826896667480469 length of segment : 427 time for calcul the mask position with numpy : 0.003186464309692383 nb_pixel_total : 149318 time to create 1 rle with old method : 0.16727733612060547 length of segment : 517 time for calcul the mask position with numpy : 0.006840944290161133 nb_pixel_total : 322003 time to create 1 rle with new method : 0.014853239059448242 length of segment : 697 time for calcul the mask position with numpy : 0.000896453857421875 nb_pixel_total : 46971 time to create 1 rle with old method : 0.06761288642883301 length of segment : 230 time for calcul the mask position with numpy : 0.0006785392761230469 nb_pixel_total : 39654 time to create 1 rle with old method : 0.04550600051879883 length of segment : 262 time for calcul the mask position with numpy : 0.007501125335693359 nb_pixel_total : 431042 time to create 1 rle with new method : 0.018316984176635742 length of segment : 956 time for calcul the mask position with numpy : 0.0060918331146240234 nb_pixel_total : 243066 time to create 1 rle with new method : 0.019327640533447266 length of segment : 550 time for calcul the mask position with numpy : 0.0053060054779052734 nb_pixel_total : 329237 time to create 1 rle with new method : 0.014214277267456055 length of segment : 804 time for calcul the mask position with numpy : 0.0004513263702392578 nb_pixel_total : 20731 time to create 1 rle with old method : 0.02321028709411621 length of segment : 141 time for calcul the mask position with numpy : 0.0005512237548828125 nb_pixel_total : 32934 time to create 1 rle with old method : 0.03688478469848633 length of segment : 237 time for calcul the mask position with numpy : 0.002035379409790039 nb_pixel_total : 104493 time to create 1 rle with old method : 0.1175236701965332 length of segment : 588 time for calcul the mask position with numpy : 0.0013918876647949219 nb_pixel_total : 52124 time to create 1 rle with old method : 0.06047773361206055 length of segment : 515 time for calcul the mask position with numpy : 0.003334522247314453 nb_pixel_total : 210722 time to create 1 rle with new method : 0.00857686996459961 length of segment : 439 time for calcul the mask position with numpy : 0.0018572807312011719 nb_pixel_total : 76948 time to create 1 rle with old method : 0.0863494873046875 length of segment : 319 time for calcul the mask position with numpy : 0.0009415149688720703 nb_pixel_total : 36842 time to create 1 rle with old method : 0.05390357971191406 length of segment : 188 time for calcul the mask position with numpy : 0.0013399124145507812 nb_pixel_total : 50187 time to create 1 rle with old method : 0.0568089485168457 length of segment : 271 time for calcul the mask position with numpy : 0.022010326385498047 nb_pixel_total : 638047 time to create 1 rle with new method : 0.15616631507873535 length of segment : 1322 time for calcul the mask position with numpy : 0.0006794929504394531 nb_pixel_total : 24410 time to create 1 rle with old method : 0.030507802963256836 length of segment : 254 time for calcul the mask position with numpy : 0.0007021427154541016 nb_pixel_total : 31878 time to create 1 rle with old method : 0.0369105339050293 length of segment : 205 time for calcul the mask position with numpy : 0.0003933906555175781 nb_pixel_total : 17879 time to create 1 rle with old method : 0.020463228225708008 length of segment : 176 time for calcul the mask position with numpy : 0.0007028579711914062 nb_pixel_total : 44227 time to create 1 rle with old method : 0.07399725914001465 length of segment : 245 time for calcul the mask position with numpy : 0.0033812522888183594 nb_pixel_total : 153677 time to create 1 rle with new method : 0.009714126586914062 length of segment : 403 time for calcul the mask position with numpy : 0.003036975860595703 nb_pixel_total : 205300 time to create 1 rle with new method : 0.007559776306152344 length of segment : 440 time for calcul the mask position with numpy : 0.004256486892700195 nb_pixel_total : 250852 time to create 1 rle with new method : 0.009633541107177734 length of segment : 611 time for calcul the mask position with numpy : 0.003499746322631836 nb_pixel_total : 189302 time to create 1 rle with new method : 0.009839057922363281 length of segment : 609 time for calcul the mask position with numpy : 1.262688159942627 nb_pixel_total : 7241926 time to create 1 rle with new method : 2.3752903938293457 length of segment : 5946 time for calcul the mask position with numpy : 0.011512517929077148 nb_pixel_total : 850270 time to create 1 rle with new method : 0.08243107795715332 length of segment : 813 time for calcul the mask position with numpy : 0.025796890258789062 nb_pixel_total : 2137491 time to create 1 rle with new method : 0.14196443557739258 length of segment : 1448 time for calcul the mask position with numpy : 0.0056345462799072266 nb_pixel_total : 461507 time to create 1 rle with new method : 0.011273384094238281 length of segment : 507 time for calcul the mask position with numpy : 0.002534151077270508 nb_pixel_total : 219247 time to create 1 rle with new method : 0.006096363067626953 length of segment : 688 time for calcul the mask position with numpy : 0.014983654022216797 nb_pixel_total : 96670 time to create 1 rle with old method : 0.11602139472961426 length of segment : 674 time for calcul the mask position with numpy : 0.14302301406860352 nb_pixel_total : 2544187 time to create 1 rle with new method : 0.18619132041931152 length of segment : 1435 time for calcul the mask position with numpy : 0.0021665096282958984 nb_pixel_total : 171769 time to create 1 rle with new method : 0.0047283172607421875 length of segment : 404 time for calcul the mask position with numpy : 0.002047300338745117 nb_pixel_total : 172986 time to create 1 rle with new method : 0.0041773319244384766 length of segment : 393 time for calcul the mask position with numpy : 0.48973703384399414 nb_pixel_total : 3533909 time to create 1 rle with new method : 1.8027081489562988 length of segment : 6816 time for calcul the mask position with numpy : 0.002317667007446289 nb_pixel_total : 157338 time to create 1 rle with new method : 0.005499601364135742 length of segment : 838 time for calcul the mask position with numpy : 0.009538888931274414 nb_pixel_total : 715914 time to create 1 rle with new method : 0.025310516357421875 length of segment : 1117 time for calcul the mask position with numpy : 0.007192850112915039 nb_pixel_total : 525962 time to create 1 rle with new method : 0.016164064407348633 length of segment : 1349 time for calcul the mask position with numpy : 0.027958154678344727 nb_pixel_total : 2282490 time to create 1 rle with new method : 0.7382175922393799 length of segment : 1553 time for calcul the mask position with numpy : 0.0077114105224609375 nb_pixel_total : 527315 time to create 1 rle with new method : 0.014894962310791016 length of segment : 1060 time for calcul the mask position with numpy : 0.0007970333099365234 nb_pixel_total : 34311 time to create 1 rle with old method : 0.04497838020324707 length of segment : 370 time spent for convertir_results : 50.72047448158264 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 333 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 143.90211987495422 time spend to save output : 0.18319034576416016 total time spend for step 2 : 144.08531022071838 step3:blur_detection Tue May 6 13:05:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (3120, 4160) 3466.2512972873333 (3120, 4160) 4150.154521630281 (3120, 4160) 4608.056408580415 (3120, 4160) 6776.4835558636105 (4160, 3120) 417.1945760873511 (4160, 3120) 278.3087073565592 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 6 time used for this insertion : 0.011678457260131836 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 6 time used for this insertion : 0.011820793151855469 save missing photos in datou_result : time spend for datou_step_exec : 3.153693199157715 time spend to save output : 0.028359651565551758 total time spend for step 3 : 3.1820528507232666 step4:brightness Tue May 6 13:05:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746529289_829539_1356331559_e0345ec79d8185c0fceef8fca152de81.jpg treat image : temp/1746529289_829539_1356331557_e2a2ae64194a34e481dd576a8eceaed6.jpg treat image : temp/1746529289_829539_1356331553_de0284f7be60c60cfc47eca7b6ba11e9.jpg treat image : temp/1746529289_829539_1356331545_a492a67d65a671edeb3015355ce10ded.jpg treat image : temp/1746529289_829539_1356331538_d4407615625af84a401844688b4e1d02.jpg treat image : temp/1746529289_829539_1356331534_0a840f453e6af68ea9282decf566ff5a.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 6 time used for this insertion : 0.010905742645263672 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 6 time used for this insertion : 0.010682106018066406 save missing photos in datou_result : time spend for datou_step_exec : 9.732370376586914 time spend to save output : 0.02623748779296875 total time spend for step 4 : 9.758607864379883 step5:rle_unique_nms_with_priority Tue May 6 13:05:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 333 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 84 nb_hashtags : 10 time to prepare the origin masks : 73.68286943435669 time for calcul the mask position with numpy : 0.3576233386993408 nb_pixel_total : 4474625 time to create 1 rle with new method : 1.4132206439971924 time for calcul the mask position with numpy : 0.07210659980773926 nb_pixel_total : 12996 time to create 1 rle with old method : 0.017123699188232422 time for calcul the mask position with numpy : 0.06727790832519531 nb_pixel_total : 158085 time to create 1 rle with new method : 2.188103199005127 time for calcul the mask position with numpy : 0.05843710899353027 nb_pixel_total : 315697 time to create 1 rle with new method : 1.2511751651763916 time for calcul the mask position with numpy : 0.05109071731567383 nb_pixel_total : 9663 time to create 1 rle with old method : 0.010488033294677734 time for calcul the mask position with numpy : 0.052834510803222656 nb_pixel_total : 2552 time to create 1 rle with old method : 0.0031921863555908203 time for calcul the mask position with numpy : 0.07894325256347656 nb_pixel_total : 839326 time to create 1 rle with new method : 0.9327187538146973 time for calcul the mask position with numpy : 0.05207967758178711 nb_pixel_total : 64260 time to create 1 rle with old method : 0.06856703758239746 time for calcul the mask position with numpy : 0.05481743812561035 nb_pixel_total : 180768 time to create 1 rle with new method : 1.148843765258789 time for calcul the mask position with numpy : 0.051844120025634766 nb_pixel_total : 37106 time to create 1 rle with old method : 0.04301857948303223 time for calcul the mask position with numpy : 0.06986665725708008 nb_pixel_total : 250634 time to create 1 rle with new method : 1.3343985080718994 time for calcul the mask position with numpy : 0.05416226387023926 nb_pixel_total : 15256 time to create 1 rle with old method : 0.017220258712768555 time for calcul the mask position with numpy : 0.05463695526123047 nb_pixel_total : 21921 time to create 1 rle with old method : 0.025290966033935547 time for calcul the mask position with numpy : 0.05945563316345215 nb_pixel_total : 36959 time to create 1 rle with old method : 0.040439605712890625 time for calcul the mask position with numpy : 0.05470418930053711 nb_pixel_total : 1097 time to create 1 rle with old method : 0.001348257064819336 time for calcul the mask position with numpy : 0.05511784553527832 nb_pixel_total : 27431 time to create 1 rle with old method : 0.0298459529876709 time for calcul the mask position with numpy : 0.053452491760253906 nb_pixel_total : 46469 time to create 1 rle with old method : 0.05096912384033203 time for calcul the mask position with numpy : 0.06004691123962402 nb_pixel_total : 521484 time to create 1 rle with new method : 1.904296636581421 time for calcul the mask position with numpy : 0.056921958923339844 nb_pixel_total : 244827 time to create 1 rle with new method : 1.302560567855835 time for calcul the mask position with numpy : 0.05214262008666992 nb_pixel_total : 127449 time to create 1 rle with old method : 0.1399095058441162 time for calcul the mask position with numpy : 0.05317568778991699 nb_pixel_total : 49148 time to create 1 rle with old method : 0.05436897277832031 time for calcul the mask position with numpy : 0.05266976356506348 nb_pixel_total : 20048 time to create 1 rle with old method : 0.022171735763549805 time for calcul the mask position with numpy : 0.05308842658996582 nb_pixel_total : 2615 time to create 1 rle with old method : 0.0029425621032714844 time for calcul the mask position with numpy : 0.05276942253112793 nb_pixel_total : 23582 time to create 1 rle with old method : 0.025728940963745117 time for calcul the mask position with numpy : 0.05441451072692871 nb_pixel_total : 221306 time to create 1 rle with new method : 1.5654866695404053 time for calcul the mask position with numpy : 0.05265235900878906 nb_pixel_total : 44879 time to create 1 rle with old method : 0.04670238494873047 time for calcul the mask position with numpy : 0.05236935615539551 nb_pixel_total : 9626 time to create 1 rle with old method : 0.011099100112915039 time for calcul the mask position with numpy : 0.0533604621887207 nb_pixel_total : 2991 time to create 1 rle with old method : 0.003462076187133789 time for calcul the mask position with numpy : 0.05855679512023926 nb_pixel_total : 11655 time to create 1 rle with old method : 0.013050079345703125 time for calcul the mask position with numpy : 0.05267620086669922 nb_pixel_total : 31213 time to create 1 rle with old method : 0.03404712677001953 time for calcul the mask position with numpy : 0.055835723876953125 nb_pixel_total : 289272 time to create 1 rle with new method : 0.7922806739807129 time for calcul the mask position with numpy : 0.05240607261657715 nb_pixel_total : 86168 time to create 1 rle with old method : 0.09148573875427246 time for calcul the mask position with numpy : 0.05175280570983887 nb_pixel_total : 3142 time to create 1 rle with old method : 0.003462076187133789 time for calcul the mask position with numpy : 0.05131697654724121 nb_pixel_total : 3690 time to create 1 rle with old method : 0.004218101501464844 time for calcul the mask position with numpy : 0.05211281776428223 nb_pixel_total : 51120 time to create 1 rle with old method : 0.05528593063354492 time for calcul the mask position with numpy : 0.051996707916259766 nb_pixel_total : 80500 time to create 1 rle with old method : 0.08433198928833008 time for calcul the mask position with numpy : 0.051494598388671875 nb_pixel_total : 76294 time to create 1 rle with old method : 0.0785977840423584 time for calcul the mask position with numpy : 0.05203390121459961 nb_pixel_total : 8227 time to create 1 rle with old method : 0.009628057479858398 time for calcul the mask position with numpy : 0.05351853370666504 nb_pixel_total : 115135 time to create 1 rle with old method : 0.12551307678222656 time for calcul the mask position with numpy : 0.05281329154968262 nb_pixel_total : 58072 time to create 1 rle with old method : 0.30289125442504883 time for calcul the mask position with numpy : 0.05616044998168945 nb_pixel_total : 160360 time to create 1 rle with new method : 1.2228572368621826 time for calcul the mask position with numpy : 0.052529096603393555 nb_pixel_total : 130764 time to create 1 rle with old method : 0.14026713371276855 time for calcul the mask position with numpy : 0.05238986015319824 nb_pixel_total : 3817 time to create 1 rle with old method : 0.0044422149658203125 time for calcul the mask position with numpy : 0.0540919303894043 nb_pixel_total : 106712 time to create 1 rle with old method : 0.1148996353149414 time for calcul the mask position with numpy : 0.053327322006225586 nb_pixel_total : 60609 time to create 1 rle with old method : 0.06534433364868164 time for calcul the mask position with numpy : 0.053475141525268555 nb_pixel_total : 84294 time to create 1 rle with old method : 0.09307646751403809 time for calcul the mask position with numpy : 0.053527116775512695 nb_pixel_total : 245748 time to create 1 rle with new method : 1.850602149963379 time for calcul the mask position with numpy : 0.053168535232543945 nb_pixel_total : 59780 time to create 1 rle with old method : 0.06329202651977539 time for calcul the mask position with numpy : 0.05194425582885742 nb_pixel_total : 7263 time to create 1 rle with old method : 0.008371829986572266 time for calcul the mask position with numpy : 0.053215980529785156 nb_pixel_total : 142875 time to create 1 rle with old method : 0.15719842910766602 time for calcul the mask position with numpy : 0.05326509475708008 nb_pixel_total : 34809 time to create 1 rle with old method : 0.03775978088378906 time for calcul the mask position with numpy : 0.0523068904876709 nb_pixel_total : 104165 time to create 1 rle with old method : 0.11264204978942871 time for calcul the mask position with numpy : 0.053366661071777344 nb_pixel_total : 227698 time to create 1 rle with new method : 1.4745595455169678 time for calcul the mask position with numpy : 0.05467534065246582 nb_pixel_total : 113928 time to create 1 rle with old method : 0.6159064769744873 time for calcul the mask position with numpy : 0.05527472496032715 nb_pixel_total : 247198 time to create 1 rle with new method : 1.3061020374298096 time for calcul the mask position with numpy : 0.05162954330444336 nb_pixel_total : 74355 time to create 1 rle with old method : 0.07841300964355469 time for calcul the mask position with numpy : 0.05206942558288574 nb_pixel_total : 57632 time to create 1 rle with old method : 0.06126689910888672 time for calcul the mask position with numpy : 0.05197787284851074 nb_pixel_total : 7195 time to create 1 rle with old method : 0.008249044418334961 time for calcul the mask position with numpy : 0.051543474197387695 nb_pixel_total : 52161 time to create 1 rle with old method : 0.05619454383850098 time for calcul the mask position with numpy : 0.0525670051574707 nb_pixel_total : 143589 time to create 1 rle with old method : 0.15277528762817383 time for calcul the mask position with numpy : 0.05347847938537598 nb_pixel_total : 56075 time to create 1 rle with old method : 0.059629201889038086 time for calcul the mask position with numpy : 0.051854848861694336 nb_pixel_total : 7440 time to create 1 rle with old method : 0.007783651351928711 time for calcul the mask position with numpy : 0.05093550682067871 nb_pixel_total : 1373 time to create 1 rle with old method : 0.0019915103912353516 time for calcul the mask position with numpy : 0.051975250244140625 nb_pixel_total : 157934 time to create 1 rle with new method : 1.1890225410461426 time for calcul the mask position with numpy : 0.05405259132385254 nb_pixel_total : 14899 time to create 1 rle with old method : 0.015887737274169922 time for calcul the mask position with numpy : 0.05303478240966797 nb_pixel_total : 5854 time to create 1 rle with old method : 0.006838798522949219 time for calcul the mask position with numpy : 0.05304837226867676 nb_pixel_total : 2124 time to create 1 rle with old method : 0.0028204917907714844 time for calcul the mask position with numpy : 0.05421900749206543 nb_pixel_total : 92230 time to create 1 rle with old method : 0.09935331344604492 time for calcul the mask position with numpy : 0.05489182472229004 nb_pixel_total : 201297 time to create 1 rle with new method : 0.6473572254180908 time for calcul the mask position with numpy : 0.054201364517211914 nb_pixel_total : 103414 time to create 1 rle with old method : 0.11255574226379395 time for calcul the mask position with numpy : 0.05574393272399902 nb_pixel_total : 994 time to create 1 rle with old method : 0.001399993896484375 time for calcul the mask position with numpy : 0.05525970458984375 nb_pixel_total : 18830 time to create 1 rle with old method : 0.025119543075561523 time for calcul the mask position with numpy : 0.05809330940246582 nb_pixel_total : 283737 time to create 1 rle with new method : 0.7803082466125488 time for calcul the mask position with numpy : 0.05345940589904785 nb_pixel_total : 243713 time to create 1 rle with new method : 0.5853455066680908 time for calcul the mask position with numpy : 0.05455374717712402 nb_pixel_total : 173519 time to create 1 rle with new method : 0.6632506847381592 time for calcul the mask position with numpy : 0.05543923377990723 nb_pixel_total : 35116 time to create 1 rle with old method : 0.038869380950927734 time for calcul the mask position with numpy : 0.056131601333618164 nb_pixel_total : 254974 time to create 1 rle with new method : 0.6857078075408936 time for calcul the mask position with numpy : 0.05489468574523926 nb_pixel_total : 129265 time to create 1 rle with old method : 0.13766694068908691 time for calcul the mask position with numpy : 0.05353665351867676 nb_pixel_total : 94602 time to create 1 rle with old method : 0.10167121887207031 time for calcul the mask position with numpy : 0.052869558334350586 nb_pixel_total : 14192 time to create 1 rle with old method : 0.015365839004516602 time for calcul the mask position with numpy : 0.053597450256347656 nb_pixel_total : 126935 time to create 1 rle with old method : 0.1374526023864746 time for calcul the mask position with numpy : 0.052857398986816406 nb_pixel_total : 2507 time to create 1 rle with old method : 0.006005287170410156 time for calcul the mask position with numpy : 0.06305813789367676 nb_pixel_total : 159218 time to create 1 rle with new method : 1.0104684829711914 time for calcul the mask position with numpy : 0.05451178550720215 nb_pixel_total : 83609 time to create 1 rle with old method : 0.0930030345916748 time for calcul the mask position with numpy : 0.05402398109436035 nb_pixel_total : 73109 time to create 1 rle with old method : 0.0786590576171875 create new chi : 35.375308990478516 time to delete rle : 0.03633403778076172 batch 1 Loaded 85 chid ids of type : 4855 Number RLEs to save : 58200 TO DO : save crop sub photo not yet done ! save time : 3.229790687561035 nb_obj : 86 nb_hashtags : 9 time to prepare the origin masks : 60.258902072906494 time for calcul the mask position with numpy : 1.0220236778259277 nb_pixel_total : 5714387 time to create 1 rle with new method : 1.2609148025512695 time for calcul the mask position with numpy : 0.051993370056152344 nb_pixel_total : 7646 time to create 1 rle with old method : 0.008456945419311523 time for calcul the mask position with numpy : 0.0508878231048584 nb_pixel_total : 35945 time to create 1 rle with old method : 0.03778195381164551 time for calcul the mask position with numpy : 0.051843881607055664 nb_pixel_total : 5906 time to create 1 rle with old method : 0.007147312164306641 time for calcul the mask position with numpy : 0.05357480049133301 nb_pixel_total : 33750 time to create 1 rle with old method : 0.03707575798034668 time for calcul the mask position with numpy : 0.41506266593933105 nb_pixel_total : 808692 time to create 1 rle with new method : 1.4545502662658691 time for calcul the mask position with numpy : 0.05057811737060547 nb_pixel_total : 4160 time to create 1 rle with old method : 0.005513668060302734 time for calcul the mask position with numpy : 0.0504457950592041 nb_pixel_total : 18728 time to create 1 rle with old method : 0.020398378372192383 time for calcul the mask position with numpy : 0.053278207778930664 nb_pixel_total : 495 time to create 1 rle with old method : 0.0006854534149169922 time for calcul the mask position with numpy : 0.05246305465698242 nb_pixel_total : 22265 time to create 1 rle with old method : 0.024001359939575195 time for calcul the mask position with numpy : 0.05553579330444336 nb_pixel_total : 405245 time to create 1 rle with new method : 1.7121682167053223 time for calcul the mask position with numpy : 0.051531314849853516 nb_pixel_total : 24192 time to create 1 rle with old method : 0.07139253616333008 time for calcul the mask position with numpy : 0.05159330368041992 nb_pixel_total : 77355 time to create 1 rle with old method : 0.08257007598876953 time for calcul the mask position with numpy : 0.052106380462646484 nb_pixel_total : 58175 time to create 1 rle with old method : 0.06044411659240723 time for calcul the mask position with numpy : 0.05108952522277832 nb_pixel_total : 253 time to create 1 rle with old method : 0.00043654441833496094 time for calcul the mask position with numpy : 0.05177664756774902 nb_pixel_total : 18191 time to create 1 rle with old method : 0.019680023193359375 time for calcul the mask position with numpy : 0.05110788345336914 nb_pixel_total : 27053 time to create 1 rle with old method : 0.03430914878845215 time for calcul the mask position with numpy : 0.0523991584777832 nb_pixel_total : 141868 time to create 1 rle with old method : 0.14590072631835938 time for calcul the mask position with numpy : 0.05234241485595703 nb_pixel_total : 37809 time to create 1 rle with old method : 0.03912711143493652 time for calcul the mask position with numpy : 0.05239534378051758 nb_pixel_total : 218384 time to create 1 rle with new method : 1.5304827690124512 time for calcul the mask position with numpy : 0.05407309532165527 nb_pixel_total : 60409 time to create 1 rle with old method : 0.06463980674743652 time for calcul the mask position with numpy : 0.05172610282897949 nb_pixel_total : 1490 time to create 1 rle with old method : 0.0016505718231201172 time for calcul the mask position with numpy : 0.052275657653808594 nb_pixel_total : 20885 time to create 1 rle with old method : 0.022238969802856445 time for calcul the mask position with numpy : 0.051728248596191406 nb_pixel_total : 6770 time to create 1 rle with old method : 0.007393360137939453 time for calcul the mask position with numpy : 0.051189422607421875 nb_pixel_total : 5 time to create 1 rle with old method : 6.389617919921875e-05 time for calcul the mask position with numpy : 0.05254507064819336 nb_pixel_total : 199326 time to create 1 rle with new method : 1.2562413215637207 time for calcul the mask position with numpy : 0.05856513977050781 nb_pixel_total : 439846 time to create 1 rle with new method : 1.2644684314727783 time for calcul the mask position with numpy : 0.052413225173950195 nb_pixel_total : 33695 time to create 1 rle with old method : 0.03579425811767578 time for calcul the mask position with numpy : 0.05083060264587402 nb_pixel_total : 19545 time to create 1 rle with old method : 0.02069711685180664 time for calcul the mask position with numpy : 0.052646636962890625 nb_pixel_total : 85942 time to create 1 rle with old method : 0.09376835823059082 time for calcul the mask position with numpy : 0.05509185791015625 nb_pixel_total : 111127 time to create 1 rle with old method : 0.12984418869018555 time for calcul the mask position with numpy : 0.054447174072265625 nb_pixel_total : 42392 time to create 1 rle with old method : 0.04733729362487793 time for calcul the mask position with numpy : 0.053783416748046875 nb_pixel_total : 89580 time to create 1 rle with old method : 0.09893321990966797 time for calcul the mask position with numpy : 0.05554914474487305 nb_pixel_total : 3062 time to create 1 rle with old method : 0.0042650699615478516 time for calcul the mask position with numpy : 0.05327939987182617 nb_pixel_total : 3057 time to create 1 rle with old method : 0.004067659378051758 time for calcul the mask position with numpy : 0.05104517936706543 nb_pixel_total : 1556 time to create 1 rle with old method : 0.0019066333770751953 time for calcul the mask position with numpy : 0.05140352249145508 nb_pixel_total : 59026 time to create 1 rle with old method : 0.06321072578430176 time for calcul the mask position with numpy : 0.0549321174621582 nb_pixel_total : 62158 time to create 1 rle with old method : 0.0930168628692627 time for calcul the mask position with numpy : 0.0540928840637207 nb_pixel_total : 387969 time to create 1 rle with new method : 1.5609550476074219 time for calcul the mask position with numpy : 0.05190920829772949 nb_pixel_total : 12428 time to create 1 rle with old method : 0.013978004455566406 time for calcul the mask position with numpy : 0.054399728775024414 nb_pixel_total : 1844 time to create 1 rle with old method : 0.003519773483276367 time for calcul the mask position with numpy : 0.060610055923461914 nb_pixel_total : 375948 time to create 1 rle with new method : 2.0743465423583984 time for calcul the mask position with numpy : 0.05281829833984375 nb_pixel_total : 4597 time to create 1 rle with old method : 0.008764028549194336 time for calcul the mask position with numpy : 0.05395913124084473 nb_pixel_total : 97469 time to create 1 rle with old method : 0.10470342636108398 time for calcul the mask position with numpy : 0.05284285545349121 nb_pixel_total : 45900 time to create 1 rle with old method : 0.05656075477600098 time for calcul the mask position with numpy : 0.053277015686035156 nb_pixel_total : 14788 time to create 1 rle with old method : 0.016914844512939453 time for calcul the mask position with numpy : 0.05420517921447754 nb_pixel_total : 31551 time to create 1 rle with old method : 0.03522324562072754 time for calcul the mask position with numpy : 0.05408883094787598 nb_pixel_total : 67103 time to create 1 rle with old method : 0.07572579383850098 time for calcul the mask position with numpy : 0.05430912971496582 nb_pixel_total : 35119 time to create 1 rle with old method : 0.039212703704833984 time for calcul the mask position with numpy : 0.05469846725463867 nb_pixel_total : 101744 time to create 1 rle with old method : 0.1149897575378418 time for calcul the mask position with numpy : 0.06185770034790039 nb_pixel_total : 3353 time to create 1 rle with old method : 0.005767822265625 time for calcul the mask position with numpy : 0.05322265625 nb_pixel_total : 23020 time to create 1 rle with old method : 0.026061058044433594 time for calcul the mask position with numpy : 0.05270719528198242 nb_pixel_total : 225167 time to create 1 rle with new method : 1.5734319686889648 time for calcul the mask position with numpy : 0.05418825149536133 nb_pixel_total : 120903 time to create 1 rle with old method : 0.13208413124084473 time for calcul the mask position with numpy : 0.05265617370605469 nb_pixel_total : 10952 time to create 1 rle with old method : 0.012886762619018555 time for calcul the mask position with numpy : 0.05392885208129883 nb_pixel_total : 67963 time to create 1 rle with old method : 0.07426261901855469 time for calcul the mask position with numpy : 0.0521547794342041 nb_pixel_total : 46624 time to create 1 rle with old method : 0.05017399787902832 time for calcul the mask position with numpy : 0.051656484603881836 nb_pixel_total : 526 time to create 1 rle with old method : 0.0008795261383056641 time for calcul the mask position with numpy : 0.05395364761352539 nb_pixel_total : 49739 time to create 1 rle with old method : 0.05948019027709961 time for calcul the mask position with numpy : 0.056767940521240234 nb_pixel_total : 441079 time to create 1 rle with new method : 1.131859302520752 time for calcul the mask position with numpy : 0.05033516883850098 nb_pixel_total : 97467 time to create 1 rle with old method : 0.13068628311157227 time for calcul the mask position with numpy : 0.05321812629699707 nb_pixel_total : 312035 time to create 1 rle with new method : 1.3336195945739746 time for calcul the mask position with numpy : 0.05396699905395508 nb_pixel_total : 9020 time to create 1 rle with old method : 0.010042428970336914 time for calcul the mask position with numpy : 0.05237603187561035 nb_pixel_total : 103522 time to create 1 rle with old method : 0.1423325538635254 time for calcul the mask position with numpy : 0.06149601936340332 nb_pixel_total : 7643 time to create 1 rle with old method : 0.008647680282592773 time for calcul the mask position with numpy : 0.05055046081542969 nb_pixel_total : 7842 time to create 1 rle with old method : 0.008569717407226562 time for calcul the mask position with numpy : 0.05205106735229492 nb_pixel_total : 7555 time to create 1 rle with old method : 0.009006261825561523 time for calcul the mask position with numpy : 0.05072283744812012 nb_pixel_total : 14696 time to create 1 rle with old method : 0.015277385711669922 time for calcul the mask position with numpy : 0.05264544486999512 nb_pixel_total : 80998 time to create 1 rle with old method : 0.0888674259185791 time for calcul the mask position with numpy : 0.05280280113220215 nb_pixel_total : 12277 time to create 1 rle with old method : 0.013452768325805664 time for calcul the mask position with numpy : 0.0537571907043457 nb_pixel_total : 133857 time to create 1 rle with old method : 0.13846659660339355 time for calcul the mask position with numpy : 0.05147409439086914 nb_pixel_total : 3885 time to create 1 rle with old method : 0.004786252975463867 time for calcul the mask position with numpy : 0.05213427543640137 nb_pixel_total : 122063 time to create 1 rle with old method : 0.127671480178833 time for calcul the mask position with numpy : 0.05059480667114258 nb_pixel_total : 21886 time to create 1 rle with old method : 0.02260303497314453 time for calcul the mask position with numpy : 0.052034616470336914 nb_pixel_total : 62022 time to create 1 rle with old method : 0.06840014457702637 time for calcul the mask position with numpy : 0.05750393867492676 nb_pixel_total : 220173 time to create 1 rle with new method : 1.1812992095947266 time for calcul the mask position with numpy : 0.05104947090148926 nb_pixel_total : 74310 time to create 1 rle with old method : 0.07904791831970215 time for calcul the mask position with numpy : 0.05069899559020996 nb_pixel_total : 52934 time to create 1 rle with old method : 0.05456948280334473 time for calcul the mask position with numpy : 0.051104068756103516 nb_pixel_total : 5824 time to create 1 rle with old method : 0.00622105598449707 time for calcul the mask position with numpy : 0.050861358642578125 nb_pixel_total : 336 time to create 1 rle with old method : 0.0004303455352783203 time for calcul the mask position with numpy : 0.05158400535583496 nb_pixel_total : 91033 time to create 1 rle with old method : 0.09406709671020508 time for calcul the mask position with numpy : 0.051197052001953125 nb_pixel_total : 44185 time to create 1 rle with old method : 0.045304059982299805 time for calcul the mask position with numpy : 0.05153846740722656 nb_pixel_total : 14098 time to create 1 rle with old method : 0.014748096466064453 time for calcul the mask position with numpy : 0.05129694938659668 nb_pixel_total : 74985 time to create 1 rle with old method : 0.07761406898498535 time for calcul the mask position with numpy : 0.052320003509521484 nb_pixel_total : 171152 time to create 1 rle with new method : 1.5836725234985352 time for calcul the mask position with numpy : 0.05527544021606445 nb_pixel_total : 167164 time to create 1 rle with new method : 1.646242380142212 time for calcul the mask position with numpy : 0.053330183029174805 nb_pixel_total : 82 time to create 1 rle with old method : 0.00015306472778320312 create new chi : 30.56047224998474 time to delete rle : 0.006571292877197266 batch 1 Loaded 87 chid ids of type : 4855 Number RLEs to save : 56780 TO DO : save crop sub photo not yet done ! save time : 3.1646511554718018 nb_obj : 76 nb_hashtags : 10 time to prepare the origin masks : 59.289528131484985 time for calcul the mask position with numpy : 0.43257641792297363 nb_pixel_total : 5107786 time to create 1 rle with new method : 1.1604037284851074 time for calcul the mask position with numpy : 0.05335497856140137 nb_pixel_total : 7679 time to create 1 rle with old method : 0.008915185928344727 time for calcul the mask position with numpy : 0.052530527114868164 nb_pixel_total : 33643 time to create 1 rle with old method : 0.0390467643737793 time for calcul the mask position with numpy : 0.062459707260131836 nb_pixel_total : 251275 time to create 1 rle with new method : 1.4635488986968994 time for calcul the mask position with numpy : 0.054559946060180664 nb_pixel_total : 18689 time to create 1 rle with old method : 0.021480560302734375 time for calcul the mask position with numpy : 0.05736398696899414 nb_pixel_total : 55343 time to create 1 rle with old method : 0.06202864646911621 time for calcul the mask position with numpy : 0.05881810188293457 nb_pixel_total : 241561 time to create 1 rle with new method : 2.602968692779541 time for calcul the mask position with numpy : 0.05526542663574219 nb_pixel_total : 31321 time to create 1 rle with old method : 0.03596687316894531 time for calcul the mask position with numpy : 0.05489397048950195 nb_pixel_total : 246239 time to create 1 rle with new method : 1.390761375427246 time for calcul the mask position with numpy : 0.05850839614868164 nb_pixel_total : 3784 time to create 1 rle with old method : 0.005006313323974609 time for calcul the mask position with numpy : 0.0589604377746582 nb_pixel_total : 15470 time to create 1 rle with old method : 0.017482519149780273 time for calcul the mask position with numpy : 0.054811954498291016 nb_pixel_total : 11328 time to create 1 rle with old method : 0.013277769088745117 time for calcul the mask position with numpy : 0.06056833267211914 nb_pixel_total : 298839 time to create 1 rle with new method : 0.9020893573760986 time for calcul the mask position with numpy : 0.05303478240966797 nb_pixel_total : 168434 time to create 1 rle with new method : 2.0017123222351074 time for calcul the mask position with numpy : 0.051308631896972656 nb_pixel_total : 1003 time to create 1 rle with old method : 0.0013098716735839844 time for calcul the mask position with numpy : 0.05198073387145996 nb_pixel_total : 95925 time to create 1 rle with old method : 0.1048734188079834 time for calcul the mask position with numpy : 0.05409860610961914 nb_pixel_total : 99498 time to create 1 rle with old method : 0.13611221313476562 time for calcul the mask position with numpy : 0.30594420433044434 nb_pixel_total : 1124037 time to create 1 rle with new method : 2.0175838470458984 time for calcul the mask position with numpy : 0.056023359298706055 nb_pixel_total : 268676 time to create 1 rle with new method : 1.6152148246765137 time for calcul the mask position with numpy : 0.0538792610168457 nb_pixel_total : 12262 time to create 1 rle with old method : 0.013897895812988281 time for calcul the mask position with numpy : 0.05357503890991211 nb_pixel_total : 49095 time to create 1 rle with old method : 0.05560779571533203 time for calcul the mask position with numpy : 0.05638742446899414 nb_pixel_total : 514490 time to create 1 rle with new method : 1.3405261039733887 time for calcul the mask position with numpy : 0.05733156204223633 nb_pixel_total : 2689 time to create 1 rle with old method : 0.0033113956451416016 time for calcul the mask position with numpy : 0.05600929260253906 nb_pixel_total : 18178 time to create 1 rle with old method : 0.020526885986328125 time for calcul the mask position with numpy : 0.056455135345458984 nb_pixel_total : 480422 time to create 1 rle with new method : 1.3202605247497559 time for calcul the mask position with numpy : 0.05839943885803223 nb_pixel_total : 18697 time to create 1 rle with old method : 0.02107381820678711 time for calcul the mask position with numpy : 0.055362701416015625 nb_pixel_total : 68831 time to create 1 rle with old method : 0.0766897201538086 time for calcul the mask position with numpy : 0.05483818054199219 nb_pixel_total : 143440 time to create 1 rle with old method : 0.18289613723754883 time for calcul the mask position with numpy : 0.0543673038482666 nb_pixel_total : 61244 time to create 1 rle with old method : 0.06822752952575684 time for calcul the mask position with numpy : 0.05427742004394531 nb_pixel_total : 84710 time to create 1 rle with old method : 0.093902587890625 time for calcul the mask position with numpy : 0.05540633201599121 nb_pixel_total : 331781 time to create 1 rle with new method : 2.075143814086914 time for calcul the mask position with numpy : 0.05324149131774902 nb_pixel_total : 28643 time to create 1 rle with old method : 0.03155779838562012 time for calcul the mask position with numpy : 0.05417895317077637 nb_pixel_total : 85555 time to create 1 rle with old method : 0.0996396541595459 time for calcul the mask position with numpy : 0.05509209632873535 nb_pixel_total : 17214 time to create 1 rle with old method : 0.0198214054107666 time for calcul the mask position with numpy : 0.054293155670166016 nb_pixel_total : 130072 time to create 1 rle with old method : 0.16152477264404297 time for calcul the mask position with numpy : 0.058351993560791016 nb_pixel_total : 19909 time to create 1 rle with old method : 0.02251911163330078 time for calcul the mask position with numpy : 0.05344581604003906 nb_pixel_total : 732 time to create 1 rle with old method : 0.0010242462158203125 time for calcul the mask position with numpy : 0.05368971824645996 nb_pixel_total : 37185 time to create 1 rle with old method : 0.042127370834350586 time for calcul the mask position with numpy : 0.05409359931945801 nb_pixel_total : 55840 time to create 1 rle with old method : 0.06195569038391113 time for calcul the mask position with numpy : 0.053803443908691406 nb_pixel_total : 35015 time to create 1 rle with old method : 0.03900742530822754 time for calcul the mask position with numpy : 0.053360700607299805 nb_pixel_total : 41565 time to create 1 rle with old method : 0.04641461372375488 time for calcul the mask position with numpy : 0.05514407157897949 nb_pixel_total : 263341 time to create 1 rle with new method : 1.9619719982147217 time for calcul the mask position with numpy : 0.05539870262145996 nb_pixel_total : 389386 time to create 1 rle with new method : 0.7706096172332764 time for calcul the mask position with numpy : 0.054656267166137695 nb_pixel_total : 174720 time to create 1 rle with new method : 2.3353583812713623 time for calcul the mask position with numpy : 0.058310747146606445 nb_pixel_total : 54282 time to create 1 rle with old method : 0.06066441535949707 time for calcul the mask position with numpy : 0.055040597915649414 nb_pixel_total : 226418 time to create 1 rle with new method : 1.7338993549346924 time for calcul the mask position with numpy : 0.0538330078125 nb_pixel_total : 3476 time to create 1 rle with old method : 0.004312992095947266 time for calcul the mask position with numpy : 0.05552864074707031 nb_pixel_total : 57212 time to create 1 rle with old method : 0.08182477951049805 time for calcul the mask position with numpy : 0.05583310127258301 nb_pixel_total : 17393 time to create 1 rle with old method : 0.0219268798828125 time for calcul the mask position with numpy : 0.05367159843444824 nb_pixel_total : 54472 time to create 1 rle with old method : 0.06087636947631836 time for calcul the mask position with numpy : 0.053659915924072266 nb_pixel_total : 724 time to create 1 rle with old method : 0.0011546611785888672 time for calcul the mask position with numpy : 0.054780006408691406 nb_pixel_total : 54265 time to create 1 rle with old method : 0.060868263244628906 time for calcul the mask position with numpy : 0.05364704132080078 nb_pixel_total : 190 time to create 1 rle with old method : 0.00030541419982910156 time for calcul the mask position with numpy : 0.05363965034484863 nb_pixel_total : 22945 time to create 1 rle with old method : 0.026070356369018555 time for calcul the mask position with numpy : 0.05566120147705078 nb_pixel_total : 219823 time to create 1 rle with new method : 1.6209712028503418 time for calcul the mask position with numpy : 0.054572343826293945 nb_pixel_total : 38794 time to create 1 rle with old method : 0.04270792007446289 time for calcul the mask position with numpy : 0.05222892761230469 nb_pixel_total : 298 time to create 1 rle with old method : 0.0003848075866699219 time for calcul the mask position with numpy : 0.0558619499206543 nb_pixel_total : 113124 time to create 1 rle with old method : 0.12308025360107422 time for calcul the mask position with numpy : 0.05348563194274902 nb_pixel_total : 20319 time to create 1 rle with old method : 0.02242755889892578 time for calcul the mask position with numpy : 0.053380727767944336 nb_pixel_total : 2537 time to create 1 rle with old method : 0.0030412673950195312 time for calcul the mask position with numpy : 0.053557395935058594 nb_pixel_total : 10377 time to create 1 rle with old method : 0.011513233184814453 time for calcul the mask position with numpy : 0.05375480651855469 nb_pixel_total : 61668 time to create 1 rle with old method : 0.08775162696838379 time for calcul the mask position with numpy : 0.057424306869506836 nb_pixel_total : 140518 time to create 1 rle with old method : 0.1560230255126953 time for calcul the mask position with numpy : 0.05304360389709473 nb_pixel_total : 2424 time to create 1 rle with old method : 0.0027647018432617188 time for calcul the mask position with numpy : 0.053906917572021484 nb_pixel_total : 39751 time to create 1 rle with old method : 0.04433488845825195 time for calcul the mask position with numpy : 0.054064273834228516 nb_pixel_total : 48476 time to create 1 rle with old method : 0.05337929725646973 time for calcul the mask position with numpy : 0.055283308029174805 nb_pixel_total : 313041 time to create 1 rle with new method : 1.479738712310791 time for calcul the mask position with numpy : 0.052452802658081055 nb_pixel_total : 790 time to create 1 rle with old method : 0.0010194778442382812 time for calcul the mask position with numpy : 0.05431175231933594 nb_pixel_total : 18117 time to create 1 rle with old method : 0.020455121994018555 time for calcul the mask position with numpy : 0.054547786712646484 nb_pixel_total : 23594 time to create 1 rle with old method : 0.026880502700805664 time for calcul the mask position with numpy : 0.05330944061279297 nb_pixel_total : 7276 time to create 1 rle with old method : 0.008297443389892578 time for calcul the mask position with numpy : 0.0546567440032959 nb_pixel_total : 121637 time to create 1 rle with old method : 0.13479351997375488 time for calcul the mask position with numpy : 0.055632591247558594 nb_pixel_total : 98937 time to create 1 rle with old method : 0.12023043632507324 time for calcul the mask position with numpy : 0.05427360534667969 nb_pixel_total : 8258 time to create 1 rle with old method : 0.009850025177001953 time for calcul the mask position with numpy : 0.055606842041015625 nb_pixel_total : 50860 time to create 1 rle with old method : 0.0576167106628418 time for calcul the mask position with numpy : 0.05289578437805176 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002288818359375 time for calcul the mask position with numpy : 0.055034637451171875 nb_pixel_total : 1483 time to create 1 rle with old method : 0.0018436908721923828 create new chi : 36.23882699012756 time to delete rle : 0.005307197570800781 batch 1 Loaded 77 chid ids of type : 4855 Number RLEs to save : 54027 TO DO : save crop sub photo not yet done ! save time : 2.9646270275115967 nb_obj : 59 nb_hashtags : 8 time to prepare the origin masks : 48.4875054359436 time for calcul the mask position with numpy : 1.0787060260772705 nb_pixel_total : 7514959 time to create 1 rle with new method : 1.5518908500671387 time for calcul the mask position with numpy : 0.0528411865234375 nb_pixel_total : 36468 time to create 1 rle with old method : 0.039549827575683594 time for calcul the mask position with numpy : 0.05321002006530762 nb_pixel_total : 11506 time to create 1 rle with old method : 0.013526678085327148 time for calcul the mask position with numpy : 0.16551637649536133 nb_pixel_total : 708683 time to create 1 rle with new method : 2.291199207305908 time for calcul the mask position with numpy : 0.06331372261047363 nb_pixel_total : 574613 time to create 1 rle with new method : 1.583005428314209 time for calcul the mask position with numpy : 0.052088260650634766 nb_pixel_total : 20035 time to create 1 rle with old method : 0.021424531936645508 time for calcul the mask position with numpy : 0.05626726150512695 nb_pixel_total : 43771 time to create 1 rle with old method : 0.04623126983642578 time for calcul the mask position with numpy : 0.0531005859375 nb_pixel_total : 19599 time to create 1 rle with old method : 0.021229267120361328 time for calcul the mask position with numpy : 0.052358388900756836 nb_pixel_total : 52990 time to create 1 rle with old method : 0.0585479736328125 time for calcul the mask position with numpy : 0.05292868614196777 nb_pixel_total : 6054 time to create 1 rle with old method : 0.0067288875579833984 time for calcul the mask position with numpy : 0.05520272254943848 nb_pixel_total : 95988 time to create 1 rle with old method : 0.10672235488891602 time for calcul the mask position with numpy : 0.05658984184265137 nb_pixel_total : 152774 time to create 1 rle with new method : 1.4575788974761963 time for calcul the mask position with numpy : 0.0537562370300293 nb_pixel_total : 104310 time to create 1 rle with old method : 0.10970187187194824 time for calcul the mask position with numpy : 0.053054094314575195 nb_pixel_total : 39004 time to create 1 rle with old method : 0.04073762893676758 time for calcul the mask position with numpy : 0.053537607192993164 nb_pixel_total : 123454 time to create 1 rle with old method : 0.1497189998626709 time for calcul the mask position with numpy : 0.05389904975891113 nb_pixel_total : 286443 time to create 1 rle with new method : 1.6795592308044434 time for calcul the mask position with numpy : 0.05637860298156738 nb_pixel_total : 26390 time to create 1 rle with old method : 0.029952287673950195 time for calcul the mask position with numpy : 0.054735422134399414 nb_pixel_total : 54384 time to create 1 rle with old method : 0.06458806991577148 time for calcul the mask position with numpy : 0.05527520179748535 nb_pixel_total : 38748 time to create 1 rle with old method : 0.043616533279418945 time for calcul the mask position with numpy : 0.05557370185852051 nb_pixel_total : 798 time to create 1 rle with old method : 0.0008995532989501953 time for calcul the mask position with numpy : 0.055858612060546875 nb_pixel_total : 19479 time to create 1 rle with old method : 0.021846294403076172 time for calcul the mask position with numpy : 0.052170753479003906 nb_pixel_total : 30689 time to create 1 rle with old method : 0.035178422927856445 time for calcul the mask position with numpy : 0.05672478675842285 nb_pixel_total : 1731 time to create 1 rle with old method : 0.002085447311401367 time for calcul the mask position with numpy : 0.0539703369140625 nb_pixel_total : 11292 time to create 1 rle with old method : 0.012441158294677734 time for calcul the mask position with numpy : 0.05728864669799805 nb_pixel_total : 28421 time to create 1 rle with old method : 0.03213787078857422 time for calcul the mask position with numpy : 0.05599856376647949 nb_pixel_total : 17155 time to create 1 rle with old method : 0.019290447235107422 time for calcul the mask position with numpy : 0.05486249923706055 nb_pixel_total : 41875 time to create 1 rle with old method : 0.04742145538330078 time for calcul the mask position with numpy : 0.0564727783203125 nb_pixel_total : 185465 time to create 1 rle with new method : 2.197361707687378 time for calcul the mask position with numpy : 0.0594477653503418 nb_pixel_total : 328950 time to create 1 rle with new method : 1.7926182746887207 time for calcul the mask position with numpy : 0.055284738540649414 nb_pixel_total : 22447 time to create 1 rle with old method : 0.025231599807739258 time for calcul the mask position with numpy : 0.05466866493225098 nb_pixel_total : 36 time to create 1 rle with old method : 0.00036597251892089844 time for calcul the mask position with numpy : 0.05282306671142578 nb_pixel_total : 210346 time to create 1 rle with new method : 1.3297536373138428 time for calcul the mask position with numpy : 0.05594921112060547 nb_pixel_total : 41849 time to create 1 rle with old method : 0.04654860496520996 time for calcul the mask position with numpy : 0.056931257247924805 nb_pixel_total : 128117 time to create 1 rle with old method : 0.13865160942077637 time for calcul the mask position with numpy : 0.05208230018615723 nb_pixel_total : 16256 time to create 1 rle with old method : 0.017064809799194336 time for calcul the mask position with numpy : 0.05254244804382324 nb_pixel_total : 38215 time to create 1 rle with old method : 0.0411067008972168 time for calcul the mask position with numpy : 0.05220365524291992 nb_pixel_total : 39514 time to create 1 rle with old method : 0.043442726135253906 time for calcul the mask position with numpy : 0.05539965629577637 nb_pixel_total : 299596 time to create 1 rle with new method : 1.6903588771820068 time for calcul the mask position with numpy : 0.05472111701965332 nb_pixel_total : 24393 time to create 1 rle with old method : 0.027341365814208984 time for calcul the mask position with numpy : 0.05717182159423828 nb_pixel_total : 216550 time to create 1 rle with new method : 0.9162464141845703 time for calcul the mask position with numpy : 0.05567002296447754 nb_pixel_total : 55669 time to create 1 rle with old method : 0.06303763389587402 time for calcul the mask position with numpy : 0.054494380950927734 nb_pixel_total : 20672 time to create 1 rle with old method : 0.02307295799255371 time for calcul the mask position with numpy : 0.0547785758972168 nb_pixel_total : 12943 time to create 1 rle with old method : 0.014693975448608398 time for calcul the mask position with numpy : 0.06024742126464844 nb_pixel_total : 315591 time to create 1 rle with new method : 2.044921636581421 time for calcul the mask position with numpy : 0.052347660064697266 nb_pixel_total : 86584 time to create 1 rle with old method : 0.09264659881591797 time for calcul the mask position with numpy : 0.052808284759521484 nb_pixel_total : 15716 time to create 1 rle with old method : 0.017759084701538086 time for calcul the mask position with numpy : 0.05390572547912598 nb_pixel_total : 161125 time to create 1 rle with new method : 1.1351242065429688 time for calcul the mask position with numpy : 0.05556082725524902 nb_pixel_total : 11973 time to create 1 rle with old method : 0.013624191284179688 time for calcul the mask position with numpy : 0.05628180503845215 nb_pixel_total : 48807 time to create 1 rle with old method : 0.05436897277832031 time for calcul the mask position with numpy : 0.05557990074157715 nb_pixel_total : 107890 time to create 1 rle with old method : 0.11902260780334473 time for calcul the mask position with numpy : 0.055866241455078125 nb_pixel_total : 101543 time to create 1 rle with old method : 0.1169586181640625 time for calcul the mask position with numpy : 0.061326026916503906 nb_pixel_total : 149712 time to create 1 rle with old method : 0.1599738597869873 time for calcul the mask position with numpy : 0.05338573455810547 nb_pixel_total : 427 time to create 1 rle with old method : 0.0009102821350097656 time for calcul the mask position with numpy : 0.0514225959777832 nb_pixel_total : 53661 time to create 1 rle with old method : 0.05772805213928223 time for calcul the mask position with numpy : 0.0525815486907959 nb_pixel_total : 39104 time to create 1 rle with old method : 0.04093194007873535 time for calcul the mask position with numpy : 0.05376410484313965 nb_pixel_total : 7640 time to create 1 rle with old method : 0.008351325988769531 time for calcul the mask position with numpy : 0.05305743217468262 nb_pixel_total : 46898 time to create 1 rle with old method : 0.05043315887451172 time for calcul the mask position with numpy : 0.054001808166503906 nb_pixel_total : 3441 time to create 1 rle with old method : 0.003949642181396484 time for calcul the mask position with numpy : 0.055278778076171875 nb_pixel_total : 108625 time to create 1 rle with old method : 0.12069439888000488 time for calcul the mask position with numpy : 0.05306553840637207 nb_pixel_total : 17832 time to create 1 rle with old method : 0.02034592628479004 create new chi : 27.212488412857056 time to delete rle : 0.004988908767700195 batch 1 Loaded 60 chid ids of type : 4855 Number RLEs to save : 42181 TO DO : save crop sub photo not yet done ! save time : 2.360473394393921 nb_obj : 13 nb_hashtags : 6 time to prepare the origin masks : 36.625542879104614 time for calcul the mask position with numpy : 0.4116387367248535 nb_pixel_total : 1467163 time to create 1 rle with new method : 1.902787208557129 time for calcul the mask position with numpy : 0.07513713836669922 nb_pixel_total : 2767 time to create 1 rle with old method : 0.0031003952026367188 time for calcul the mask position with numpy : 0.07367825508117676 nb_pixel_total : 64826 time to create 1 rle with old method : 0.0702962875366211 time for calcul the mask position with numpy : 0.07711434364318848 nb_pixel_total : 369285 time to create 1 rle with new method : 2.0573441982269287 time for calcul the mask position with numpy : 0.06715106964111328 nb_pixel_total : 77647 time to create 1 rle with old method : 0.08758664131164551 time for calcul the mask position with numpy : 0.04661417007446289 nb_pixel_total : 19883 time to create 1 rle with old method : 0.022538185119628906 time for calcul the mask position with numpy : 0.04555773735046387 nb_pixel_total : 19913 time to create 1 rle with old method : 0.031052112579345703 time for calcul the mask position with numpy : 0.04442191123962402 nb_pixel_total : 161241 time to create 1 rle with new method : 1.9966001510620117 time for calcul the mask position with numpy : 0.11038064956665039 nb_pixel_total : 1771239 time to create 1 rle with new method : 2.2242026329040527 time for calcul the mask position with numpy : 0.08092117309570312 nb_pixel_total : 25984 time to create 1 rle with old method : 0.03019428253173828 time for calcul the mask position with numpy : 0.0804896354675293 nb_pixel_total : 448146 time to create 1 rle with new method : 2.2808663845062256 time for calcul the mask position with numpy : 0.08113312721252441 nb_pixel_total : 495147 time to create 1 rle with new method : 1.4743096828460693 time for calcul the mask position with numpy : 0.053940534591674805 nb_pixel_total : 819626 time to create 1 rle with new method : 1.2998523712158203 time for calcul the mask position with numpy : 0.8707931041717529 nb_pixel_total : 7236333 time to create 1 rle with new method : 1.9746501445770264 create new chi : 18.01084876060486 time to delete rle : 0.002982616424560547 batch 1 Loaded 14 chid ids of type : 4855 Number RLEs to save : 27279 TO DO : save crop sub photo not yet done ! save time : 1.4788551330566406 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 26.5678653717041 time for calcul the mask position with numpy : 0.41056132316589355 nb_pixel_total : 4417283 time to create 1 rle with new method : 1.2762305736541748 time for calcul the mask position with numpy : 0.06943559646606445 nb_pixel_total : 81 time to create 1 rle with old method : 0.000152587890625 time for calcul the mask position with numpy : 0.07600259780883789 nb_pixel_total : 32404 time to create 1 rle with old method : 0.036255598068237305 time for calcul the mask position with numpy : 0.06958627700805664 nb_pixel_total : 3301 time to create 1 rle with old method : 0.004254341125488281 time for calcul the mask position with numpy : 0.0688631534576416 nb_pixel_total : 561 time to create 1 rle with old method : 0.0010373592376708984 time for calcul the mask position with numpy : 0.07068681716918945 nb_pixel_total : 11447 time to create 1 rle with old method : 0.014390230178833008 time for calcul the mask position with numpy : 0.0816354751586914 nb_pixel_total : 573780 time to create 1 rle with new method : 1.220895528793335 time for calcul the mask position with numpy : 0.30375146865844727 nb_pixel_total : 2136935 time to create 1 rle with new method : 1.2845830917358398 time for calcul the mask position with numpy : 0.07152128219604492 nb_pixel_total : 525734 time to create 1 rle with new method : 1.2867000102996826 time for calcul the mask position with numpy : 0.07617449760437012 nb_pixel_total : 662136 time to create 1 rle with new method : 1.671217441558838 time for calcul the mask position with numpy : 0.07959461212158203 nb_pixel_total : 993101 time to create 1 rle with new method : 0.9476771354675293 time for calcul the mask position with numpy : 0.06850290298461914 nb_pixel_total : 97242 time to create 1 rle with old method : 0.10881543159484863 time for calcul the mask position with numpy : 0.4260876178741455 nb_pixel_total : 3525195 time to create 1 rle with new method : 0.9951701164245605 create new chi : 11.066378116607666 time to delete rle : 0.0037894248962402344 batch 1 Loaded 13 chid ids of type : 4855 Number RLEs to save : 30226 TO DO : save crop sub photo not yet done ! save time : 1.693716287612915 map_output_result : {1356331559: (0.38152165058341075, 'Should be the crop_list due to order', 0.3730428270170931), 1356331557: (0.38152165058341075, 'Should be the crop_list due to order', 0.2975159940553483), 1356331553: (0.38152165058341075, 'Should be the crop_list due to order', 0.5577164188672439), 1356331545: (0.38152165058341075, 'Should be the crop_list due to order', 0.3725342229072873), 1356331538: (0.38152165058341075, 'Should be the crop_list due to order', 0.5310371985464155), 1356331534: (0.38152165058341075, 'Should be the crop_list due to order', 0.1572832421070763)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 6 /1356331559.Didn't retrieve data . /1356331557.Didn't retrieve data . /1356331553.Didn't retrieve data . /1356331545.Didn't retrieve data . /1356331538.Didn't retrieve data . /1356331534.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.012664794921875 save_final save missing photos in datou_result : time spend for datou_step_exec : 480.79900908470154 time spend to save output : 0.022154808044433594 total time spend for step 5 : 480.821163892746 step6:crop_condition Tue May 6 13:13:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 6 ! batch 1 Loaded 336 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 3015255 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530027_829539 we have uploaded 19 photos in the portfolio 3015255 time of upload the photos Elapsed time : 4.976799964904785 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 59 About to insert : list_path_to_insert length 59 new photo from crops ! About to upload 59 photos upload in portfolio : 3015255 init cache_photo without model_param we have 59 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530071_829539 we have uploaded 59 photos in the portfolio 3015255 time of upload the photos Elapsed time : 13.713775873184204 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 44 About to insert : list_path_to_insert length 44 new photo from crops ! About to upload 44 photos upload in portfolio : 3015255 init cache_photo without model_param we have 44 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530099_829539 we have uploaded 44 photos in the portfolio 3015255 time of upload the photos Elapsed time : 10.861149072647095 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3015255 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530114_829539 we have uploaded 3 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.0405547618865967 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3015255 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530121_829539 we have uploaded 5 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.3641488552093506 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 32 About to insert : list_path_to_insert length 32 new photo from crops ! About to upload 32 photos upload in portfolio : 3015255 init cache_photo without model_param we have 32 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530145_829539 we have uploaded 32 photos in the portfolio 3015255 time of upload the photos Elapsed time : 8.316970586776733 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530157_829539 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.6823470592498779 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530162_829539 we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.6860854625701904 we have finished the crop for the class : encombrant begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 3015255 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530183_829539 we have uploaded 19 photos in the portfolio 3015255 time of upload the photos Elapsed time : 5.811980724334717 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! About to upload 13 photos upload in portfolio : 3015255 init cache_photo without model_param we have 13 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530200_829539 we have uploaded 13 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.999265193939209 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 43 About to insert : list_path_to_insert length 43 new photo from crops ! About to upload 43 photos upload in portfolio : 3015255 init cache_photo without model_param we have 43 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746530406_829539 we have uploaded 43 photos in the portfolio 3015255 time of upload the photos Elapsed time : 12.580236911773682 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 240 /1356351678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351696Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351767Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351906Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351907Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351957Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356351999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356352358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 726 time used for this insertion : 0.06691145896911621 save_final save missing photos in datou_result : time spend for datou_step_exec : 423.9088044166565 time spend to save output : 0.08527112007141113 total time spend for step 6 : 423.9940755367279 step7:ventilate_hashtags_in_portfolio Tue May 6 13:20:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22713553 get user id for portfolio 22713553 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713553 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713553 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713553 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713553 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22715725,22715726,22715727,22715728,22715729,22715734,22715736,22715737,22715738,22715739,22715740,22715741,22715743,22715744,22715745,22715746,22715747,22715748,22715749,22715750?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713553 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 1 /22713553. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.013854265213012695 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.1730217933654785 time spend to save output : 0.014259099960327148 total time spend for step 7 : 4.187280893325806 step8:final Tue May 6 13:20:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356331559: ('0.38812842977262446',), 1356331557: ('0.38812842977262446',), 1356331553: ('0.38812842977262446',), 1356331545: ('0.38812842977262446',), 1356331538: ('0.38812842977262446',), 1356331534: ('0.38812842977262446',)} new output for save of step final : {1356331559: ('0.38812842977262446',), 1356331557: ('0.38812842977262446',), 1356331553: ('0.38812842977262446',), 1356331545: ('0.38812842977262446',), 1356331538: ('0.38812842977262446',), 1356331534: ('0.38812842977262446',)} [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 6 /1356331559.Didn't retrieve data . /1356331557.Didn't retrieve data . /1356331553.Didn't retrieve data . /1356331545.Didn't retrieve data . /1356331538.Didn't retrieve data . /1356331534.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.012525558471679688 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11969637870788574 time spend to save output : 0.012972831726074219 total time spend for step 8 : 0.13266921043395996 step9:velours_tree Tue May 6 13:20:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.3733510971069336 time spend to save output : 7.772445678710938e-05 total time spend for step 9 : 0.3734288215637207 step10:send_mail_cod Tue May 6 13:20:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22713553_06-05-2025_13_20_26.pdf 22715699 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227156991746530426 22715700 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157001746530428 22715701 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157011746530430 22715702 change filename to text .change filename to text .change filename to text .imagette227157021746530431 22715703 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157031746530431 22715704 imagette227157041746530432 22715705 imagette227157051746530432 22715706 imagette227157061746530432 22715707 imagette227157071746530432 22715708 imagette227157081746530432 22715709 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157091746530432 22715710 change filename to text .change filename to text .imagette227157101746530433 22715711 imagette227157111746530434 22715712 imagette227157121746530434 22715713 imagette227157131746530434 22715714 imagette227157141746530434 22715715 change filename to text .imagette227157151746530434 22715716 imagette227157161746530434 22715717 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157171746530434 22715719 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227157191746530435 22715720 imagette227157201746530437 22715721 imagette227157211746530437 22715722 imagette227157221746530437 22715723 imagette227157231746530437 22715724 imagette227157241746530437 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22713553 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22715725,22715726,22715727,22715728,22715729,22715734,22715736,22715737,22715738,22715739,22715740,22715741,22715743,22715744,22715745,22715746,22715747,22715748,22715749,22715750?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713553 your option no_mail is active, we will not send the real mail to your client args[1356331559] : ((1356331559, 3466.2512972873333, 492609224), (1356331559, -0.32245878578947745, 496442774), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331557] : ((1356331557, 4150.154521630281, 492609224), (1356331557, -0.40699100811854577, 496442774), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331553] : ((1356331553, 4608.056408580415, 492609224), (1356331553, -0.35266089241336934, 496442774), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331545] : ((1356331545, 6776.4835558636105, 492609224), (1356331545, -0.3728085802633376, 496442774), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331538] : ((1356331538, 417.1945760873511, 492688767), (1356331538, -0.622947492019942, 501862349), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331534] : ((1356331534, 278.3087073565592, 492688767), (1356331534, 0.17983277832816963, 2107752395), '0.38812842977262446') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.38812842977262446 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713553_06-05-2025_13_20_26.pdf results_Auto_P22713553_06-05-2025_13_20_26.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713553_06-05-2025_13_20_26.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22713553','results_Auto_P22713553_06-05-2025_13_20_26.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713553_06-05-2025_13_20_26.pdf','pdf','','2.76','0.38812842977262446') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.012191295623779297 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.610160112380981 time spend to save output : 0.012358665466308594 total time spend for step 10 : 14.62251877784729 step11:split_time_score Tue May 6 13:20:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('12', 2), ('13', 3), ('14', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 29042025 22713553 Nombre de photos uploadées : 6 / 23040 (0%) 29042025 22713553 Nombre de photos taguées (types de déchets): 0 / 6 (0%) 29042025 22713553 Nombre de photos taguées (volume) : 0 / 6 (0%) elapsed_time : load_data_split_time_score 3.337860107421875e-06 elapsed_time : order_list_meta_photo_and_scores 1.0967254638671875e-05 ?????? elapsed_time : fill_and_build_computed_from_old_data 0.0004019737243652344 elapsed_time : insert_dashboard_record_day_entry 0.024260759353637695 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.38812842977262446 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713553_06-05-2025_13_20_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713553 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713553 AND mptpi.`type`=4855 To do Qualite : 0.34832000040650074 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713554_06-05-2025_11_57_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713554 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713554 AND mptpi.`type`=4855 To do Qualite : 0.43681127303932576 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713555_06-05-2025_12_51_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713555 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713555 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'29042025': {'nb_upload': 6, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356331559, 1356331557, 1356331553, 1356331545, 1356331538, 1356331534] Looping around the photos to save general results len do output : 1 /22713553Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331559', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331557', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331553', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331545', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331538', None, None, None, None, None, '2797335') ('4741', None, None, None, None, None, None, None, '2797335') ('4741', '22713553', '1356331534', None, None, None, None, None, '2797335') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.013498067855834961 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3689537048339844 time spend to save output : 0.01368403434753418 total time spend for step 11 : 0.38263773918151855 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 6 set_done_treatment 689.77user 408.05system 19:16.56elapsed 94%CPU (0avgtext+0avgdata 6729976maxresident)k 2116512inputs+843280outputs (29310major+54660348minor)pagefaults 0swaps