python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3814560 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['3623814'] with mtr_portfolio_ids : ['26291505'] and first list_photo_ids : [] new path : /proc/3814560/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 4 ; length of list_pids : 4 ; length of list_args : 4 time to download the photos : 0.9306766986846924 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Aug 28 14:41:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 1534 wait 20 seconds l 3637 free memory gpu now : 1534 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-28 14:41:51.504454: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-28 14:41:51.532508: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-28 14:41:51.533850: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5efc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-28 14:41:51.533873: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-28 14:41:51.536716: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-28 14:41:51.670347: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4623820 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-28 14:41:51.670397: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-28 14:41:51.671496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:41:51.671918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:41:51.674934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:41:51.677522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:41:51.677889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:41:51.702422: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:41:51.704507: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:41:51.712408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:41:51.714040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:41:51.714101: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:41:51.714771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 14:41:51.714787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 14:41:51.714796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 14:41:51.715970: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6271 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-28 14:41:52.007762: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:41:52.007878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:41:52.007905: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:41:52.007931: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:41:52.007955: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:41:52.007979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:41:52.008002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:41:52.008025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:41:52.009424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:41:52.010920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:41:52.010982: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:41:52.011004: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:41:52.011035: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:41:52.011055: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:41:52.011074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:41:52.011092: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:41:52.011111: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:41:52.012473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:41:52.012517: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 14:41:52.012530: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 14:41:52.012540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 14:41:52.013953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6271 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-28 14:42:03.040472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:42:03.254125: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 3816013 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 13 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6823 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.0020825862884521484 nb_pixel_total : 70672 time to create 1 rle with old method : 0.08518695831298828 length of segment : 243 time for calcul the mask position with numpy : 0.012709856033325195 nb_pixel_total : 280392 time to create 1 rle with new method : 0.015568017959594727 length of segment : 478 time for calcul the mask position with numpy : 0.0029401779174804688 nb_pixel_total : 92492 time to create 1 rle with old method : 0.10920023918151855 length of segment : 546 time for calcul the mask position with numpy : 0.0014357566833496094 nb_pixel_total : 101505 time to create 1 rle with old method : 0.13304376602172852 length of segment : 466 time for calcul the mask position with numpy : 0.0006685256958007812 nb_pixel_total : 44512 time to create 1 rle with old method : 0.0639650821685791 length of segment : 229 time for calcul the mask position with numpy : 0.0002803802490234375 nb_pixel_total : 13576 time to create 1 rle with old method : 0.01609635353088379 length of segment : 147 time for calcul the mask position with numpy : 0.0013637542724609375 nb_pixel_total : 76758 time to create 1 rle with old method : 0.08940005302429199 length of segment : 495 time for calcul the mask position with numpy : 0.0012841224670410156 nb_pixel_total : 93417 time to create 1 rle with old method : 0.13022613525390625 length of segment : 202 time for calcul the mask position with numpy : 0.0021283626556396484 nb_pixel_total : 58387 time to create 1 rle with old method : 0.10117483139038086 length of segment : 329 time for calcul the mask position with numpy : 0.0017406940460205078 nb_pixel_total : 37980 time to create 1 rle with old method : 0.05467820167541504 length of segment : 312 time for calcul the mask position with numpy : 0.0005381107330322266 nb_pixel_total : 28124 time to create 1 rle with old method : 0.034027099609375 length of segment : 149 time for calcul the mask position with numpy : 0.001039266586303711 nb_pixel_total : 25472 time to create 1 rle with old method : 0.029650211334228516 length of segment : 263 time for calcul the mask position with numpy : 0.001590728759765625 nb_pixel_total : 85073 time to create 1 rle with old method : 0.09902191162109375 length of segment : 530 time for calcul the mask position with numpy : 0.0011060237884521484 nb_pixel_total : 47454 time to create 1 rle with old method : 0.0654456615447998 length of segment : 277 time for calcul the mask position with numpy : 0.008994102478027344 nb_pixel_total : 76092 time to create 1 rle with old method : 0.09513115882873535 length of segment : 502 time for calcul the mask position with numpy : 0.0006351470947265625 nb_pixel_total : 38547 time to create 1 rle with old method : 0.04748654365539551 length of segment : 250 time for calcul the mask position with numpy : 0.0013322830200195312 nb_pixel_total : 83889 time to create 1 rle with old method : 0.10814642906188965 length of segment : 533 time for calcul the mask position with numpy : 0.0024602413177490234 nb_pixel_total : 99804 time to create 1 rle with old method : 0.12083601951599121 length of segment : 315 time for calcul the mask position with numpy : 0.0005822181701660156 nb_pixel_total : 34757 time to create 1 rle with old method : 0.040601491928100586 length of segment : 368 time for calcul the mask position with numpy : 0.0012485980987548828 nb_pixel_total : 92226 time to create 1 rle with old method : 0.10811305046081543 length of segment : 418 time for calcul the mask position with numpy : 0.00048732757568359375 nb_pixel_total : 23541 time to create 1 rle with old method : 0.04196667671203613 length of segment : 121 time for calcul the mask position with numpy : 0.002681255340576172 nb_pixel_total : 76514 time to create 1 rle with old method : 0.09178757667541504 length of segment : 367 time for calcul the mask position with numpy : 0.0009644031524658203 nb_pixel_total : 25528 time to create 1 rle with old method : 0.03412747383117676 length of segment : 401 time for calcul the mask position with numpy : 0.0008261203765869141 nb_pixel_total : 19057 time to create 1 rle with old method : 0.022849321365356445 length of segment : 209 time for calcul the mask position with numpy : 0.00030517578125 nb_pixel_total : 5852 time to create 1 rle with old method : 0.007464170455932617 length of segment : 59 time for calcul the mask position with numpy : 0.0016663074493408203 nb_pixel_total : 86103 time to create 1 rle with old method : 0.10104107856750488 length of segment : 316 time for calcul the mask position with numpy : 0.0017426013946533203 nb_pixel_total : 71180 time to create 1 rle with old method : 0.08796977996826172 length of segment : 350 time for calcul the mask position with numpy : 0.00043392181396484375 nb_pixel_total : 23702 time to create 1 rle with old method : 0.0596308708190918 length of segment : 120 time for calcul the mask position with numpy : 0.0024564266204833984 nb_pixel_total : 57367 time to create 1 rle with old method : 0.10427618026733398 length of segment : 272 time for calcul the mask position with numpy : 0.0007929801940917969 nb_pixel_total : 51329 time to create 1 rle with old method : 0.05932450294494629 length of segment : 284 time for calcul the mask position with numpy : 0.0021047592163085938 nb_pixel_total : 81818 time to create 1 rle with old method : 0.0949394702911377 length of segment : 309 time for calcul the mask position with numpy : 0.000881195068359375 nb_pixel_total : 61847 time to create 1 rle with old method : 0.07098722457885742 length of segment : 242 time for calcul the mask position with numpy : 0.00017547607421875 nb_pixel_total : 7542 time to create 1 rle with old method : 0.009332418441772461 length of segment : 81 time for calcul the mask position with numpy : 0.0003681182861328125 nb_pixel_total : 24827 time to create 1 rle with old method : 0.030901432037353516 length of segment : 179 time for calcul the mask position with numpy : 0.0011706352233886719 nb_pixel_total : 76812 time to create 1 rle with old method : 0.08817505836486816 length of segment : 525 time for calcul the mask position with numpy : 0.0002968311309814453 nb_pixel_total : 17563 time to create 1 rle with old method : 0.025325775146484375 length of segment : 115 time for calcul the mask position with numpy : 0.0021986961364746094 nb_pixel_total : 79274 time to create 1 rle with old method : 0.10942625999450684 length of segment : 358 time for calcul the mask position with numpy : 0.005066633224487305 nb_pixel_total : 200626 time to create 1 rle with new method : 0.01391148567199707 length of segment : 516 time for calcul the mask position with numpy : 0.025144100189208984 nb_pixel_total : 561191 time to create 1 rle with new method : 0.919381856918335 length of segment : 834 time for calcul the mask position with numpy : 0.13454842567443848 nb_pixel_total : 987180 time to create 1 rle with new method : 0.0513157844543457 length of segment : 631 time for calcul the mask position with numpy : 0.27800488471984863 nb_pixel_total : 2288814 time to create 1 rle with new method : 0.47418880462646484 length of segment : 3284 time for calcul the mask position with numpy : 0.3449246883392334 nb_pixel_total : 4046178 time to create 1 rle with new method : 0.375044584274292 length of segment : 1942 time for calcul the mask position with numpy : 0.02242445945739746 nb_pixel_total : 956434 time to create 1 rle with new method : 0.2034451961517334 length of segment : 2130 time for calcul the mask position with numpy : 0.1708364486694336 nb_pixel_total : 2696362 time to create 1 rle with new method : 0.37554073333740234 length of segment : 1218 time for calcul the mask position with numpy : 0.1253528594970703 nb_pixel_total : 2412592 time to create 1 rle with new method : 0.2149348258972168 length of segment : 1435 time spent for convertir_results : 14.50862979888916 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 45 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 69.86680912971497 time spend to save output : 0.10110306739807129 total time spend for step 1 : 69.96791219711304 step2:mask_detect Thu Aug 28 14:42:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6823 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-08-28 14:42:41.600070: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-08-28 14:42:41.608183: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-08-28 14:42:41.609744: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5f00000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-08-28 14:42:41.609799: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-08-28 14:42:41.612522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-08-28 14:42:41.770876: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5a28d70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-08-28 14:42:41.770940: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-08-28 14:42:41.772337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:42:41.776610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:42:41.781041: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:42:41.784625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:42:41.785283: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:42:41.789099: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:42:41.790968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:42:41.797787: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:42:41.799277: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:42:41.799364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:42:41.800103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 14:42:41.800119: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 14:42:41.800128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 14:42:41.801449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-08-28 14:42:41.977188: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:42:41.977347: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:42:41.977384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:42:41.977418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:42:41.977450: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:42:41.977481: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:42:41.977513: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:42:41.977544: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:42:41.978766: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:42:41.979909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-08-28 14:42:41.979943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-08-28 14:42:41.979961: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:42:41.979977: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-08-28 14:42:41.979993: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-08-28 14:42:41.980008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-08-28 14:42:41.980024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-08-28 14:42:41.980040: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-08-28 14:42:41.981288: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-08-28 14:42:41.981323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-08-28 14:42:41.981331: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-08-28 14:42:41.981339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-08-28 14:42:41.982573: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9821 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-08-28 14:42:52.515365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-08-28 14:42:52.685382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 61 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 74 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 77 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 3818606 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5158 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6823 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.01557612419128418 nb_pixel_total : 225746 time to create 1 rle with new method : 0.049799203872680664 length of segment : 759 time for calcul the mask position with numpy : 0.002431154251098633 nb_pixel_total : 65148 time to create 1 rle with old method : 0.0875852108001709 length of segment : 314 time for calcul the mask position with numpy : 0.0018215179443359375 nb_pixel_total : 79071 time to create 1 rle with old method : 0.09415745735168457 length of segment : 391 time for calcul the mask position with numpy : 0.0027735233306884766 nb_pixel_total : 109505 time to create 1 rle with old method : 0.16535711288452148 length of segment : 488 time for calcul the mask position with numpy : 0.0061724185943603516 nb_pixel_total : 258375 time to create 1 rle with new method : 0.019672870635986328 length of segment : 671 time for calcul the mask position with numpy : 0.025205373764038086 nb_pixel_total : 865510 time to create 1 rle with new method : 0.485576868057251 length of segment : 1309 time for calcul the mask position with numpy : 0.0034127235412597656 nb_pixel_total : 156813 time to create 1 rle with new method : 0.00946950912475586 length of segment : 459 time for calcul the mask position with numpy : 0.012640953063964844 nb_pixel_total : 549023 time to create 1 rle with new method : 0.13661670684814453 length of segment : 929 time for calcul the mask position with numpy : 0.0006768703460693359 nb_pixel_total : 21937 time to create 1 rle with old method : 0.02637481689453125 length of segment : 221 time for calcul the mask position with numpy : 0.017023563385009766 nb_pixel_total : 717416 time to create 1 rle with new method : 0.07691574096679688 length of segment : 1310 time for calcul the mask position with numpy : 0.002833843231201172 nb_pixel_total : 167378 time to create 1 rle with new method : 0.007798671722412109 length of segment : 446 time for calcul the mask position with numpy : 0.005469322204589844 nb_pixel_total : 264478 time to create 1 rle with new method : 0.01645207405090332 length of segment : 518 time for calcul the mask position with numpy : 0.005360126495361328 nb_pixel_total : 240046 time to create 1 rle with new method : 0.014124155044555664 length of segment : 745 time for calcul the mask position with numpy : 0.0018610954284667969 nb_pixel_total : 96698 time to create 1 rle with old method : 0.142930269241333 length of segment : 557 time for calcul the mask position with numpy : 0.0041043758392333984 nb_pixel_total : 205189 time to create 1 rle with new method : 0.012367486953735352 length of segment : 564 time for calcul the mask position with numpy : 0.0008761882781982422 nb_pixel_total : 38055 time to create 1 rle with old method : 0.043746232986450195 length of segment : 275 time for calcul the mask position with numpy : 0.01403355598449707 nb_pixel_total : 298194 time to create 1 rle with new method : 0.07440352439880371 length of segment : 1007 time for calcul the mask position with numpy : 0.005082845687866211 nb_pixel_total : 188182 time to create 1 rle with new method : 0.013437986373901367 length of segment : 482 time for calcul the mask position with numpy : 0.003391265869140625 nb_pixel_total : 189324 time to create 1 rle with new method : 0.010087251663208008 length of segment : 381 time for calcul the mask position with numpy : 0.0033028125762939453 nb_pixel_total : 113458 time to create 1 rle with old method : 0.1331193447113037 length of segment : 705 time for calcul the mask position with numpy : 0.0009224414825439453 nb_pixel_total : 39193 time to create 1 rle with old method : 0.04583930969238281 length of segment : 206 time for calcul the mask position with numpy : 0.002875089645385742 nb_pixel_total : 113789 time to create 1 rle with old method : 0.13530373573303223 length of segment : 408 time for calcul the mask position with numpy : 0.0015511512756347656 nb_pixel_total : 57400 time to create 1 rle with old method : 0.06813287734985352 length of segment : 331 time for calcul the mask position with numpy : 0.0021953582763671875 nb_pixel_total : 105336 time to create 1 rle with old method : 0.12876296043395996 length of segment : 457 time for calcul the mask position with numpy : 0.0005109310150146484 nb_pixel_total : 15385 time to create 1 rle with old method : 0.018563508987426758 length of segment : 275 time for calcul the mask position with numpy : 0.0011701583862304688 nb_pixel_total : 56599 time to create 1 rle with old method : 0.0667870044708252 length of segment : 358 time for calcul the mask position with numpy : 0.0061187744140625 nb_pixel_total : 321868 time to create 1 rle with new method : 0.015935897827148438 length of segment : 649 time for calcul the mask position with numpy : 0.01005101203918457 nb_pixel_total : 399360 time to create 1 rle with new method : 0.02822566032409668 length of segment : 1160 time for calcul the mask position with numpy : 0.004797220230102539 nb_pixel_total : 240142 time to create 1 rle with new method : 0.012516021728515625 length of segment : 361 time for calcul the mask position with numpy : 0.0010495185852050781 nb_pixel_total : 39117 time to create 1 rle with old method : 0.05501866340637207 length of segment : 231 time for calcul the mask position with numpy : 0.0015625953674316406 nb_pixel_total : 41273 time to create 1 rle with old method : 0.07888436317443848 length of segment : 213 time for calcul the mask position with numpy : 0.0022470951080322266 nb_pixel_total : 68175 time to create 1 rle with old method : 0.0984189510345459 length of segment : 513 time for calcul the mask position with numpy : 0.04014992713928223 nb_pixel_total : 1943281 time to create 1 rle with new method : 0.33098673820495605 length of segment : 1501 time for calcul the mask position with numpy : 0.002980470657348633 nb_pixel_total : 165407 time to create 1 rle with new method : 0.008661031723022461 length of segment : 805 time for calcul the mask position with numpy : 0.0010557174682617188 nb_pixel_total : 47194 time to create 1 rle with old method : 0.06736612319946289 length of segment : 307 time for calcul the mask position with numpy : 0.0020132064819335938 nb_pixel_total : 35073 time to create 1 rle with old method : 0.042531728744506836 length of segment : 265 time for calcul the mask position with numpy : 0.0038459300994873047 nb_pixel_total : 165882 time to create 1 rle with new method : 0.01155710220336914 length of segment : 546 time for calcul the mask position with numpy : 0.0038928985595703125 nb_pixel_total : 182129 time to create 1 rle with new method : 0.012224912643432617 length of segment : 491 time for calcul the mask position with numpy : 0.0014257431030273438 nb_pixel_total : 62211 time to create 1 rle with old method : 0.07147645950317383 length of segment : 184 time for calcul the mask position with numpy : 0.0012056827545166016 nb_pixel_total : 46380 time to create 1 rle with old method : 0.061309099197387695 length of segment : 245 time for calcul the mask position with numpy : 0.005608797073364258 nb_pixel_total : 234427 time to create 1 rle with new method : 0.017969369888305664 length of segment : 797 time for calcul the mask position with numpy : 0.007857084274291992 nb_pixel_total : 270385 time to create 1 rle with new method : 0.02129840850830078 length of segment : 474 time for calcul the mask position with numpy : 0.002774953842163086 nb_pixel_total : 98406 time to create 1 rle with old method : 0.1318666934967041 length of segment : 206 time for calcul the mask position with numpy : 0.0027437210083007812 nb_pixel_total : 89172 time to create 1 rle with old method : 0.1260547637939453 length of segment : 307 time for calcul the mask position with numpy : 0.0009748935699462891 nb_pixel_total : 42043 time to create 1 rle with old method : 0.051286935806274414 length of segment : 178 time for calcul the mask position with numpy : 0.0042362213134765625 nb_pixel_total : 134404 time to create 1 rle with old method : 0.15557217597961426 length of segment : 576 time for calcul the mask position with numpy : 0.0027093887329101562 nb_pixel_total : 91903 time to create 1 rle with old method : 0.10585141181945801 length of segment : 338 time for calcul the mask position with numpy : 0.008239269256591797 nb_pixel_total : 200616 time to create 1 rle with new method : 0.03662109375 length of segment : 1157 time for calcul the mask position with numpy : 0.0026001930236816406 nb_pixel_total : 139320 time to create 1 rle with old method : 0.1634817123413086 length of segment : 730 time for calcul the mask position with numpy : 0.0008778572082519531 nb_pixel_total : 37711 time to create 1 rle with old method : 0.04444742202758789 length of segment : 232 time for calcul the mask position with numpy : 0.008435726165771484 nb_pixel_total : 274364 time to create 1 rle with new method : 0.015699148178100586 length of segment : 806 time for calcul the mask position with numpy : 0.0008180141448974609 nb_pixel_total : 37424 time to create 1 rle with old method : 0.06440234184265137 length of segment : 267 time for calcul the mask position with numpy : 0.0024230480194091797 nb_pixel_total : 88576 time to create 1 rle with old method : 0.10422945022583008 length of segment : 866 time for calcul the mask position with numpy : 0.0008218288421630859 nb_pixel_total : 32396 time to create 1 rle with old method : 0.03838706016540527 length of segment : 203 time for calcul the mask position with numpy : 0.003020763397216797 nb_pixel_total : 181403 time to create 1 rle with new method : 0.008486509323120117 length of segment : 323 time for calcul the mask position with numpy : 0.0011894702911376953 nb_pixel_total : 41238 time to create 1 rle with old method : 0.055219173431396484 length of segment : 241 time for calcul the mask position with numpy : 0.004254817962646484 nb_pixel_total : 143117 time to create 1 rle with old method : 0.16754579544067383 length of segment : 787 time for calcul the mask position with numpy : 0.014718055725097656 nb_pixel_total : 769295 time to create 1 rle with new method : 0.17400264739990234 length of segment : 1357 time for calcul the mask position with numpy : 0.001554727554321289 nb_pixel_total : 87146 time to create 1 rle with old method : 0.10009646415710449 length of segment : 409 time for calcul the mask position with numpy : 0.0006582736968994141 nb_pixel_total : 32744 time to create 1 rle with old method : 0.03830862045288086 length of segment : 210 time for calcul the mask position with numpy : 0.002413034439086914 nb_pixel_total : 98279 time to create 1 rle with old method : 0.11425542831420898 length of segment : 592 time for calcul the mask position with numpy : 0.0023941993713378906 nb_pixel_total : 107245 time to create 1 rle with old method : 0.1259007453918457 length of segment : 365 time for calcul the mask position with numpy : 0.0015134811401367188 nb_pixel_total : 72572 time to create 1 rle with old method : 0.08479595184326172 length of segment : 292 time for calcul the mask position with numpy : 0.0018396377563476562 nb_pixel_total : 60784 time to create 1 rle with old method : 0.08403897285461426 length of segment : 251 time for calcul the mask position with numpy : 0.0062792301177978516 nb_pixel_total : 289347 time to create 1 rle with new method : 0.018714189529418945 length of segment : 729 time for calcul the mask position with numpy : 0.0006561279296875 nb_pixel_total : 22812 time to create 1 rle with old method : 0.029287338256835938 length of segment : 113 time for calcul the mask position with numpy : 0.0015614032745361328 nb_pixel_total : 55797 time to create 1 rle with old method : 0.07904791831970215 length of segment : 298 time for calcul the mask position with numpy : 0.0014879703521728516 nb_pixel_total : 58567 time to create 1 rle with old method : 0.06923460960388184 length of segment : 370 time for calcul the mask position with numpy : 0.006805896759033203 nb_pixel_total : 229280 time to create 1 rle with new method : 0.018636703491210938 length of segment : 755 time for calcul the mask position with numpy : 0.0007550716400146484 nb_pixel_total : 30173 time to create 1 rle with old method : 0.03577756881713867 length of segment : 307 time for calcul the mask position with numpy : 0.0009641647338867188 nb_pixel_total : 40408 time to create 1 rle with old method : 0.047882080078125 length of segment : 255 time for calcul the mask position with numpy : 0.001878976821899414 nb_pixel_total : 64853 time to create 1 rle with old method : 0.10923528671264648 length of segment : 350 time for calcul the mask position with numpy : 0.0029103755950927734 nb_pixel_total : 129335 time to create 1 rle with old method : 0.15080642700195312 length of segment : 443 time for calcul the mask position with numpy : 0.0011589527130126953 nb_pixel_total : 33525 time to create 1 rle with old method : 0.040902137756347656 length of segment : 226 time for calcul the mask position with numpy : 0.0019664764404296875 nb_pixel_total : 65905 time to create 1 rle with old method : 0.07885861396789551 length of segment : 415 time for calcul the mask position with numpy : 0.003454446792602539 nb_pixel_total : 134687 time to create 1 rle with old method : 0.1567535400390625 length of segment : 607 time for calcul the mask position with numpy : 0.0013363361358642578 nb_pixel_total : 35721 time to create 1 rle with old method : 0.04189610481262207 length of segment : 185 time for calcul the mask position with numpy : 0.0006110668182373047 nb_pixel_total : 30258 time to create 1 rle with old method : 0.0359501838684082 length of segment : 177 time for calcul the mask position with numpy : 0.005061626434326172 nb_pixel_total : 198494 time to create 1 rle with new method : 0.016334056854248047 length of segment : 622 time for calcul the mask position with numpy : 0.0027234554290771484 nb_pixel_total : 116015 time to create 1 rle with old method : 0.13208460807800293 length of segment : 524 time for calcul the mask position with numpy : 0.0048370361328125 nb_pixel_total : 209821 time to create 1 rle with new method : 0.012548446655273438 length of segment : 561 time for calcul the mask position with numpy : 0.0007152557373046875 nb_pixel_total : 23110 time to create 1 rle with old method : 0.031107425689697266 length of segment : 215 time for calcul the mask position with numpy : 0.003260374069213867 nb_pixel_total : 51670 time to create 1 rle with old method : 0.06471967697143555 length of segment : 364 time for calcul the mask position with numpy : 0.0037262439727783203 nb_pixel_total : 65284 time to create 1 rle with old method : 0.07812905311584473 length of segment : 418 time for calcul the mask position with numpy : 0.0019152164459228516 nb_pixel_total : 80252 time to create 1 rle with old method : 0.09356212615966797 length of segment : 342 time for calcul the mask position with numpy : 0.0021009445190429688 nb_pixel_total : 84983 time to create 1 rle with old method : 0.10043072700500488 length of segment : 480 time for calcul the mask position with numpy : 0.002089977264404297 nb_pixel_total : 83300 time to create 1 rle with old method : 0.09736847877502441 length of segment : 498 time for calcul the mask position with numpy : 0.0009396076202392578 nb_pixel_total : 27508 time to create 1 rle with old method : 0.032915353775024414 length of segment : 200 time for calcul the mask position with numpy : 0.0008044242858886719 nb_pixel_total : 25116 time to create 1 rle with old method : 0.029552221298217773 length of segment : 278 time for calcul the mask position with numpy : 0.020018339157104492 nb_pixel_total : 809937 time to create 1 rle with new method : 0.06563568115234375 length of segment : 1171 time for calcul the mask position with numpy : 0.0037572383880615234 nb_pixel_total : 177490 time to create 1 rle with new method : 0.010774374008178711 length of segment : 478 time for calcul the mask position with numpy : 0.006991386413574219 nb_pixel_total : 343966 time to create 1 rle with new method : 0.026739835739135742 length of segment : 653 time for calcul the mask position with numpy : 0.001323699951171875 nb_pixel_total : 51256 time to create 1 rle with old method : 0.06193089485168457 length of segment : 233 time for calcul the mask position with numpy : 0.00466156005859375 nb_pixel_total : 183265 time to create 1 rle with new method : 0.01458883285522461 length of segment : 595 time for calcul the mask position with numpy : 0.002053976058959961 nb_pixel_total : 95884 time to create 1 rle with old method : 0.12073254585266113 length of segment : 436 time for calcul the mask position with numpy : 0.003961801528930664 nb_pixel_total : 147258 time to create 1 rle with old method : 0.22242307662963867 length of segment : 436 time for calcul the mask position with numpy : 0.0027875900268554688 nb_pixel_total : 79905 time to create 1 rle with old method : 0.09367156028747559 length of segment : 298 time for calcul the mask position with numpy : 0.00272369384765625 nb_pixel_total : 105641 time to create 1 rle with old method : 0.12366056442260742 length of segment : 641 time for calcul the mask position with numpy : 0.030913114547729492 nb_pixel_total : 1009045 time to create 1 rle with new method : 0.23785161972045898 length of segment : 1832 time for calcul the mask position with numpy : 0.001443624496459961 nb_pixel_total : 66313 time to create 1 rle with old method : 0.07790446281433105 length of segment : 409 time for calcul the mask position with numpy : 0.002483844757080078 nb_pixel_total : 104751 time to create 1 rle with old method : 0.11919736862182617 length of segment : 470 time for calcul the mask position with numpy : 0.003997325897216797 nb_pixel_total : 171121 time to create 1 rle with new method : 0.01204061508178711 length of segment : 699 time for calcul the mask position with numpy : 0.0017452239990234375 nb_pixel_total : 77214 time to create 1 rle with old method : 0.08962154388427734 length of segment : 391 time for calcul the mask position with numpy : 0.006884098052978516 nb_pixel_total : 195347 time to create 1 rle with new method : 0.030236244201660156 length of segment : 1024 time for calcul the mask position with numpy : 0.0014688968658447266 nb_pixel_total : 46796 time to create 1 rle with old method : 0.05585956573486328 length of segment : 193 time for calcul the mask position with numpy : 0.0034956932067871094 nb_pixel_total : 184445 time to create 1 rle with new method : 0.009666919708251953 length of segment : 451 time for calcul the mask position with numpy : 0.00500798225402832 nb_pixel_total : 228986 time to create 1 rle with new method : 0.01635599136352539 length of segment : 666 time for calcul the mask position with numpy : 0.01056361198425293 nb_pixel_total : 376305 time to create 1 rle with new method : 0.03797030448913574 length of segment : 746 time for calcul the mask position with numpy : 0.0046405792236328125 nb_pixel_total : 96664 time to create 1 rle with old method : 0.11422276496887207 length of segment : 519 time for calcul the mask position with numpy : 0.002748250961303711 nb_pixel_total : 122695 time to create 1 rle with old method : 0.14508271217346191 length of segment : 386 time for calcul the mask position with numpy : 0.0005195140838623047 nb_pixel_total : 14319 time to create 1 rle with old method : 0.017719745635986328 length of segment : 172 time for calcul the mask position with numpy : 0.0002741813659667969 nb_pixel_total : 9632 time to create 1 rle with old method : 0.01164555549621582 length of segment : 135 time for calcul the mask position with numpy : 0.004849433898925781 nb_pixel_total : 192653 time to create 1 rle with new method : 0.019306182861328125 length of segment : 605 time for calcul the mask position with numpy : 0.0018110275268554688 nb_pixel_total : 70887 time to create 1 rle with old method : 0.0820157527923584 length of segment : 500 time for calcul the mask position with numpy : 0.0026979446411132812 nb_pixel_total : 121487 time to create 1 rle with old method : 0.141951322555542 length of segment : 555 time for calcul the mask position with numpy : 0.005383729934692383 nb_pixel_total : 192633 time to create 1 rle with new method : 0.016770601272583008 length of segment : 601 time for calcul the mask position with numpy : 0.003155946731567383 nb_pixel_total : 176089 time to create 1 rle with new method : 0.008980035781860352 length of segment : 451 time for calcul the mask position with numpy : 0.005521297454833984 nb_pixel_total : 255513 time to create 1 rle with new method : 0.016356706619262695 length of segment : 634 time for calcul the mask position with numpy : 0.001827239990234375 nb_pixel_total : 66704 time to create 1 rle with old method : 0.07720780372619629 length of segment : 375 time for calcul the mask position with numpy : 0.0012972354888916016 nb_pixel_total : 36632 time to create 1 rle with old method : 0.053098440170288086 length of segment : 212 time for calcul the mask position with numpy : 0.005835771560668945 nb_pixel_total : 151052 time to create 1 rle with new method : 0.012307405471801758 length of segment : 329 time for calcul the mask position with numpy : 0.0038793087005615234 nb_pixel_total : 98616 time to create 1 rle with old method : 0.13653206825256348 length of segment : 370 time for calcul the mask position with numpy : 0.0018796920776367188 nb_pixel_total : 84900 time to create 1 rle with old method : 0.10073995590209961 length of segment : 319 time for calcul the mask position with numpy : 0.0009186267852783203 nb_pixel_total : 39024 time to create 1 rle with old method : 0.046709537506103516 length of segment : 225 time for calcul the mask position with numpy : 0.006467103958129883 nb_pixel_total : 281606 time to create 1 rle with new method : 0.019523143768310547 length of segment : 563 time for calcul the mask position with numpy : 0.0014204978942871094 nb_pixel_total : 50348 time to create 1 rle with old method : 0.059923410415649414 length of segment : 218 time for calcul the mask position with numpy : 0.019797325134277344 nb_pixel_total : 486498 time to create 1 rle with new method : 0.16814756393432617 length of segment : 1608 time for calcul the mask position with numpy : 0.017254352569580078 nb_pixel_total : 796975 time to create 1 rle with new method : 0.3410954475402832 length of segment : 1081 time for calcul the mask position with numpy : 0.005034446716308594 nb_pixel_total : 190558 time to create 1 rle with new method : 0.015997648239135742 length of segment : 699 time for calcul the mask position with numpy : 0.0036649703979492188 nb_pixel_total : 142909 time to create 1 rle with old method : 0.17998194694519043 length of segment : 494 time for calcul the mask position with numpy : 0.031131744384765625 nb_pixel_total : 712822 time to create 1 rle with new method : 0.515674352645874 length of segment : 2833 time for calcul the mask position with numpy : 0.0025479793548583984 nb_pixel_total : 106594 time to create 1 rle with old method : 0.12488222122192383 length of segment : 509 time for calcul the mask position with numpy : 0.0016925334930419922 nb_pixel_total : 75484 time to create 1 rle with old method : 0.11229777336120605 length of segment : 262 time for calcul the mask position with numpy : 0.00528407096862793 nb_pixel_total : 163453 time to create 1 rle with new method : 0.018680572509765625 length of segment : 657 time for calcul the mask position with numpy : 0.0011184215545654297 nb_pixel_total : 49297 time to create 1 rle with old method : 0.05858039855957031 length of segment : 195 time for calcul the mask position with numpy : 0.004587411880493164 nb_pixel_total : 206436 time to create 1 rle with new method : 0.013076543807983398 length of segment : 649 time for calcul the mask position with numpy : 0.007746219635009766 nb_pixel_total : 350519 time to create 1 rle with new method : 0.0198824405670166 length of segment : 568 time for calcul the mask position with numpy : 0.006367921829223633 nb_pixel_total : 261049 time to create 1 rle with new method : 0.02000117301940918 length of segment : 551 time for calcul the mask position with numpy : 0.002319812774658203 nb_pixel_total : 44336 time to create 1 rle with old method : 0.06190848350524902 length of segment : 206 time for calcul the mask position with numpy : 0.0036699771881103516 nb_pixel_total : 96762 time to create 1 rle with old method : 0.11340951919555664 length of segment : 436 time for calcul the mask position with numpy : 0.0071637630462646484 nb_pixel_total : 236204 time to create 1 rle with new method : 0.019620656967163086 length of segment : 656 time for calcul the mask position with numpy : 0.0007982254028320312 nb_pixel_total : 19826 time to create 1 rle with old method : 0.024569034576416016 length of segment : 147 time for calcul the mask position with numpy : 0.0030384063720703125 nb_pixel_total : 73648 time to create 1 rle with old method : 0.08940458297729492 length of segment : 442 time for calcul the mask position with numpy : 0.01082468032836914 nb_pixel_total : 506620 time to create 1 rle with new method : 0.025240659713745117 length of segment : 1289 time for calcul the mask position with numpy : 0.001253366470336914 nb_pixel_total : 41641 time to create 1 rle with old method : 0.04728055000305176 length of segment : 260 time for calcul the mask position with numpy : 0.0006806850433349609 nb_pixel_total : 27485 time to create 1 rle with old method : 0.031844377517700195 length of segment : 209 time for calcul the mask position with numpy : 0.0006310939788818359 nb_pixel_total : 31442 time to create 1 rle with old method : 0.03601408004760742 length of segment : 195 time for calcul the mask position with numpy : 0.007923364639282227 nb_pixel_total : 213310 time to create 1 rle with new method : 0.015127420425415039 length of segment : 623 time for calcul the mask position with numpy : 0.001468658447265625 nb_pixel_total : 27453 time to create 1 rle with old method : 0.032782554626464844 length of segment : 162 time for calcul the mask position with numpy : 0.004511833190917969 nb_pixel_total : 218746 time to create 1 rle with new method : 0.013038158416748047 length of segment : 458 time for calcul the mask position with numpy : 0.004561901092529297 nb_pixel_total : 184078 time to create 1 rle with new method : 0.01204061508178711 length of segment : 500 time for calcul the mask position with numpy : 0.006011486053466797 nb_pixel_total : 272334 time to create 1 rle with new method : 0.01789236068725586 length of segment : 717 time for calcul the mask position with numpy : 0.005549430847167969 nb_pixel_total : 211634 time to create 1 rle with new method : 0.015917301177978516 length of segment : 704 time for calcul the mask position with numpy : 0.008662700653076172 nb_pixel_total : 397618 time to create 1 rle with new method : 0.02228403091430664 length of segment : 979 time for calcul the mask position with numpy : 0.004732370376586914 nb_pixel_total : 216491 time to create 1 rle with new method : 0.012630224227905273 length of segment : 649 time for calcul the mask position with numpy : 0.0032999515533447266 nb_pixel_total : 144280 time to create 1 rle with old method : 0.16371679306030273 length of segment : 424 time for calcul the mask position with numpy : 0.0031354427337646484 nb_pixel_total : 111805 time to create 1 rle with old method : 0.13048863410949707 length of segment : 466 time for calcul the mask position with numpy : 0.005902290344238281 nb_pixel_total : 204335 time to create 1 rle with new method : 0.016597986221313477 length of segment : 442 time for calcul the mask position with numpy : 0.001560211181640625 nb_pixel_total : 65165 time to create 1 rle with old method : 0.07666254043579102 length of segment : 246 time for calcul the mask position with numpy : 0.0012984275817871094 nb_pixel_total : 50919 time to create 1 rle with old method : 0.06560587882995605 length of segment : 220 time for calcul the mask position with numpy : 0.0015239715576171875 nb_pixel_total : 63309 time to create 1 rle with old method : 0.0772256851196289 length of segment : 400 time for calcul the mask position with numpy : 0.015059947967529297 nb_pixel_total : 721352 time to create 1 rle with new method : 0.04446887969970703 length of segment : 872 time for calcul the mask position with numpy : 0.0018415451049804688 nb_pixel_total : 37648 time to create 1 rle with old method : 0.0510706901550293 length of segment : 207 time for calcul the mask position with numpy : 0.0016739368438720703 nb_pixel_total : 32287 time to create 1 rle with old method : 0.044623613357543945 length of segment : 234 time for calcul the mask position with numpy : 0.011525392532348633 nb_pixel_total : 94984 time to create 1 rle with old method : 0.11257576942443848 length of segment : 611 time for calcul the mask position with numpy : 0.00045943260192871094 nb_pixel_total : 13089 time to create 1 rle with old method : 0.015903234481811523 length of segment : 94 time for calcul the mask position with numpy : 0.0036156177520751953 nb_pixel_total : 140086 time to create 1 rle with old method : 0.18836212158203125 length of segment : 408 time for calcul the mask position with numpy : 0.0020453929901123047 nb_pixel_total : 88705 time to create 1 rle with old method : 0.1052243709564209 length of segment : 282 time for calcul the mask position with numpy : 0.001775503158569336 nb_pixel_total : 47126 time to create 1 rle with old method : 0.0564723014831543 length of segment : 219 time for calcul the mask position with numpy : 0.0010433197021484375 nb_pixel_total : 23500 time to create 1 rle with old method : 0.028547048568725586 length of segment : 226 time for calcul the mask position with numpy : 0.0019071102142333984 nb_pixel_total : 78596 time to create 1 rle with old method : 0.09255194664001465 length of segment : 322 time for calcul the mask position with numpy : 0.002262115478515625 nb_pixel_total : 84975 time to create 1 rle with old method : 0.13768267631530762 length of segment : 348 time for calcul the mask position with numpy : 0.012145519256591797 nb_pixel_total : 354436 time to create 1 rle with new method : 0.03936767578125 length of segment : 640 time for calcul the mask position with numpy : 0.0013127326965332031 nb_pixel_total : 40839 time to create 1 rle with old method : 0.04792976379394531 length of segment : 434 time for calcul the mask position with numpy : 0.0005035400390625 nb_pixel_total : 17756 time to create 1 rle with old method : 0.021146297454833984 length of segment : 228 time for calcul the mask position with numpy : 0.0022814273834228516 nb_pixel_total : 48586 time to create 1 rle with old method : 0.05778813362121582 length of segment : 433 time for calcul the mask position with numpy : 0.006433248519897461 nb_pixel_total : 148740 time to create 1 rle with old method : 0.18062782287597656 length of segment : 690 time for calcul the mask position with numpy : 0.0014386177062988281 nb_pixel_total : 62481 time to create 1 rle with old method : 0.07622838020324707 length of segment : 273 time for calcul the mask position with numpy : 0.0010471343994140625 nb_pixel_total : 39162 time to create 1 rle with old method : 0.04833650588989258 length of segment : 295 time for calcul the mask position with numpy : 0.0029129981994628906 nb_pixel_total : 93286 time to create 1 rle with old method : 0.10863518714904785 length of segment : 518 time for calcul the mask position with numpy : 0.001550912857055664 nb_pixel_total : 53911 time to create 1 rle with old method : 0.0639495849609375 length of segment : 523 time for calcul the mask position with numpy : 0.004010438919067383 nb_pixel_total : 42413 time to create 1 rle with old method : 0.04980063438415527 length of segment : 954 time for calcul the mask position with numpy : 0.021578311920166016 nb_pixel_total : 270000 time to create 1 rle with new method : 0.3334650993347168 length of segment : 1102 time for calcul the mask position with numpy : 0.001562356948852539 nb_pixel_total : 27496 time to create 1 rle with old method : 0.03323769569396973 length of segment : 499 time for calcul the mask position with numpy : 0.0023674964904785156 nb_pixel_total : 97217 time to create 1 rle with old method : 0.11666131019592285 length of segment : 341 time for calcul the mask position with numpy : 0.0026941299438476562 nb_pixel_total : 112720 time to create 1 rle with old method : 0.13747143745422363 length of segment : 414 time for calcul the mask position with numpy : 0.0006127357482910156 nb_pixel_total : 17452 time to create 1 rle with old method : 0.02215433120727539 length of segment : 134 time for calcul the mask position with numpy : 0.0062029361724853516 nb_pixel_total : 358794 time to create 1 rle with new method : 0.015038251876831055 length of segment : 664 time for calcul the mask position with numpy : 0.0008108615875244141 nb_pixel_total : 11411 time to create 1 rle with old method : 0.014055013656616211 length of segment : 155 time for calcul the mask position with numpy : 0.0026421546936035156 nb_pixel_total : 104396 time to create 1 rle with old method : 0.12370491027832031 length of segment : 972 time for calcul the mask position with numpy : 0.0024259090423583984 nb_pixel_total : 40043 time to create 1 rle with old method : 0.0493617057800293 length of segment : 271 time for calcul the mask position with numpy : 0.0076253414154052734 nb_pixel_total : 99567 time to create 1 rle with old method : 0.11922740936279297 length of segment : 373 time for calcul the mask position with numpy : 0.0075228214263916016 nb_pixel_total : 276947 time to create 1 rle with new method : 0.01883411407470703 length of segment : 461 time for calcul the mask position with numpy : 0.0007419586181640625 nb_pixel_total : 35329 time to create 1 rle with old method : 0.040451765060424805 length of segment : 329 time for calcul the mask position with numpy : 0.001003265380859375 nb_pixel_total : 28171 time to create 1 rle with old method : 0.03807473182678223 length of segment : 236 time for calcul the mask position with numpy : 0.007079124450683594 nb_pixel_total : 401417 time to create 1 rle with new method : 0.018759965896606445 length of segment : 656 time for calcul the mask position with numpy : 0.004762411117553711 nb_pixel_total : 39753 time to create 1 rle with old method : 0.04694724082946777 length of segment : 770 time for calcul the mask position with numpy : 0.011008501052856445 nb_pixel_total : 212205 time to create 1 rle with new method : 0.011201143264770508 length of segment : 407 time for calcul the mask position with numpy : 0.013407707214355469 nb_pixel_total : 282428 time to create 1 rle with new method : 0.018747329711914062 length of segment : 671 time for calcul the mask position with numpy : 0.0071527957916259766 nb_pixel_total : 199218 time to create 1 rle with new method : 0.009786844253540039 length of segment : 376 time for calcul the mask position with numpy : 0.0021278858184814453 nb_pixel_total : 92976 time to create 1 rle with old method : 0.10930752754211426 length of segment : 483 time for calcul the mask position with numpy : 0.0019724369049072266 nb_pixel_total : 51279 time to create 1 rle with old method : 0.06844162940979004 length of segment : 256 time for calcul the mask position with numpy : 0.0072634220123291016 nb_pixel_total : 275648 time to create 1 rle with new method : 0.021686792373657227 length of segment : 466 time for calcul the mask position with numpy : 0.012618780136108398 nb_pixel_total : 391880 time to create 1 rle with new method : 0.07422375679016113 length of segment : 629 time for calcul the mask position with numpy : 0.026246309280395508 nb_pixel_total : 783010 time to create 1 rle with new method : 0.3179931640625 length of segment : 1057 time for calcul the mask position with numpy : 0.004292011260986328 nb_pixel_total : 192468 time to create 1 rle with new method : 0.013904333114624023 length of segment : 851 time for calcul the mask position with numpy : 0.0019538402557373047 nb_pixel_total : 75665 time to create 1 rle with old method : 0.0872032642364502 length of segment : 300 time for calcul the mask position with numpy : 0.0012450218200683594 nb_pixel_total : 70982 time to create 1 rle with old method : 0.08171916007995605 length of segment : 313 time for calcul the mask position with numpy : 0.0039653778076171875 nb_pixel_total : 189163 time to create 1 rle with new method : 0.014186620712280273 length of segment : 857 time for calcul the mask position with numpy : 0.002051115036010742 nb_pixel_total : 96906 time to create 1 rle with old method : 0.11088371276855469 length of segment : 456 time for calcul the mask position with numpy : 0.0013544559478759766 nb_pixel_total : 49471 time to create 1 rle with old method : 0.05880236625671387 length of segment : 321 time for calcul the mask position with numpy : 0.0031495094299316406 nb_pixel_total : 120584 time to create 1 rle with old method : 0.17362689971923828 length of segment : 690 time for calcul the mask position with numpy : 0.36684131622314453 nb_pixel_total : 5739389 time to create 1 rle with new method : 1.082127571105957 length of segment : 6060 time for calcul the mask position with numpy : 0.017644166946411133 nb_pixel_total : 716326 time to create 1 rle with new method : 0.5353848934173584 length of segment : 1890 time for calcul the mask position with numpy : 0.009262800216674805 nb_pixel_total : 738611 time to create 1 rle with new method : 0.014950752258300781 length of segment : 546 time spent for convertir_results : 43.77294993400574 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 260 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 119.84049582481384 time spend to save output : 0.1747596263885498 total time spend for step 2 : 120.01525545120239 step3:blur_detection Thu Aug 28 14:44:38 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (4160, 3120) 988.7439875814889 (4160, 3120) 1496.3255671577597 (4160, 3120) 1760.1775979454248 (4160, 3120) 214.98366129230544 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.012983083724975586 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.011971712112426758 save missing photos in datou_result : time spend for datou_step_exec : 0.745887279510498 time spend to save output : 0.02960991859436035 total time spend for step 3 : 0.7754971981048584 step4:brightness Thu Aug 28 14:44:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1756384887_3814560_1379845121_b659734ca1459e98c4bb2aaea712646c.jpg treat image : temp/1756384887_3814560_1379845112_573b36625caf5640efc499e99b5f1ee0.jpg treat image : temp/1756384887_3814560_1379844924_48f3e988c89dd303f67e7cd2249bbc5b.jpg treat image : temp/1756384887_3814560_1379844911_e95b72b9b3c3c91c444c45b320456e8c.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.0124359130859375 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.011167049407958984 save missing photos in datou_result : time spend for datou_step_exec : 6.0579729080200195 time spend to save output : 0.028786897659301758 total time spend for step 4 : 6.086759805679321 step5:rle_unique_nms_with_priority Thu Aug 28 14:44:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 260 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 76 nb_hashtags : 9 time to prepare the origin masks : 74.88669943809509 time for calcul the mask position with numpy : 0.24985218048095703 nb_pixel_total : 3457541 time to create 1 rle with new method : 0.6345603466033936 time for calcul the mask position with numpy : 0.0760498046875 nb_pixel_total : 279641 time to create 1 rle with new method : 1.1803250312805176 time for calcul the mask position with numpy : 0.05495142936706543 nb_pixel_total : 1223 time to create 1 rle with old method : 0.0017135143280029297 time for calcul the mask position with numpy : 0.05664181709289551 nb_pixel_total : 92286 time to create 1 rle with old method : 0.11220765113830566 time for calcul the mask position with numpy : 0.05726265907287598 nb_pixel_total : 64797 time to create 1 rle with old method : 0.0759284496307373 time for calcul the mask position with numpy : 0.10763716697692871 nb_pixel_total : 1654872 time to create 1 rle with new method : 1.0384552478790283 time for calcul the mask position with numpy : 0.05816197395324707 nb_pixel_total : 232179 time to create 1 rle with new method : 0.6830060482025146 time for calcul the mask position with numpy : 0.05710601806640625 nb_pixel_total : 314283 time to create 1 rle with new method : 0.6875240802764893 time for calcul the mask position with numpy : 0.060286521911621094 nb_pixel_total : 654 time to create 1 rle with old method : 0.0011665821075439453 time for calcul the mask position with numpy : 0.05925750732421875 nb_pixel_total : 37791 time to create 1 rle with old method : 0.043883562088012695 time for calcul the mask position with numpy : 0.052953243255615234 nb_pixel_total : 3991 time to create 1 rle with old method : 0.004635810852050781 time for calcul the mask position with numpy : 0.054386138916015625 nb_pixel_total : 187424 time to create 1 rle with new method : 0.547513484954834 time for calcul the mask position with numpy : 0.05670595169067383 nb_pixel_total : 57221 time to create 1 rle with old method : 0.0704503059387207 time for calcul the mask position with numpy : 0.05554962158203125 nb_pixel_total : 1359 time to create 1 rle with old method : 0.0017778873443603516 time for calcul the mask position with numpy : 0.06066131591796875 nb_pixel_total : 25336 time to create 1 rle with old method : 0.03477358818054199 time for calcul the mask position with numpy : 0.057335853576660156 nb_pixel_total : 297210 time to create 1 rle with new method : 0.7196331024169922 time for calcul the mask position with numpy : 0.05600571632385254 nb_pixel_total : 139171 time to create 1 rle with old method : 0.1835017204284668 time for calcul the mask position with numpy : 0.06459856033325195 nb_pixel_total : 14474 time to create 1 rle with old method : 0.017940044403076172 time for calcul the mask position with numpy : 0.055486202239990234 nb_pixel_total : 80145 time to create 1 rle with old method : 0.10033416748046875 time for calcul the mask position with numpy : 0.06018829345703125 nb_pixel_total : 321468 time to create 1 rle with new method : 0.7482078075408936 time for calcul the mask position with numpy : 0.05520796775817871 nb_pixel_total : 156528 time to create 1 rle with new method : 0.6297369003295898 time for calcul the mask position with numpy : 0.05524396896362305 nb_pixel_total : 68935 time to create 1 rle with old method : 0.0809776782989502 time for calcul the mask position with numpy : 0.05441117286682129 nb_pixel_total : 70140 time to create 1 rle with old method : 0.08297276496887207 time for calcul the mask position with numpy : 0.05475306510925293 nb_pixel_total : 69123 time to create 1 rle with old method : 0.08049488067626953 time for calcul the mask position with numpy : 0.054300785064697266 nb_pixel_total : 144492 time to create 1 rle with old method : 0.16770720481872559 time for calcul the mask position with numpy : 0.054848432540893555 nb_pixel_total : 18946 time to create 1 rle with old method : 0.021750211715698242 time for calcul the mask position with numpy : 0.05418729782104492 nb_pixel_total : 7668 time to create 1 rle with old method : 0.009398460388183594 time for calcul the mask position with numpy : 0.05451154708862305 nb_pixel_total : 44457 time to create 1 rle with old method : 0.06438493728637695 time for calcul the mask position with numpy : 0.061217308044433594 nb_pixel_total : 93179 time to create 1 rle with old method : 0.10629916191101074 time for calcul the mask position with numpy : 0.05645895004272461 nb_pixel_total : 109329 time to create 1 rle with old method : 0.12414336204528809 time for calcul the mask position with numpy : 0.055808067321777344 nb_pixel_total : 41102 time to create 1 rle with old method : 0.04791665077209473 time for calcul the mask position with numpy : 0.0553286075592041 nb_pixel_total : 166963 time to create 1 rle with new method : 0.5923519134521484 time for calcul the mask position with numpy : 0.05498790740966797 nb_pixel_total : 46618 time to create 1 rle with old method : 0.05361008644104004 time for calcul the mask position with numpy : 0.053665876388549805 nb_pixel_total : 75826 time to create 1 rle with old method : 0.08552432060241699 time for calcul the mask position with numpy : 0.057680368423461914 nb_pixel_total : 412350 time to create 1 rle with new method : 0.9660305976867676 time for calcul the mask position with numpy : 0.05553841590881348 nb_pixel_total : 47399 time to create 1 rle with old method : 0.05817103385925293 time for calcul the mask position with numpy : 0.0578615665435791 nb_pixel_total : 547581 time to create 1 rle with new method : 0.7805225849151611 time for calcul the mask position with numpy : 0.06103992462158203 nb_pixel_total : 39080 time to create 1 rle with old method : 0.04569697380065918 time for calcul the mask position with numpy : 0.057472944259643555 nb_pixel_total : 222745 time to create 1 rle with new method : 0.6200945377349854 time for calcul the mask position with numpy : 0.05338859558105469 nb_pixel_total : 45431 time to create 1 rle with old method : 0.05180001258850098 time for calcul the mask position with numpy : 0.05549907684326172 nb_pixel_total : 8942 time to create 1 rle with old method : 0.01891779899597168 time for calcul the mask position with numpy : 0.060868263244628906 nb_pixel_total : 258196 time to create 1 rle with new method : 0.8294715881347656 time for calcul the mask position with numpy : 0.05575728416442871 nb_pixel_total : 38476 time to create 1 rle with old method : 0.04479241371154785 time for calcul the mask position with numpy : 0.05649733543395996 nb_pixel_total : 83094 time to create 1 rle with old method : 0.0951395034790039 time for calcul the mask position with numpy : 0.05701446533203125 nb_pixel_total : 263881 time to create 1 rle with new method : 0.8070008754730225 time for calcul the mask position with numpy : 0.05425453186035156 nb_pixel_total : 60428 time to create 1 rle with old method : 0.07016992568969727 time for calcul the mask position with numpy : 0.05595135688781738 nb_pixel_total : 165743 time to create 1 rle with new method : 0.6560938358306885 time for calcul the mask position with numpy : 0.0551600456237793 nb_pixel_total : 113290 time to create 1 rle with old method : 0.1406233310699463 time for calcul the mask position with numpy : 0.05402660369873047 nb_pixel_total : 28236 time to create 1 rle with old method : 0.03324413299560547 time for calcul the mask position with numpy : 0.05442929267883301 nb_pixel_total : 10894 time to create 1 rle with old method : 0.012621879577636719 time for calcul the mask position with numpy : 0.054480552673339844 nb_pixel_total : 99399 time to create 1 rle with old method : 0.11895203590393066 time for calcul the mask position with numpy : 0.06286478042602539 nb_pixel_total : 695099 time to create 1 rle with new method : 0.9432480335235596 time for calcul the mask position with numpy : 0.05893397331237793 nb_pixel_total : 15376 time to create 1 rle with old method : 0.019346952438354492 time for calcul the mask position with numpy : 0.06163191795349121 nb_pixel_total : 33963 time to create 1 rle with old method : 0.039019107818603516 time for calcul the mask position with numpy : 0.055922508239746094 nb_pixel_total : 54549 time to create 1 rle with old method : 0.06248068809509277 time for calcul the mask position with numpy : 0.05530691146850586 nb_pixel_total : 180501 time to create 1 rle with new method : 1.6565775871276855 time for calcul the mask position with numpy : 0.057260990142822266 nb_pixel_total : 1991 time to create 1 rle with old method : 0.0026192665100097656 time for calcul the mask position with numpy : 0.058045148849487305 nb_pixel_total : 273069 time to create 1 rle with new method : 1.324540376663208 time for calcul the mask position with numpy : 0.054695844650268555 nb_pixel_total : 56735 time to create 1 rle with old method : 0.06511569023132324 time for calcul the mask position with numpy : 0.054886817932128906 nb_pixel_total : 429 time to create 1 rle with old method : 0.0012428760528564453 time for calcul the mask position with numpy : 0.06304693222045898 nb_pixel_total : 85032 time to create 1 rle with old method : 0.10050702095031738 time for calcul the mask position with numpy : 0.054232120513916016 nb_pixel_total : 2162 time to create 1 rle with old method : 0.0026264190673828125 time for calcul the mask position with numpy : 0.056119441986083984 nb_pixel_total : 240011 time to create 1 rle with new method : 1.150280237197876 time for calcul the mask position with numpy : 0.056118011474609375 nb_pixel_total : 96623 time to create 1 rle with old method : 0.1134026050567627 time for calcul the mask position with numpy : 0.054236650466918945 nb_pixel_total : 960 time to create 1 rle with old method : 0.0012423992156982422 time for calcul the mask position with numpy : 0.05523037910461426 nb_pixel_total : 7047 time to create 1 rle with old method : 0.008545637130737305 time for calcul the mask position with numpy : 0.054457902908325195 nb_pixel_total : 21893 time to create 1 rle with old method : 0.025153160095214844 time for calcul the mask position with numpy : 0.055635929107666016 nb_pixel_total : 78997 time to create 1 rle with old method : 0.08969879150390625 time for calcul the mask position with numpy : 0.05549168586730957 nb_pixel_total : 101456 time to create 1 rle with old method : 0.11176443099975586 time for calcul the mask position with numpy : 0.05385017395019531 nb_pixel_total : 6381 time to create 1 rle with old method : 0.007823944091796875 time for calcul the mask position with numpy : 0.05471205711364746 nb_pixel_total : 56308 time to create 1 rle with old method : 0.06256985664367676 time for calcul the mask position with numpy : 0.05343365669250488 nb_pixel_total : 37945 time to create 1 rle with old method : 0.04754328727722168 time for calcul the mask position with numpy : 0.061888694763183594 nb_pixel_total : 9028 time to create 1 rle with old method : 0.014151811599731445 time for calcul the mask position with numpy : 0.1366565227508545 nb_pixel_total : 117 time to create 1 rle with old method : 0.0004730224609375 time for calcul the mask position with numpy : 0.08739328384399414 nb_pixel_total : 985 time to create 1 rle with old method : 0.0013773441314697266 time for calcul the mask position with numpy : 0.055056095123291016 nb_pixel_total : 39068 time to create 1 rle with old method : 0.04584026336669922 time for calcul the mask position with numpy : 0.060202836990356445 nb_pixel_total : 61938 time to create 1 rle with old method : 0.07208037376403809 create new chi : 26.032500743865967 time to delete rle : 0.04219627380371094 batch 1 Loaded 77 chid ids of type : 4855 Number RLEs to save : 61292 TO DO : save crop sub photo not yet done ! save time : 3.454185724258423 nb_obj : 88 nb_hashtags : 11 time to prepare the origin masks : 63.667136430740356 time for calcul the mask position with numpy : 0.49538755416870117 nb_pixel_total : 4244686 time to create 1 rle with new method : 1.5669548511505127 time for calcul the mask position with numpy : 0.05549454689025879 nb_pixel_total : 100349 time to create 1 rle with old method : 0.12324643135070801 time for calcul the mask position with numpy : 0.05504035949707031 nb_pixel_total : 58326 time to create 1 rle with old method : 0.06988787651062012 time for calcul the mask position with numpy : 0.05508756637573242 nb_pixel_total : 106894 time to create 1 rle with old method : 0.12799453735351562 time for calcul the mask position with numpy : 0.053888797760009766 nb_pixel_total : 5895 time to create 1 rle with old method : 0.007429361343383789 time for calcul the mask position with numpy : 0.05519723892211914 nb_pixel_total : 683 time to create 1 rle with old method : 0.001255035400390625 time for calcul the mask position with numpy : 0.05769848823547363 nb_pixel_total : 81916 time to create 1 rle with old method : 0.10651206970214844 time for calcul the mask position with numpy : 0.054215431213378906 nb_pixel_total : 11808 time to create 1 rle with old method : 0.014120101928710938 time for calcul the mask position with numpy : 0.054682016372680664 nb_pixel_total : 80139 time to create 1 rle with old method : 0.0936579704284668 time for calcul the mask position with numpy : 0.054654836654663086 nb_pixel_total : 1668 time to create 1 rle with old method : 0.0023055076599121094 time for calcul the mask position with numpy : 0.055124759674072266 nb_pixel_total : 129091 time to create 1 rle with old method : 0.14635133743286133 time for calcul the mask position with numpy : 0.05748915672302246 nb_pixel_total : 63688 time to create 1 rle with old method : 0.09686589241027832 time for calcul the mask position with numpy : 0.05674552917480469 nb_pixel_total : 228918 time to create 1 rle with new method : 1.4397718906402588 time for calcul the mask position with numpy : 0.055239200592041016 nb_pixel_total : 165904 time to create 1 rle with new method : 1.2622787952423096 time for calcul the mask position with numpy : 0.05422663688659668 nb_pixel_total : 22323 time to create 1 rle with old method : 0.028536558151245117 time for calcul the mask position with numpy : 0.05490541458129883 nb_pixel_total : 115902 time to create 1 rle with old method : 0.13361430168151855 time for calcul the mask position with numpy : 0.06122255325317383 nb_pixel_total : 658549 time to create 1 rle with new method : 1.5749189853668213 time for calcul the mask position with numpy : 0.055307865142822266 nb_pixel_total : 149948 time to create 1 rle with old method : 0.17274761199951172 time for calcul the mask position with numpy : 0.053847312927246094 nb_pixel_total : 6717 time to create 1 rle with old method : 0.008479595184326172 time for calcul the mask position with numpy : 0.05438661575317383 nb_pixel_total : 129037 time to create 1 rle with old method : 0.14811420440673828 time for calcul the mask position with numpy : 0.05400252342224121 nb_pixel_total : 14231 time to create 1 rle with old method : 0.016370296478271484 time for calcul the mask position with numpy : 0.05379343032836914 nb_pixel_total : 25486 time to create 1 rle with old method : 0.029228687286376953 time for calcul the mask position with numpy : 0.055037498474121094 nb_pixel_total : 209587 time to create 1 rle with new method : 1.2573070526123047 time for calcul the mask position with numpy : 0.0544588565826416 nb_pixel_total : 93952 time to create 1 rle with old method : 0.11304974555969238 time for calcul the mask position with numpy : 0.06008148193359375 nb_pixel_total : 808893 time to create 1 rle with new method : 1.191368579864502 time for calcul the mask position with numpy : 0.052585601806640625 nb_pixel_total : 65249 time to create 1 rle with old method : 0.07432746887207031 time for calcul the mask position with numpy : 0.053252220153808594 nb_pixel_total : 157752 time to create 1 rle with new method : 0.78369140625 time for calcul the mask position with numpy : 0.053611040115356445 nb_pixel_total : 29877 time to create 1 rle with old method : 0.03491687774658203 time for calcul the mask position with numpy : 0.05185079574584961 nb_pixel_total : 175193 time to create 1 rle with new method : 1.0863239765167236 time for calcul the mask position with numpy : 0.05297064781188965 nb_pixel_total : 41134 time to create 1 rle with old method : 0.04896378517150879 time for calcul the mask position with numpy : 0.053960323333740234 nb_pixel_total : 72429 time to create 1 rle with old method : 0.08385801315307617 time for calcul the mask position with numpy : 0.05553007125854492 nb_pixel_total : 462000 time to create 1 rle with new method : 0.8163292407989502 time for calcul the mask position with numpy : 0.05402398109436035 nb_pixel_total : 36529 time to create 1 rle with old method : 0.04180622100830078 time for calcul the mask position with numpy : 0.05369138717651367 nb_pixel_total : 19019 time to create 1 rle with old method : 0.02166914939880371 time for calcul the mask position with numpy : 0.0539546012878418 nb_pixel_total : 33296 time to create 1 rle with old method : 0.038133859634399414 time for calcul the mask position with numpy : 0.05385947227478027 nb_pixel_total : 2818 time to create 1 rle with old method : 0.003657817840576172 time for calcul the mask position with numpy : 0.054203033447265625 nb_pixel_total : 46607 time to create 1 rle with old method : 0.053550004959106445 time for calcul the mask position with numpy : 0.054189205169677734 nb_pixel_total : 66187 time to create 1 rle with old method : 0.07553935050964355 time for calcul the mask position with numpy : 0.055118560791015625 nb_pixel_total : 235682 time to create 1 rle with new method : 1.1228973865509033 time for calcul the mask position with numpy : 0.05386972427368164 nb_pixel_total : 1383 time to create 1 rle with old method : 0.0017631053924560547 time for calcul the mask position with numpy : 0.05518364906311035 nb_pixel_total : 356457 time to create 1 rle with new method : 0.942479133605957 time for calcul the mask position with numpy : 0.05520033836364746 nb_pixel_total : 109779 time to create 1 rle with old method : 0.1254129409790039 time for calcul the mask position with numpy : 0.05423474311828613 nb_pixel_total : 105500 time to create 1 rle with old method : 0.12122559547424316 time for calcul the mask position with numpy : 0.054047346115112305 nb_pixel_total : 38929 time to create 1 rle with old method : 0.044606685638427734 time for calcul the mask position with numpy : 0.05461621284484863 nb_pixel_total : 196699 time to create 1 rle with new method : 2.1058499813079834 time for calcul the mask position with numpy : 0.05429363250732422 nb_pixel_total : 40320 time to create 1 rle with old method : 0.04673171043395996 time for calcul the mask position with numpy : 0.05411529541015625 nb_pixel_total : 83073 time to create 1 rle with old method : 0.09450483322143555 time for calcul the mask position with numpy : 0.053812265396118164 nb_pixel_total : 60008 time to create 1 rle with old method : 0.06996822357177734 time for calcul the mask position with numpy : 0.05593729019165039 nb_pixel_total : 77082 time to create 1 rle with old method : 0.09207940101623535 time for calcul the mask position with numpy : 0.053665876388549805 nb_pixel_total : 1781 time to create 1 rle with old method : 0.002547740936279297 time for calcul the mask position with numpy : 0.0544741153717041 nb_pixel_total : 79647 time to create 1 rle with old method : 0.09100818634033203 time for calcul the mask position with numpy : 0.05370473861694336 nb_pixel_total : 30160 time to create 1 rle with old method : 0.03529047966003418 time for calcul the mask position with numpy : 0.05377554893493652 nb_pixel_total : 35561 time to create 1 rle with old method : 0.041516780853271484 time for calcul the mask position with numpy : 0.05378150939941406 nb_pixel_total : 70763 time to create 1 rle with old method : 0.08182215690612793 time for calcul the mask position with numpy : 0.05377507209777832 nb_pixel_total : 30125 time to create 1 rle with old method : 0.0348200798034668 time for calcul the mask position with numpy : 0.05386757850646973 nb_pixel_total : 47681 time to create 1 rle with old method : 0.05577445030212402 time for calcul the mask position with numpy : 0.05403780937194824 nb_pixel_total : 51067 time to create 1 rle with old method : 0.05956101417541504 time for calcul the mask position with numpy : 0.05375504493713379 nb_pixel_total : 263 time to create 1 rle with old method : 0.0005507469177246094 time for calcul the mask position with numpy : 0.054552555084228516 nb_pixel_total : 2044 time to create 1 rle with old method : 0.0027005672454833984 time for calcul the mask position with numpy : 0.05411124229431152 nb_pixel_total : 22753 time to create 1 rle with old method : 0.02657294273376465 time for calcul the mask position with numpy : 0.05335497856140137 nb_pixel_total : 96149 time to create 1 rle with old method : 0.1114201545715332 time for calcul the mask position with numpy : 0.05403447151184082 nb_pixel_total : 55746 time to create 1 rle with old method : 0.06353235244750977 time for calcul the mask position with numpy : 0.05395007133483887 nb_pixel_total : 58625 time to create 1 rle with old method : 0.06752657890319824 time for calcul the mask position with numpy : 0.055005788803100586 nb_pixel_total : 288878 time to create 1 rle with new method : 1.0385699272155762 time for calcul the mask position with numpy : 0.05388283729553223 nb_pixel_total : 9574 time to create 1 rle with old method : 0.011102676391601562 time for calcul the mask position with numpy : 0.053650617599487305 nb_pixel_total : 20930 time to create 1 rle with old method : 0.02465677261352539 time for calcul the mask position with numpy : 0.054151058197021484 nb_pixel_total : 146854 time to create 1 rle with old method : 0.17261838912963867 time for calcul the mask position with numpy : 0.0540614128112793 nb_pixel_total : 145846 time to create 1 rle with old method : 0.1661663055419922 time for calcul the mask position with numpy : 0.05523252487182617 nb_pixel_total : 91692 time to create 1 rle with old method : 0.10251712799072266 time for calcul the mask position with numpy : 0.052707672119140625 nb_pixel_total : 134541 time to create 1 rle with old method : 0.15207934379577637 time for calcul the mask position with numpy : 0.0528569221496582 nb_pixel_total : 65582 time to create 1 rle with old method : 0.0739891529083252 time for calcul the mask position with numpy : 0.05289578437805176 nb_pixel_total : 20482 time to create 1 rle with old method : 0.02313542366027832 time for calcul the mask position with numpy : 0.05396318435668945 nb_pixel_total : 182116 time to create 1 rle with new method : 1.9116220474243164 time for calcul the mask position with numpy : 0.05724954605102539 nb_pixel_total : 8050 time to create 1 rle with old method : 0.014414548873901367 time for calcul the mask position with numpy : 0.057671546936035156 nb_pixel_total : 60452 time to create 1 rle with old method : 0.0692138671875 time for calcul the mask position with numpy : 0.05411553382873535 nb_pixel_total : 36030 time to create 1 rle with old method : 0.042031288146972656 time for calcul the mask position with numpy : 0.05413174629211426 nb_pixel_total : 86031 time to create 1 rle with old method : 0.09861445426940918 time for calcul the mask position with numpy : 0.05443906784057617 nb_pixel_total : 25100 time to create 1 rle with old method : 0.028929948806762695 time for calcul the mask position with numpy : 0.05444169044494629 nb_pixel_total : 51678 time to create 1 rle with old method : 0.06240200996398926 time for calcul the mask position with numpy : 0.056437015533447266 nb_pixel_total : 34734 time to create 1 rle with old method : 0.04058027267456055 time for calcul the mask position with numpy : 0.05603957176208496 nb_pixel_total : 104711 time to create 1 rle with old method : 0.11962389945983887 time for calcul the mask position with numpy : 0.05261874198913574 nb_pixel_total : 27340 time to create 1 rle with old method : 0.03129100799560547 time for calcul the mask position with numpy : 0.055966854095458984 nb_pixel_total : 343392 time to create 1 rle with new method : 1.089677333831787 time for calcul the mask position with numpy : 0.052289724349975586 nb_pixel_total : 107722 time to create 1 rle with old method : 0.11765432357788086 time for calcul the mask position with numpy : 0.05408167839050293 nb_pixel_total : 177100 time to create 1 rle with new method : 0.9621851444244385 time for calcul the mask position with numpy : 0.05740189552307129 nb_pixel_total : 184124 time to create 1 rle with new method : 1.45255708694458 time for calcul the mask position with numpy : 0.05346202850341797 nb_pixel_total : 274 time to create 1 rle with old method : 0.0005788803100585938 time for calcul the mask position with numpy : 0.05471062660217285 nb_pixel_total : 10170 time to create 1 rle with old method : 0.011510372161865234 time for calcul the mask position with numpy : 0.05273747444152832 nb_pixel_total : 5840 time to create 1 rle with old method : 0.006871700286865234 create new chi : 32.200066566467285 time to delete rle : 0.00652766227722168 batch 1 Loaded 89 chid ids of type : 4855 Number RLEs to save : 69647 TO DO : save crop sub photo not yet done ! save time : 4.215392589569092 nb_obj : 85 nb_hashtags : 9 time to prepare the origin masks : 71.17941236495972 time for calcul the mask position with numpy : 0.40991711616516113 nb_pixel_total : 3856484 time to create 1 rle with new method : 1.3632845878601074 time for calcul the mask position with numpy : 0.05231952667236328 nb_pixel_total : 44267 time to create 1 rle with old method : 0.048429012298583984 time for calcul the mask position with numpy : 0.06308174133300781 nb_pixel_total : 281357 time to create 1 rle with new method : 2.1552557945251465 time for calcul the mask position with numpy : 0.05295515060424805 nb_pixel_total : 32381 time to create 1 rle with old method : 0.036305904388427734 time for calcul the mask position with numpy : 0.0531008243560791 nb_pixel_total : 9894 time to create 1 rle with old method : 0.011634588241577148 time for calcul the mask position with numpy : 0.05455517768859863 nb_pixel_total : 210992 time to create 1 rle with new method : 1.3169634342193604 time for calcul the mask position with numpy : 0.05447053909301758 nb_pixel_total : 137646 time to create 1 rle with old method : 0.15675568580627441 time for calcul the mask position with numpy : 0.054898738861083984 nb_pixel_total : 78046 time to create 1 rle with old method : 0.08961248397827148 time for calcul the mask position with numpy : 0.05370903015136719 nb_pixel_total : 45554 time to create 1 rle with old method : 0.05233049392700195 time for calcul the mask position with numpy : 0.05353593826293945 nb_pixel_total : 12638 time to create 1 rle with old method : 0.014965057373046875 time for calcul the mask position with numpy : 0.05400991439819336 nb_pixel_total : 94373 time to create 1 rle with old method : 0.11334705352783203 time for calcul the mask position with numpy : 0.06058788299560547 nb_pixel_total : 196 time to create 1 rle with old method : 0.0005049705505371094 time for calcul the mask position with numpy : 0.054711103439331055 nb_pixel_total : 70130 time to create 1 rle with old method : 0.07960844039916992 time for calcul the mask position with numpy : 0.05355548858642578 nb_pixel_total : 27302 time to create 1 rle with old method : 0.03171253204345703 time for calcul the mask position with numpy : 0.05322003364562988 nb_pixel_total : 212667 time to create 1 rle with new method : 1.1987545490264893 time for calcul the mask position with numpy : 0.05455756187438965 nb_pixel_total : 46950 time to create 1 rle with old method : 0.05395770072937012 time for calcul the mask position with numpy : 0.05416989326477051 nb_pixel_total : 17467 time to create 1 rle with old method : 0.02017974853515625 time for calcul the mask position with numpy : 0.0583343505859375 nb_pixel_total : 4041 time to create 1 rle with old method : 0.0058650970458984375 time for calcul the mask position with numpy : 0.05434560775756836 nb_pixel_total : 262154 time to create 1 rle with new method : 1.1189422607421875 time for calcul the mask position with numpy : 0.05299234390258789 nb_pixel_total : 4902 time to create 1 rle with old method : 0.006012678146362305 time for calcul the mask position with numpy : 0.05637645721435547 nb_pixel_total : 397024 time to create 1 rle with new method : 1.0885791778564453 time for calcul the mask position with numpy : 0.054723262786865234 nb_pixel_total : 206231 time to create 1 rle with new method : 1.4279015064239502 time for calcul the mask position with numpy : 0.0537261962890625 nb_pixel_total : 11342 time to create 1 rle with old method : 0.01315450668334961 time for calcul the mask position with numpy : 0.053615570068359375 nb_pixel_total : 4130 time to create 1 rle with old method : 0.0053560733795166016 time for calcul the mask position with numpy : 0.05784726142883301 nb_pixel_total : 191271 time to create 1 rle with new method : 1.2451200485229492 time for calcul the mask position with numpy : 0.06179332733154297 nb_pixel_total : 713844 time to create 1 rle with new method : 1.0330119132995605 time for calcul the mask position with numpy : 0.05256962776184082 nb_pixel_total : 20672 time to create 1 rle with old method : 0.024900197982788086 time for calcul the mask position with numpy : 0.05286860466003418 nb_pixel_total : 7478 time to create 1 rle with old method : 0.0085906982421875 time for calcul the mask position with numpy : 0.053493499755859375 nb_pixel_total : 24725 time to create 1 rle with old method : 0.028426170349121094 time for calcul the mask position with numpy : 0.055901527404785156 nb_pixel_total : 5262 time to create 1 rle with old method : 0.009074211120605469 time for calcul the mask position with numpy : 0.059801340103149414 nb_pixel_total : 61779 time to create 1 rle with old method : 0.0702357292175293 time for calcul the mask position with numpy : 0.0552973747253418 nb_pixel_total : 358085 time to create 1 rle with new method : 1.0432958602905273 time for calcul the mask position with numpy : 0.05370640754699707 nb_pixel_total : 4672 time to create 1 rle with old method : 0.006118297576904297 time for calcul the mask position with numpy : 0.05343198776245117 nb_pixel_total : 111555 time to create 1 rle with old method : 0.1295621395111084 time for calcul the mask position with numpy : 0.05371904373168945 nb_pixel_total : 7546 time to create 1 rle with old method : 0.09150910377502441 time for calcul the mask position with numpy : 0.053131818771362305 nb_pixel_total : 19268 time to create 1 rle with old method : 0.02192544937133789 time for calcul the mask position with numpy : 0.05376100540161133 nb_pixel_total : 92864 time to create 1 rle with old method : 0.10966730117797852 time for calcul the mask position with numpy : 0.05340385437011719 nb_pixel_total : 84737 time to create 1 rle with old method : 0.0942988395690918 time for calcul the mask position with numpy : 0.054428815841674805 nb_pixel_total : 218998 time to create 1 rle with new method : 1.1245431900024414 time for calcul the mask position with numpy : 0.054633140563964844 nb_pixel_total : 147192 time to create 1 rle with old method : 0.1671428680419922 time for calcul the mask position with numpy : 0.0536496639251709 nb_pixel_total : 285 time to create 1 rle with old method : 0.0004074573516845703 time for calcul the mask position with numpy : 0.055191993713378906 nb_pixel_total : 139763 time to create 1 rle with old method : 0.1569375991821289 time for calcul the mask position with numpy : 0.05301165580749512 nb_pixel_total : 96520 time to create 1 rle with old method : 0.10864949226379395 time for calcul the mask position with numpy : 0.05396270751953125 nb_pixel_total : 73588 time to create 1 rle with old method : 0.08568120002746582 time for calcul the mask position with numpy : 0.06154799461364746 nb_pixel_total : 39779 time to create 1 rle with old method : 0.05218672752380371 time for calcul the mask position with numpy : 0.0569150447845459 nb_pixel_total : 353411 time to create 1 rle with new method : 1.18056321144104 time for calcul the mask position with numpy : 0.054367780685424805 nb_pixel_total : 17753 time to create 1 rle with old method : 0.020364761352539062 time for calcul the mask position with numpy : 0.05414414405822754 nb_pixel_total : 41533 time to create 1 rle with old method : 0.0465388298034668 time for calcul the mask position with numpy : 0.05390429496765137 nb_pixel_total : 27421 time to create 1 rle with old method : 0.030990123748779297 time for calcul the mask position with numpy : 0.05461859703063965 nb_pixel_total : 143660 time to create 1 rle with old method : 0.16901373863220215 time for calcul the mask position with numpy : 0.05452609062194824 nb_pixel_total : 74030 time to create 1 rle with old method : 0.09653878211975098 time for calcul the mask position with numpy : 0.06020236015319824 nb_pixel_total : 16711 time to create 1 rle with old method : 0.019158363342285156 time for calcul the mask position with numpy : 0.05300283432006836 nb_pixel_total : 50695 time to create 1 rle with old method : 0.05773353576660156 time for calcul the mask position with numpy : 0.05295610427856445 nb_pixel_total : 86260 time to create 1 rle with old method : 0.09531450271606445 time for calcul the mask position with numpy : 0.05193376541137695 nb_pixel_total : 15660 time to create 1 rle with old method : 0.017209529876708984 time for calcul the mask position with numpy : 0.051421403884887695 nb_pixel_total : 53549 time to create 1 rle with old method : 0.05871987342834473 time for calcul the mask position with numpy : 0.05543375015258789 nb_pixel_total : 51151 time to create 1 rle with old method : 0.05864548683166504 time for calcul the mask position with numpy : 0.05426168441772461 nb_pixel_total : 203834 time to create 1 rle with new method : 1.613034963607788 time for calcul the mask position with numpy : 0.06113433837890625 nb_pixel_total : 62381 time to create 1 rle with old method : 0.07085728645324707 time for calcul the mask position with numpy : 0.05487513542175293 nb_pixel_total : 77350 time to create 1 rle with old method : 0.08936047554016113 time for calcul the mask position with numpy : 0.054135799407958984 nb_pixel_total : 26038 time to create 1 rle with old method : 0.03023552894592285 time for calcul the mask position with numpy : 0.0542144775390625 nb_pixel_total : 38936 time to create 1 rle with old method : 0.04569840431213379 time for calcul the mask position with numpy : 0.05476188659667969 nb_pixel_total : 183725 time to create 1 rle with new method : 1.5246200561523438 time for calcul the mask position with numpy : 0.0543212890625 nb_pixel_total : 63171 time to create 1 rle with old method : 0.07214045524597168 time for calcul the mask position with numpy : 0.05208230018615723 nb_pixel_total : 5648 time to create 1 rle with old method : 0.006745100021362305 time for calcul the mask position with numpy : 0.05185437202453613 nb_pixel_total : 47937 time to create 1 rle with old method : 0.05501580238342285 time for calcul the mask position with numpy : 0.05790305137634277 nb_pixel_total : 506198 time to create 1 rle with new method : 2.0754849910736084 time for calcul the mask position with numpy : 0.05596446990966797 nb_pixel_total : 272371 time to create 1 rle with new method : 1.7067248821258545 time for calcul the mask position with numpy : 0.053971052169799805 nb_pixel_total : 23536 time to create 1 rle with old method : 0.027035951614379883 time for calcul the mask position with numpy : 0.05377697944641113 nb_pixel_total : 12886 time to create 1 rle with old method : 0.01490640640258789 time for calcul the mask position with numpy : 0.05447030067443848 nb_pixel_total : 90040 time to create 1 rle with old method : 0.10302972793579102 time for calcul the mask position with numpy : 0.05427122116088867 nb_pixel_total : 43736 time to create 1 rle with old method : 0.05084967613220215 time for calcul the mask position with numpy : 0.054075002670288086 nb_pixel_total : 35257 time to create 1 rle with old method : 0.04024696350097656 time for calcul the mask position with numpy : 0.05626678466796875 nb_pixel_total : 211533 time to create 1 rle with new method : 1.4269750118255615 time for calcul the mask position with numpy : 0.05394268035888672 nb_pixel_total : 37548 time to create 1 rle with old method : 0.042966365814208984 time for calcul the mask position with numpy : 0.05476021766662598 nb_pixel_total : 216112 time to create 1 rle with new method : 3.0676753520965576 time for calcul the mask position with numpy : 0.053832292556762695 nb_pixel_total : 32245 time to create 1 rle with old method : 0.03708624839782715 time for calcul the mask position with numpy : 0.060124874114990234 nb_pixel_total : 349962 time to create 1 rle with new method : 1.5670263767242432 time for calcul the mask position with numpy : 0.056344032287597656 nb_pixel_total : 401050 time to create 1 rle with new method : 1.3202650547027588 time for calcul the mask position with numpy : 0.0541684627532959 nb_pixel_total : 96982 time to create 1 rle with old method : 0.11264896392822266 time for calcul the mask position with numpy : 0.055097103118896484 nb_pixel_total : 218186 time to create 1 rle with new method : 1.6608648300170898 time for calcul the mask position with numpy : 0.06379127502441406 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0017571449279785156 time for calcul the mask position with numpy : 0.054549455642700195 nb_pixel_total : 27147 time to create 1 rle with old method : 0.031585693359375 time for calcul the mask position with numpy : 0.05608963966369629 nb_pixel_total : 246934 time to create 1 rle with new method : 2.49946665763855 time for calcul the mask position with numpy : 0.058669328689575195 nb_pixel_total : 16747 time to create 1 rle with old method : 0.024128198623657227 time for calcul the mask position with numpy : 0.056827545166015625 nb_pixel_total : 10812 time to create 1 rle with old method : 0.012398719787597656 create new chi : 43.25555372238159 time to delete rle : 0.006621360778808594 batch 1 Loaded 86 chid ids of type : 4855 Number RLEs to save : 66430 TO DO : save crop sub photo not yet done ! save time : 3.6205339431762695 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 19.667885541915894 time for calcul the mask position with numpy : 0.393848180770874 nb_pixel_total : 2718678 time to create 1 rle with new method : 1.0710053443908691 time for calcul the mask position with numpy : 0.06899881362915039 nb_pixel_total : 48696 time to create 1 rle with old method : 0.07402777671813965 time for calcul the mask position with numpy : 0.042029619216918945 nb_pixel_total : 60035 time to create 1 rle with old method : 0.07361888885498047 time for calcul the mask position with numpy : 0.04393291473388672 nb_pixel_total : 135673 time to create 1 rle with old method : 0.16651296615600586 time for calcul the mask position with numpy : 0.04043149948120117 nb_pixel_total : 11702 time to create 1 rle with old method : 0.015600204467773438 time for calcul the mask position with numpy : 0.04200577735900879 nb_pixel_total : 227912 time to create 1 rle with new method : 1.259321928024292 time for calcul the mask position with numpy : 0.0915684700012207 nb_pixel_total : 1873814 time to create 1 rle with new method : 1.606243371963501 time for calcul the mask position with numpy : 0.07450652122497559 nb_pixel_total : 959897 time to create 1 rle with new method : 1.3708672523498535 time for calcul the mask position with numpy : 0.07288646697998047 nb_pixel_total : 807342 time to create 1 rle with new method : 1.6645264625549316 time for calcul the mask position with numpy : 0.31263232231140137 nb_pixel_total : 6135451 time to create 1 rle with new method : 1.4566445350646973 create new chi : 10.239115715026855 time to delete rle : 0.0029828548431396484 batch 1 Loaded 10 chid ids of type : 4855 Number RLEs to save : 23627 TO DO : save crop sub photo not yet done ! save time : 1.306358814239502 map_output_result : {1379845121: (0.48125708768820497, 'Should be the crop_list due to order', 0.42504470357445073), 1379845112: (0.48125708768820497, 'Should be the crop_list due to order', 0.30512310828928907), 1379844924: (0.48125708768820497, 'Should be the crop_list due to order', 0.19486053888907998), 1379844911: (0.48125708768820497, 'Should be the crop_list due to order', 1.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 4 /1379845121.Didn't retrieve data . /1379845112.Didn't retrieve data . /1379844924.Didn't retrieve data . /1379844911.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.015341758728027344 save_final save missing photos in datou_result : time spend for datou_step_exec : 355.3573441505432 time spend to save output : 0.018059492111206055 total time spend for step 5 : 355.3754036426544 step6:crop_condition Thu Aug 28 14:50:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 4 ! batch 1 Loaded 262 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! About to upload 38 photos upload in portfolio : 3015255 init cache_photo without model_param we have 38 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385474_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 38 photos in the portfolio 3015255 time of upload the photos Elapsed time : 12.041334629058838 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 55 About to insert : list_path_to_insert length 55 new photo from crops ! About to upload 55 photos upload in portfolio : 3015255 init cache_photo without model_param we have 55 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385521_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 55 photos in the portfolio 3015255 time of upload the photos Elapsed time : 13.470341920852661 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3015255 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385539_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 12 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.2924306392669678 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3015255 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385549_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.736621141433716 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385555_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.7816617488861084 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 3015255 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385560_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 9 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.753413677215576 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385568_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.960890293121338 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! About to upload 23 photos upload in portfolio : 3015255 init cache_photo without model_param we have 23 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385599_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 23 photos in the portfolio 3015255 time of upload the photos Elapsed time : 7.427531003952026 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385617_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.9440078735351562 we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 3015255 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385623_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 10 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.069148302078247 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 31 About to insert : list_path_to_insert length 31 new photo from crops ! About to upload 31 photos upload in portfolio : 3015255 init cache_photo without model_param we have 31 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1756385689_3814560 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 31 photos in the portfolio 3015255 time of upload the photos Elapsed time : 9.076475143432617 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 193 /1379884244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884246Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884274Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884568Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1379884949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 583 time used for this insertion : 0.06378960609436035 save_final save missing photos in datou_result : time spend for datou_step_exec : 260.0745379924774 time spend to save output : 0.07045340538024902 total time spend for step 6 : 260.14499139785767 step7:ventilate_hashtags_in_portfolio Thu Aug 28 14:55:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26291505 get user id for portfolio 26291505 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291505 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291505 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291505 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291505 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/26294734,26294735,26294736,26294737,26294738,26294743,26294745,26294746,26294747,26294748,26294749,26294750,26294752,26294753,26294754,26294755,26294756,26294757,26294758,26294759?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26291505 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 1 /26291505. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.016234874725341797 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.558348894119263 time spend to save output : 0.019663572311401367 total time spend for step 7 : 4.578012466430664 step8:final Thu Aug 28 14:55:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1379845121: ('0.35901833075514267',), 1379845112: ('0.35901833075514267',), 1379844924: ('0.35901833075514267',), 1379844911: ('0.35901833075514267',)} new output for save of step final : {1379845121: ('0.35901833075514267',), 1379845112: ('0.35901833075514267',), 1379844924: ('0.35901833075514267',), 1379844911: ('0.35901833075514267',)} [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 4 /1379845121.Didn't retrieve data . /1379845112.Didn't retrieve data . /1379844924.Didn't retrieve data . /1379844911.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.014639139175415039 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12686443328857422 time spend to save output : 0.015116691589355469 total time spend for step 8 : 0.1419811248779297 step9:velours_tree Thu Aug 28 14:55:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.0826518535614014 time spend to save output : 3.4332275390625e-05 total time spend for step 9 : 1.082686185836792 step10:send_mail_cod Thu Aug 28 14:55:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26291505_28-08-2025_14_55_06.pdf 26294707 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947071756385706 26294708 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947081756385708 26294709 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947091756385709 26294710 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947101756385710 26294711 change filename to text .change filename to text .imagette262947111756385711 26294712 imagette262947121756385711 26294713 imagette262947131756385711 26294714 imagette262947141756385711 26294715 imagette262947151756385711 26294716 imagette262947161756385711 26294717 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947171756385711 26294718 change filename to text .change filename to text .imagette262947181756385712 26294719 imagette262947191756385712 26294720 imagette262947201756385712 26294721 imagette262947211756385712 26294722 imagette262947221756385712 26294723 imagette262947231756385712 26294724 change filename to text .change filename to text .imagette262947241756385712 26294725 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947251756385712 26294727 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette262947271756385713 26294728 imagette262947281756385715 26294729 imagette262947291756385715 26294730 imagette262947301756385715 26294731 imagette262947311756385715 26294732 imagette262947321756385715 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26291505 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/26294734,26294735,26294736,26294737,26294738,26294743,26294745,26294746,26294747,26294748,26294749,26294750,26294752,26294753,26294754,26294755,26294756,26294757,26294758,26294759?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26291505 your option no_mail is active, we will not send the real mail to your client args[1379845121] : ((1379845121, 988.7439875814889, 2107751945), (1379845121, -0.3144491257103454, 496442774), '0.35901833075514267') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1379845112] : ((1379845112, 1496.3255671577597, 2107751945), (1379845112, -0.38693917076083206, 496442774), '0.35901833075514267') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1379844924] : ((1379844924, 1760.1775979454248, 2107751945), (1379844924, -0.3609100249017539, 496442774), '0.35901833075514267') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1379844911] : ((1379844911, 214.98366129230544, 492688767), (1379844911, -0.6306277279166363, 501862349), '0.35901833075514267') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.35901833075514267 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291505_28-08-2025_14_55_06.pdf results_Auto_P26291505_28-08-2025_14_55_06.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291505_28-08-2025_14_55_06.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','26291505','results_Auto_P26291505_28-08-2025_14_55_06.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291505_28-08-2025_14_55_06.pdf','pdf','','2.1','0.35901833075514267') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.01444697380065918 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.573070526123047 time spend to save output : 0.014606714248657227 total time spend for step 10 : 11.587677240371704 step11:split_time_score Thu Aug 28 14:55:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('32', 1), ('33', 3)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 26082025 26291505 Nombre de photos uploadées : 4 / 23040 (0%) 26082025 26291505 Nombre de photos taguées (types de déchets): 0 / 4 (0%) 26082025 26291505 Nombre de photos taguées (volume) : 0 / 4 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ???? elapsed_time : fill_and_build_computed_from_old_data 0.00026416778564453125 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20601344108581543 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291504 order by id desc limit 1 Qualite : 0.35901833075514267 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291505_28-08-2025_14_55_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291505 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291505 AND mptpi.`type`=4855 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291506 order by id desc limit 1 Qualite : 0.2567984737532319 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291507_28-08-2025_14_35_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291507 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291507 AND mptpi.`type`=4855 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291508 order by id desc limit 1 Qualite : 0.44521359328707644 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291509_28-08-2025_14_18_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291509 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291509 AND mptpi.`type`=4855 To do Qualite : 0.5227519267425498 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26291510_28-08-2025_12_42_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26291510 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26291510 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'26082025': {'nb_upload': 4, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1379845121, 1379845112, 1379844924, 1379844911] Looping around the photos to save general results len do output : 1 /26291505Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845121', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379845112', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844924', None, None, None, None, None, '3623814') ('4741', None, None, None, None, None, None, None, '3623814') ('4741', '26291505', '1379844911', None, None, None, None, None, '3623814') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.01803755760192871 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.4764750003814697 time spend to save output : 0.018232345581054688 total time spend for step 11 : 3.4947073459625244 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 4 set_done_treatment 506.98user 254.20system 13:56.84elapsed 90%CPU (0avgtext+0avgdata 7006216maxresident)k 1564224inputs+477568outputs (28848major+41270156minor)pagefaults 0swaps