python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2219837 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2799188'] with mtr_portfolio_ids : ['22731303'] and first list_photo_ids : [] new path : /proc/2219837/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 4 ; length of list_pids : 4 ; length of list_args : 4 time to download the photos : 1.0187008380889893 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue May 6 22:01:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 22:01:31.016763: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 22:01:31.043436: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 22:01:31.045801: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8ed0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 22:01:31.045843: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 22:01:31.050431: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 22:01:31.305806: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x25ce32d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 22:01:31.305866: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 22:01:31.307246: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:01:31.307771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:01:31.311619: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:01:31.314795: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:01:31.315378: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:01:31.317751: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:01:31.318835: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:01:31.323688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:01:31.325156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:01:31.325236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:01:31.325986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 22:01:31.326002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 22:01:31.326011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 22:01:31.327309: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 22:01:31.641842: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:01:31.641932: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:01:31.641954: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:01:31.641973: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:01:31.641991: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:01:31.642008: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:01:31.642025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:01:31.642044: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:01:31.643634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:01:31.645090: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:01:31.645124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:01:31.645143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:01:31.645160: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:01:31.645193: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:01:31.645211: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:01:31.645228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:01:31.645245: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:01:31.646798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:01:31.646833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 22:01:31.646843: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 22:01:31.646853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 22:01:31.648526: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 22:01:45.045031: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:01:45.228920: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 4 Detection mask done ! Trying to reset tf kernel 2220045 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.013585805892944336 nb_pixel_total : 324751 time to create 1 rle with new method : 0.01946544647216797 length of segment : 531 time for calcul the mask position with numpy : 0.0020177364349365234 nb_pixel_total : 114194 time to create 1 rle with old method : 0.13223528861999512 length of segment : 436 time for calcul the mask position with numpy : 0.0010232925415039062 nb_pixel_total : 47729 time to create 1 rle with old method : 0.05848836898803711 length of segment : 366 time for calcul the mask position with numpy : 0.0006704330444335938 nb_pixel_total : 25926 time to create 1 rle with old method : 0.0302431583404541 length of segment : 253 time for calcul the mask position with numpy : 0.0006921291351318359 nb_pixel_total : 41612 time to create 1 rle with old method : 0.05255627632141113 length of segment : 170 time for calcul the mask position with numpy : 0.0010905265808105469 nb_pixel_total : 65686 time to create 1 rle with old method : 0.08271646499633789 length of segment : 321 time for calcul the mask position with numpy : 0.0009102821350097656 nb_pixel_total : 55770 time to create 1 rle with old method : 0.06361079216003418 length of segment : 349 time for calcul the mask position with numpy : 0.002138853073120117 nb_pixel_total : 120515 time to create 1 rle with old method : 0.14629054069519043 length of segment : 682 time for calcul the mask position with numpy : 0.006262540817260742 nb_pixel_total : 249126 time to create 1 rle with new method : 0.01582932472229004 length of segment : 948 time for calcul the mask position with numpy : 0.0052013397216796875 nb_pixel_total : 262282 time to create 1 rle with new method : 0.015067815780639648 length of segment : 577 time for calcul the mask position with numpy : 0.0005397796630859375 nb_pixel_total : 15177 time to create 1 rle with old method : 0.01866936683654785 length of segment : 170 time for calcul the mask position with numpy : 0.0007653236389160156 nb_pixel_total : 43592 time to create 1 rle with old method : 0.050050973892211914 length of segment : 344 time for calcul the mask position with numpy : 0.0005660057067871094 nb_pixel_total : 17159 time to create 1 rle with old method : 0.025165557861328125 length of segment : 309 time for calcul the mask position with numpy : 0.0814962387084961 nb_pixel_total : 337882 time to create 1 rle with new method : 0.022591352462768555 length of segment : 553 time for calcul the mask position with numpy : 0.0021605491638183594 nb_pixel_total : 134196 time to create 1 rle with old method : 0.1611473560333252 length of segment : 513 time for calcul the mask position with numpy : 0.0006535053253173828 nb_pixel_total : 27686 time to create 1 rle with old method : 0.031713008880615234 length of segment : 180 time for calcul the mask position with numpy : 0.01878833770751953 nb_pixel_total : 460098 time to create 1 rle with new method : 0.0359647274017334 length of segment : 997 time for calcul the mask position with numpy : 0.002498149871826172 nb_pixel_total : 121257 time to create 1 rle with old method : 0.15650177001953125 length of segment : 403 time for calcul the mask position with numpy : 0.0011048316955566406 nb_pixel_total : 33745 time to create 1 rle with old method : 0.04143357276916504 length of segment : 240 time for calcul the mask position with numpy : 0.0009067058563232422 nb_pixel_total : 32979 time to create 1 rle with old method : 0.03941059112548828 length of segment : 255 time for calcul the mask position with numpy : 0.0007493495941162109 nb_pixel_total : 22455 time to create 1 rle with old method : 0.02558755874633789 length of segment : 168 time for calcul the mask position with numpy : 0.0005018711090087891 nb_pixel_total : 15280 time to create 1 rle with old method : 0.017639875411987305 length of segment : 211 time for calcul the mask position with numpy : 0.0008671283721923828 nb_pixel_total : 29361 time to create 1 rle with old method : 0.04059791564941406 length of segment : 269 time for calcul the mask position with numpy : 0.00042557716369628906 nb_pixel_total : 9100 time to create 1 rle with old method : 0.014614582061767578 length of segment : 108 time for calcul the mask position with numpy : 0.0009448528289794922 nb_pixel_total : 15778 time to create 1 rle with old method : 0.025084972381591797 length of segment : 200 time for calcul the mask position with numpy : 0.010785102844238281 nb_pixel_total : 434207 time to create 1 rle with new method : 0.0332186222076416 length of segment : 604 time for calcul the mask position with numpy : 0.0010559558868408203 nb_pixel_total : 38249 time to create 1 rle with old method : 0.043160438537597656 length of segment : 298 time for calcul the mask position with numpy : 0.0019154548645019531 nb_pixel_total : 86144 time to create 1 rle with old method : 0.09709405899047852 length of segment : 270 time for calcul the mask position with numpy : 0.003723621368408203 nb_pixel_total : 140014 time to create 1 rle with old method : 0.15633344650268555 length of segment : 569 time for calcul the mask position with numpy : 0.0010175704956054688 nb_pixel_total : 28527 time to create 1 rle with old method : 0.03297758102416992 length of segment : 238 time for calcul the mask position with numpy : 0.0005807876586914062 nb_pixel_total : 12281 time to create 1 rle with old method : 0.02044367790222168 length of segment : 127 time for calcul the mask position with numpy : 0.0005400180816650391 nb_pixel_total : 17308 time to create 1 rle with old method : 0.019901752471923828 length of segment : 189 time for calcul the mask position with numpy : 0.00025010108947753906 nb_pixel_total : 12466 time to create 1 rle with old method : 0.014513254165649414 length of segment : 117 time for calcul the mask position with numpy : 0.0002853870391845703 nb_pixel_total : 17087 time to create 1 rle with old method : 0.019527673721313477 length of segment : 168 time for calcul the mask position with numpy : 0.15940523147583008 nb_pixel_total : 2459799 time to create 1 rle with new method : 0.46068310737609863 length of segment : 4166 time for calcul the mask position with numpy : 0.3509550094604492 nb_pixel_total : 4478828 time to create 1 rle with new method : 0.24343562126159668 length of segment : 2033 time for calcul the mask position with numpy : 0.30410099029541016 nb_pixel_total : 6092469 time to create 1 rle with new method : 0.7492828369140625 length of segment : 2267 time for calcul the mask position with numpy : 0.017026185989379883 nb_pixel_total : 1193216 time to create 1 rle with new method : 0.10506749153137207 length of segment : 804 time spent for convertir_results : 10.445659399032593 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 38 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 48.575798988342285 time spend to save output : 0.09366059303283691 total time spend for step 1 : 48.66945958137512 step2:mask_detect Tue May 6 22:02:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 22:02:19.544228: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 22:02:19.575061: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 22:02:19.577157: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f8ec4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 22:02:19.577190: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 22:02:19.581672: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 22:02:19.828557: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x26c852e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 22:02:19.828613: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 22:02:19.830169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:02:19.830891: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:02:19.835402: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:02:19.838002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:02:19.838444: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:02:19.841506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:02:19.843000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:02:19.847983: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:02:19.849465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:02:19.849549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:02:19.850299: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 22:02:19.850314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 22:02:19.850323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 22:02:19.851642: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 22:02:19.966259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:02:19.966373: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:02:19.966398: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:02:19.966449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:02:19.966472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:02:19.966493: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:02:19.966514: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:02:19.966536: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:02:19.968203: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:02:19.969523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 22:02:19.969562: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 22:02:19.969584: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:02:19.969604: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 22:02:19.969624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 22:02:19.969644: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 22:02:19.969664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 22:02:19.969683: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 22:02:19.971401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 22:02:19.971434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 22:02:19.971444: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 22:02:19.971453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 22:02:19.973167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 22:02:32.538612: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 22:02:32.697703: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 4 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 52 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 77 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (4160, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 2222072 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 tf kernel not reseted sub process len(results) : 4 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 4 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.01619100570678711 nb_pixel_total : 313905 time to create 1 rle with new method : 0.02868342399597168 length of segment : 789 time for calcul the mask position with numpy : 0.0035026073455810547 nb_pixel_total : 191042 time to create 1 rle with new method : 0.009939432144165039 length of segment : 467 time for calcul the mask position with numpy : 0.000978231430053711 nb_pixel_total : 47582 time to create 1 rle with old method : 0.054427385330200195 length of segment : 235 time for calcul the mask position with numpy : 0.0018191337585449219 nb_pixel_total : 87716 time to create 1 rle with old method : 0.1013944149017334 length of segment : 494 time for calcul the mask position with numpy : 0.0014324188232421875 nb_pixel_total : 72257 time to create 1 rle with old method : 0.08279037475585938 length of segment : 404 time for calcul the mask position with numpy : 0.005101203918457031 nb_pixel_total : 229035 time to create 1 rle with new method : 0.014487266540527344 length of segment : 882 time for calcul the mask position with numpy : 0.0011472702026367188 nb_pixel_total : 57462 time to create 1 rle with old method : 0.0632479190826416 length of segment : 250 time for calcul the mask position with numpy : 0.0008602142333984375 nb_pixel_total : 38607 time to create 1 rle with old method : 0.05786609649658203 length of segment : 165 time for calcul the mask position with numpy : 0.0066835880279541016 nb_pixel_total : 222147 time to create 1 rle with new method : 0.015146970748901367 length of segment : 961 time for calcul the mask position with numpy : 0.0054552555084228516 nb_pixel_total : 184818 time to create 1 rle with new method : 0.01846909523010254 length of segment : 476 time for calcul the mask position with numpy : 0.007197380065917969 nb_pixel_total : 268219 time to create 1 rle with new method : 0.014687776565551758 length of segment : 738 time for calcul the mask position with numpy : 0.0308380126953125 nb_pixel_total : 1000801 time to create 1 rle with new method : 0.17987704277038574 length of segment : 1764 time for calcul the mask position with numpy : 0.0033888816833496094 nb_pixel_total : 136611 time to create 1 rle with old method : 0.15358662605285645 length of segment : 499 time for calcul the mask position with numpy : 0.012829065322875977 nb_pixel_total : 659079 time to create 1 rle with new method : 0.15410423278808594 length of segment : 1099 time for calcul the mask position with numpy : 0.0011281967163085938 nb_pixel_total : 55400 time to create 1 rle with old method : 0.06310558319091797 length of segment : 236 time for calcul the mask position with numpy : 0.026622772216796875 nb_pixel_total : 1034173 time to create 1 rle with new method : 0.13093066215515137 length of segment : 2069 time for calcul the mask position with numpy : 0.011649847030639648 nb_pixel_total : 650183 time to create 1 rle with new method : 0.05646038055419922 length of segment : 1058 time for calcul the mask position with numpy : 0.0009698867797851562 nb_pixel_total : 33475 time to create 1 rle with old method : 0.053170204162597656 length of segment : 251 time for calcul the mask position with numpy : 0.003953456878662109 nb_pixel_total : 159638 time to create 1 rle with new method : 0.012093782424926758 length of segment : 426 time for calcul the mask position with numpy : 0.0011184215545654297 nb_pixel_total : 63401 time to create 1 rle with old method : 0.06982111930847168 length of segment : 281 time for calcul the mask position with numpy : 0.008977651596069336 nb_pixel_total : 379616 time to create 1 rle with new method : 0.02358698844909668 length of segment : 590 time for calcul the mask position with numpy : 0.006103992462158203 nb_pixel_total : 147514 time to create 1 rle with old method : 0.17433905601501465 length of segment : 549 time for calcul the mask position with numpy : 0.008816003799438477 nb_pixel_total : 295136 time to create 1 rle with new method : 0.02301502227783203 length of segment : 640 time for calcul the mask position with numpy : 0.0035758018493652344 nb_pixel_total : 157483 time to create 1 rle with new method : 0.008853912353515625 length of segment : 546 time for calcul the mask position with numpy : 0.004647254943847656 nb_pixel_total : 261016 time to create 1 rle with new method : 0.011733531951904297 length of segment : 719 time for calcul the mask position with numpy : 0.02059650421142578 nb_pixel_total : 236757 time to create 1 rle with new method : 0.016800880432128906 length of segment : 1256 time for calcul the mask position with numpy : 0.0016837120056152344 nb_pixel_total : 76163 time to create 1 rle with old method : 0.08798098564147949 length of segment : 308 time for calcul the mask position with numpy : 0.0009694099426269531 nb_pixel_total : 62070 time to create 1 rle with old method : 0.06749320030212402 length of segment : 197 time for calcul the mask position with numpy : 0.0024476051330566406 nb_pixel_total : 119366 time to create 1 rle with old method : 0.12804365158081055 length of segment : 359 time for calcul the mask position with numpy : 0.026908159255981445 nb_pixel_total : 1224942 time to create 1 rle with new method : 0.27449607849121094 length of segment : 2359 time for calcul the mask position with numpy : 0.013196945190429688 nb_pixel_total : 480154 time to create 1 rle with new method : 0.08104062080383301 length of segment : 1247 time for calcul the mask position with numpy : 0.0015647411346435547 nb_pixel_total : 99308 time to create 1 rle with old method : 0.11233067512512207 length of segment : 468 time for calcul the mask position with numpy : 0.0010483264923095703 nb_pixel_total : 32950 time to create 1 rle with old method : 0.036319732666015625 length of segment : 304 time for calcul the mask position with numpy : 0.03902029991149902 nb_pixel_total : 1159450 time to create 1 rle with new method : 0.15720129013061523 length of segment : 1668 time for calcul the mask position with numpy : 0.0009777545928955078 nb_pixel_total : 69353 time to create 1 rle with old method : 0.07538127899169922 length of segment : 290 time for calcul the mask position with numpy : 0.0018579959869384766 nb_pixel_total : 100317 time to create 1 rle with old method : 0.10862541198730469 length of segment : 351 time for calcul the mask position with numpy : 0.003690004348754883 nb_pixel_total : 205426 time to create 1 rle with new method : 0.010285139083862305 length of segment : 552 time for calcul the mask position with numpy : 0.003667116165161133 nb_pixel_total : 205112 time to create 1 rle with new method : 0.010197162628173828 length of segment : 545 time for calcul the mask position with numpy : 0.004689693450927734 nb_pixel_total : 322040 time to create 1 rle with new method : 0.01141977310180664 length of segment : 531 time for calcul the mask position with numpy : 0.0014846324920654297 nb_pixel_total : 79486 time to create 1 rle with old method : 0.08738994598388672 length of segment : 339 time for calcul the mask position with numpy : 0.0070841312408447266 nb_pixel_total : 307763 time to create 1 rle with new method : 0.03165149688720703 length of segment : 792 time for calcul the mask position with numpy : 0.001027822494506836 nb_pixel_total : 60543 time to create 1 rle with old method : 0.06520295143127441 length of segment : 309 time for calcul the mask position with numpy : 0.0012176036834716797 nb_pixel_total : 75302 time to create 1 rle with old method : 0.08249235153198242 length of segment : 346 time for calcul the mask position with numpy : 0.0005271434783935547 nb_pixel_total : 25494 time to create 1 rle with old method : 0.028957605361938477 length of segment : 168 time for calcul the mask position with numpy : 0.0016756057739257812 nb_pixel_total : 59175 time to create 1 rle with old method : 0.06525802612304688 length of segment : 292 time for calcul the mask position with numpy : 0.003299713134765625 nb_pixel_total : 181549 time to create 1 rle with new method : 0.009099006652832031 length of segment : 673 time for calcul the mask position with numpy : 0.0009720325469970703 nb_pixel_total : 57393 time to create 1 rle with old method : 0.06550931930541992 length of segment : 209 time for calcul the mask position with numpy : 0.005315542221069336 nb_pixel_total : 305374 time to create 1 rle with new method : 0.014789581298828125 length of segment : 676 time for calcul the mask position with numpy : 0.0034165382385253906 nb_pixel_total : 135249 time to create 1 rle with old method : 0.14724278450012207 length of segment : 468 time for calcul the mask position with numpy : 0.0029478073120117188 nb_pixel_total : 153614 time to create 1 rle with new method : 0.006646394729614258 length of segment : 373 time for calcul the mask position with numpy : 0.0025076866149902344 nb_pixel_total : 142984 time to create 1 rle with old method : 0.19429469108581543 length of segment : 399 time for calcul the mask position with numpy : 0.0019195079803466797 nb_pixel_total : 127659 time to create 1 rle with old method : 0.13640499114990234 length of segment : 303 time for calcul the mask position with numpy : 0.0013692378997802734 nb_pixel_total : 41576 time to create 1 rle with old method : 0.04528617858886719 length of segment : 285 time for calcul the mask position with numpy : 0.0027077198028564453 nb_pixel_total : 94898 time to create 1 rle with old method : 0.1187584400177002 length of segment : 408 time for calcul the mask position with numpy : 0.002750873565673828 nb_pixel_total : 124932 time to create 1 rle with old method : 0.1311352252960205 length of segment : 594 time for calcul the mask position with numpy : 0.010663509368896484 nb_pixel_total : 393774 time to create 1 rle with new method : 0.04176950454711914 length of segment : 1445 time for calcul the mask position with numpy : 0.0029296875 nb_pixel_total : 145067 time to create 1 rle with old method : 0.1520993709564209 length of segment : 512 time for calcul the mask position with numpy : 0.0010008811950683594 nb_pixel_total : 42543 time to create 1 rle with old method : 0.0456392765045166 length of segment : 395 time for calcul the mask position with numpy : 0.0008099079132080078 nb_pixel_total : 29711 time to create 1 rle with old method : 0.047347068786621094 length of segment : 209 time for calcul the mask position with numpy : 0.0009145736694335938 nb_pixel_total : 32879 time to create 1 rle with old method : 0.03717660903930664 length of segment : 195 time for calcul the mask position with numpy : 0.0009732246398925781 nb_pixel_total : 44042 time to create 1 rle with old method : 0.04772520065307617 length of segment : 189 time for calcul the mask position with numpy : 0.0009174346923828125 nb_pixel_total : 26577 time to create 1 rle with old method : 0.029652118682861328 length of segment : 173 time for calcul the mask position with numpy : 0.006516218185424805 nb_pixel_total : 314576 time to create 1 rle with new method : 0.019869565963745117 length of segment : 909 time for calcul the mask position with numpy : 0.0005478858947753906 nb_pixel_total : 25168 time to create 1 rle with old method : 0.028035402297973633 length of segment : 217 time for calcul the mask position with numpy : 0.0009984970092773438 nb_pixel_total : 46259 time to create 1 rle with old method : 0.05147552490234375 length of segment : 330 time for calcul the mask position with numpy : 0.0017843246459960938 nb_pixel_total : 74258 time to create 1 rle with old method : 0.08376741409301758 length of segment : 354 time for calcul the mask position with numpy : 0.0014469623565673828 nb_pixel_total : 50921 time to create 1 rle with old method : 0.057961463928222656 length of segment : 203 time for calcul the mask position with numpy : 0.000732421875 nb_pixel_total : 29788 time to create 1 rle with old method : 0.033857107162475586 length of segment : 167 time for calcul the mask position with numpy : 0.002879619598388672 nb_pixel_total : 110939 time to create 1 rle with old method : 0.11960172653198242 length of segment : 448 time for calcul the mask position with numpy : 0.0017242431640625 nb_pixel_total : 45390 time to create 1 rle with old method : 0.05012679100036621 length of segment : 338 time for calcul the mask position with numpy : 0.005532503128051758 nb_pixel_total : 296336 time to create 1 rle with new method : 0.013084650039672852 length of segment : 504 time for calcul the mask position with numpy : 0.0012788772583007812 nb_pixel_total : 54821 time to create 1 rle with old method : 0.05942082405090332 length of segment : 372 time for calcul the mask position with numpy : 0.0006127357482910156 nb_pixel_total : 24218 time to create 1 rle with old method : 0.02727961540222168 length of segment : 160 time for calcul the mask position with numpy : 0.005877971649169922 nb_pixel_total : 258272 time to create 1 rle with new method : 0.016677379608154297 length of segment : 602 time for calcul the mask position with numpy : 0.0010251998901367188 nb_pixel_total : 42790 time to create 1 rle with old method : 0.047089338302612305 length of segment : 201 time for calcul the mask position with numpy : 0.0008614063262939453 nb_pixel_total : 29583 time to create 1 rle with old method : 0.03267359733581543 length of segment : 235 time for calcul the mask position with numpy : 0.0037310123443603516 nb_pixel_total : 148267 time to create 1 rle with old method : 0.16225910186767578 length of segment : 535 time for calcul the mask position with numpy : 0.02178192138671875 nb_pixel_total : 805504 time to create 1 rle with new method : 0.2730379104614258 length of segment : 1605 time for calcul the mask position with numpy : 0.001154184341430664 nb_pixel_total : 38932 time to create 1 rle with old method : 0.06275129318237305 length of segment : 285 time for calcul the mask position with numpy : 0.002856731414794922 nb_pixel_total : 155394 time to create 1 rle with new method : 0.008083820343017578 length of segment : 529 time for calcul the mask position with numpy : 0.008014678955078125 nb_pixel_total : 364572 time to create 1 rle with new method : 0.029567956924438477 length of segment : 684 time for calcul the mask position with numpy : 0.0005276203155517578 nb_pixel_total : 17052 time to create 1 rle with old method : 0.019093751907348633 length of segment : 183 time for calcul the mask position with numpy : 0.0027036666870117188 nb_pixel_total : 120346 time to create 1 rle with old method : 0.15949583053588867 length of segment : 663 time for calcul the mask position with numpy : 0.0012359619140625 nb_pixel_total : 65401 time to create 1 rle with old method : 0.0723874568939209 length of segment : 268 time for calcul the mask position with numpy : 0.004252433776855469 nb_pixel_total : 185433 time to create 1 rle with new method : 0.011562347412109375 length of segment : 930 time for calcul the mask position with numpy : 0.001811981201171875 nb_pixel_total : 80828 time to create 1 rle with old method : 0.09224247932434082 length of segment : 548 time for calcul the mask position with numpy : 0.01550149917602539 nb_pixel_total : 667177 time to create 1 rle with new method : 0.1019277572631836 length of segment : 1256 time for calcul the mask position with numpy : 0.0008304119110107422 nb_pixel_total : 33629 time to create 1 rle with old method : 0.03652596473693848 length of segment : 278 time for calcul the mask position with numpy : 0.0026912689208984375 nb_pixel_total : 130771 time to create 1 rle with old method : 0.14940953254699707 length of segment : 580 time for calcul the mask position with numpy : 0.0025713443756103516 nb_pixel_total : 103437 time to create 1 rle with old method : 0.12266063690185547 length of segment : 326 time for calcul the mask position with numpy : 0.017007827758789062 nb_pixel_total : 569621 time to create 1 rle with new method : 0.10930562019348145 length of segment : 998 time for calcul the mask position with numpy : 0.0010967254638671875 nb_pixel_total : 41016 time to create 1 rle with old method : 0.04679608345031738 length of segment : 192 time for calcul the mask position with numpy : 0.0020952224731445312 nb_pixel_total : 121494 time to create 1 rle with old method : 0.15869927406311035 length of segment : 414 time for calcul the mask position with numpy : 0.0006978511810302734 nb_pixel_total : 27990 time to create 1 rle with old method : 0.033484697341918945 length of segment : 216 time for calcul the mask position with numpy : 0.0030851364135742188 nb_pixel_total : 102794 time to create 1 rle with old method : 0.13125371932983398 length of segment : 626 time for calcul the mask position with numpy : 0.0058917999267578125 nb_pixel_total : 267782 time to create 1 rle with new method : 0.026105880737304688 length of segment : 1553 time for calcul the mask position with numpy : 0.0008146762847900391 nb_pixel_total : 34153 time to create 1 rle with old method : 0.0389859676361084 length of segment : 183 time for calcul the mask position with numpy : 0.001439809799194336 nb_pixel_total : 57030 time to create 1 rle with old method : 0.07690310478210449 length of segment : 308 time for calcul the mask position with numpy : 0.0008869171142578125 nb_pixel_total : 14631 time to create 1 rle with old method : 0.01655721664428711 length of segment : 225 time for calcul the mask position with numpy : 0.00045871734619140625 nb_pixel_total : 17770 time to create 1 rle with old method : 0.020623445510864258 length of segment : 124 time for calcul the mask position with numpy : 0.0022101402282714844 nb_pixel_total : 106121 time to create 1 rle with old method : 0.1208949089050293 length of segment : 415 time for calcul the mask position with numpy : 0.0009844303131103516 nb_pixel_total : 23827 time to create 1 rle with old method : 0.02676081657409668 length of segment : 278 time for calcul the mask position with numpy : 0.0030825138092041016 nb_pixel_total : 138191 time to create 1 rle with old method : 0.1923828125 length of segment : 282 time for calcul the mask position with numpy : 0.031195402145385742 nb_pixel_total : 1014026 time to create 1 rle with new method : 0.2567012310028076 length of segment : 1203 time for calcul the mask position with numpy : 0.0008795261383056641 nb_pixel_total : 23403 time to create 1 rle with old method : 0.02701282501220703 length of segment : 266 time for calcul the mask position with numpy : 0.0020749568939208984 nb_pixel_total : 85422 time to create 1 rle with old method : 0.09619975090026855 length of segment : 448 time for calcul the mask position with numpy : 0.0016782283782958984 nb_pixel_total : 75510 time to create 1 rle with old method : 0.08219552040100098 length of segment : 262 time for calcul the mask position with numpy : 0.002454042434692383 nb_pixel_total : 32412 time to create 1 rle with old method : 0.037039995193481445 length of segment : 316 time for calcul the mask position with numpy : 0.006525754928588867 nb_pixel_total : 377635 time to create 1 rle with new method : 0.018476486206054688 length of segment : 455 time for calcul the mask position with numpy : 0.0007202625274658203 nb_pixel_total : 22665 time to create 1 rle with old method : 0.025985240936279297 length of segment : 135 time for calcul the mask position with numpy : 0.0007021427154541016 nb_pixel_total : 44391 time to create 1 rle with old method : 0.0503084659576416 length of segment : 237 time for calcul the mask position with numpy : 0.0016498565673828125 nb_pixel_total : 56107 time to create 1 rle with old method : 0.06184101104736328 length of segment : 368 time for calcul the mask position with numpy : 0.0053827762603759766 nb_pixel_total : 170377 time to create 1 rle with new method : 0.017212867736816406 length of segment : 515 time for calcul the mask position with numpy : 0.003305196762084961 nb_pixel_total : 146459 time to create 1 rle with old method : 0.1762533187866211 length of segment : 614 time for calcul the mask position with numpy : 0.0026006698608398438 nb_pixel_total : 133469 time to create 1 rle with old method : 0.1445331573486328 length of segment : 416 time for calcul the mask position with numpy : 0.0174715518951416 nb_pixel_total : 655396 time to create 1 rle with new method : 0.09599781036376953 length of segment : 1031 time for calcul the mask position with numpy : 0.002229928970336914 nb_pixel_total : 112587 time to create 1 rle with old method : 0.12398219108581543 length of segment : 229 time for calcul the mask position with numpy : 0.002037525177001953 nb_pixel_total : 84825 time to create 1 rle with old method : 0.09781312942504883 length of segment : 315 time for calcul the mask position with numpy : 0.0009102821350097656 nb_pixel_total : 44708 time to create 1 rle with old method : 0.05305171012878418 length of segment : 196 time for calcul the mask position with numpy : 0.006005287170410156 nb_pixel_total : 254611 time to create 1 rle with new method : 0.01802825927734375 length of segment : 914 time for calcul the mask position with numpy : 0.002168893814086914 nb_pixel_total : 70391 time to create 1 rle with old method : 0.09923028945922852 length of segment : 354 time for calcul the mask position with numpy : 0.0023267269134521484 nb_pixel_total : 104328 time to create 1 rle with old method : 0.11732101440429688 length of segment : 374 time for calcul the mask position with numpy : 0.0025751590728759766 nb_pixel_total : 133296 time to create 1 rle with old method : 0.15043878555297852 length of segment : 632 time for calcul the mask position with numpy : 0.0013737678527832031 nb_pixel_total : 66457 time to create 1 rle with old method : 0.07733798027038574 length of segment : 364 time for calcul the mask position with numpy : 0.008863210678100586 nb_pixel_total : 403222 time to create 1 rle with new method : 0.04343748092651367 length of segment : 481 time for calcul the mask position with numpy : 0.003516674041748047 nb_pixel_total : 146841 time to create 1 rle with old method : 0.1897745132446289 length of segment : 492 time for calcul the mask position with numpy : 0.001569509506225586 nb_pixel_total : 81937 time to create 1 rle with old method : 0.1316087245941162 length of segment : 271 time for calcul the mask position with numpy : 0.003124713897705078 nb_pixel_total : 118383 time to create 1 rle with old method : 0.13085126876831055 length of segment : 472 time for calcul the mask position with numpy : 0.002399921417236328 nb_pixel_total : 119408 time to create 1 rle with old method : 0.134446382522583 length of segment : 376 time for calcul the mask position with numpy : 0.0007607936859130859 nb_pixel_total : 28454 time to create 1 rle with old method : 0.032378196716308594 length of segment : 139 time for calcul the mask position with numpy : 0.0036962032318115234 nb_pixel_total : 206863 time to create 1 rle with new method : 0.009406805038452148 length of segment : 475 time for calcul the mask position with numpy : 0.0026123523712158203 nb_pixel_total : 156913 time to create 1 rle with new method : 0.006199359893798828 length of segment : 595 time for calcul the mask position with numpy : 0.001190185546875 nb_pixel_total : 41224 time to create 1 rle with old method : 0.05105710029602051 length of segment : 699 time for calcul the mask position with numpy : 0.0066375732421875 nb_pixel_total : 430762 time to create 1 rle with new method : 0.016404151916503906 length of segment : 938 time for calcul the mask position with numpy : 0.007609367370605469 nb_pixel_total : 416515 time to create 1 rle with new method : 0.01634955406188965 length of segment : 645 time for calcul the mask position with numpy : 0.03601384162902832 nb_pixel_total : 748730 time to create 1 rle with new method : 0.08615636825561523 length of segment : 832 time for calcul the mask position with numpy : 0.0047779083251953125 nb_pixel_total : 268086 time to create 1 rle with new method : 0.01909613609313965 length of segment : 701 time for calcul the mask position with numpy : 0.013689756393432617 nb_pixel_total : 754196 time to create 1 rle with new method : 0.1858808994293213 length of segment : 715 time for calcul the mask position with numpy : 0.0009815692901611328 nb_pixel_total : 52970 time to create 1 rle with old method : 0.06135201454162598 length of segment : 157 time for calcul the mask position with numpy : 0.0006301403045654297 nb_pixel_total : 25410 time to create 1 rle with old method : 0.02820134162902832 length of segment : 349 time for calcul the mask position with numpy : 0.0030710697174072266 nb_pixel_total : 165672 time to create 1 rle with new method : 0.007720232009887695 length of segment : 427 time for calcul the mask position with numpy : 0.007500886917114258 nb_pixel_total : 397698 time to create 1 rle with new method : 0.02027297019958496 length of segment : 1159 time for calcul the mask position with numpy : 0.005083799362182617 nb_pixel_total : 241140 time to create 1 rle with new method : 0.014997005462646484 length of segment : 471 time for calcul the mask position with numpy : 0.005093812942504883 nb_pixel_total : 181475 time to create 1 rle with new method : 0.017642736434936523 length of segment : 647 time for calcul the mask position with numpy : 0.005068302154541016 nb_pixel_total : 217205 time to create 1 rle with new method : 0.016224145889282227 length of segment : 1023 time for calcul the mask position with numpy : 0.0008153915405273438 nb_pixel_total : 44422 time to create 1 rle with old method : 0.0481719970703125 length of segment : 219 time for calcul the mask position with numpy : 0.002599477767944336 nb_pixel_total : 127106 time to create 1 rle with old method : 0.15223121643066406 length of segment : 325 time for calcul the mask position with numpy : 0.0007905960083007812 nb_pixel_total : 31903 time to create 1 rle with old method : 0.035216569900512695 length of segment : 291 time for calcul the mask position with numpy : 0.003965616226196289 nb_pixel_total : 203347 time to create 1 rle with new method : 0.013003349304199219 length of segment : 727 time for calcul the mask position with numpy : 0.0019173622131347656 nb_pixel_total : 103385 time to create 1 rle with old method : 0.11250615119934082 length of segment : 417 time for calcul the mask position with numpy : 0.0023450851440429688 nb_pixel_total : 149227 time to create 1 rle with old method : 0.17804598808288574 length of segment : 609 time for calcul the mask position with numpy : 0.001567840576171875 nb_pixel_total : 67749 time to create 1 rle with old method : 0.07593321800231934 length of segment : 292 time for calcul the mask position with numpy : 0.020193815231323242 nb_pixel_total : 679861 time to create 1 rle with new method : 0.1750040054321289 length of segment : 1315 time for calcul the mask position with numpy : 0.005983829498291016 nb_pixel_total : 402181 time to create 1 rle with new method : 0.020433664321899414 length of segment : 601 time for calcul the mask position with numpy : 0.0006222724914550781 nb_pixel_total : 29996 time to create 1 rle with old method : 0.032533884048461914 length of segment : 147 time for calcul the mask position with numpy : 0.0024411678314208984 nb_pixel_total : 126929 time to create 1 rle with old method : 0.16583919525146484 length of segment : 418 time for calcul the mask position with numpy : 0.000293731689453125 nb_pixel_total : 15391 time to create 1 rle with old method : 0.018293380737304688 length of segment : 185 time for calcul the mask position with numpy : 0.0029523372650146484 nb_pixel_total : 152029 time to create 1 rle with new method : 0.009041309356689453 length of segment : 492 time for calcul the mask position with numpy : 0.005129575729370117 nb_pixel_total : 227900 time to create 1 rle with new method : 0.01426243782043457 length of segment : 723 time for calcul the mask position with numpy : 0.0027527809143066406 nb_pixel_total : 166161 time to create 1 rle with new method : 0.0074405670166015625 length of segment : 488 time for calcul the mask position with numpy : 0.005309104919433594 nb_pixel_total : 346238 time to create 1 rle with new method : 0.013503313064575195 length of segment : 892 time for calcul the mask position with numpy : 0.004670619964599609 nb_pixel_total : 277131 time to create 1 rle with new method : 0.012257575988769531 length of segment : 745 time for calcul the mask position with numpy : 0.004331350326538086 nb_pixel_total : 239501 time to create 1 rle with new method : 0.01075887680053711 length of segment : 349 time for calcul the mask position with numpy : 0.0028676986694335938 nb_pixel_total : 138946 time to create 1 rle with old method : 0.1544322967529297 length of segment : 413 time for calcul the mask position with numpy : 0.001544952392578125 nb_pixel_total : 80583 time to create 1 rle with old method : 0.08940625190734863 length of segment : 409 time for calcul the mask position with numpy : 0.012732267379760742 nb_pixel_total : 708318 time to create 1 rle with new method : 0.0651545524597168 length of segment : 941 time for calcul the mask position with numpy : 0.0008208751678466797 nb_pixel_total : 49706 time to create 1 rle with old method : 0.0553898811340332 length of segment : 243 time for calcul the mask position with numpy : 0.0012598037719726562 nb_pixel_total : 66278 time to create 1 rle with old method : 0.07350373268127441 length of segment : 394 time for calcul the mask position with numpy : 0.0008778572082519531 nb_pixel_total : 56924 time to create 1 rle with old method : 0.0638875961303711 length of segment : 181 time for calcul the mask position with numpy : 0.0025072097778320312 nb_pixel_total : 122577 time to create 1 rle with old method : 0.13395476341247559 length of segment : 327 time for calcul the mask position with numpy : 0.010356426239013672 nb_pixel_total : 331179 time to create 1 rle with new method : 0.0708467960357666 length of segment : 1161 time for calcul the mask position with numpy : 0.0027806758880615234 nb_pixel_total : 156174 time to create 1 rle with new method : 0.00778961181640625 length of segment : 505 time for calcul the mask position with numpy : 0.021101951599121094 nb_pixel_total : 788618 time to create 1 rle with new method : 0.20777320861816406 length of segment : 766 time for calcul the mask position with numpy : 0.0017459392547607422 nb_pixel_total : 104154 time to create 1 rle with old method : 0.11586880683898926 length of segment : 262 time for calcul the mask position with numpy : 0.488131046295166 nb_pixel_total : 6203348 time to create 1 rle with new method : 1.125927448272705 length of segment : 7426 time for calcul the mask position with numpy : 0.01675105094909668 nb_pixel_total : 1141387 time to create 1 rle with new method : 0.24507784843444824 length of segment : 718 time for calcul the mask position with numpy : 0.20671916007995605 nb_pixel_total : 2159749 time to create 1 rle with new method : 0.3297736644744873 length of segment : 1380 time for calcul the mask position with numpy : 0.13939833641052246 nb_pixel_total : 2899296 time to create 1 rle with new method : 0.3977701663970947 length of segment : 1425 time for calcul the mask position with numpy : 0.02216172218322754 nb_pixel_total : 1394589 time to create 1 rle with new method : 0.19941210746765137 length of segment : 1934 time spent for convertir_results : 43.14860010147095 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 217 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 112.04410576820374 time spend to save output : 0.15253496170043945 total time spend for step 2 : 112.19664072990417 step3:blur_detection Tue May 6 22:04:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (4160, 3120) 820.327219447486 (4160, 3120) 2341.3283598206795 (4160, 3120) 1034.7216852982658 (4160, 3120) 130.86716434383777 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.013379573822021484 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.012448787689208984 save missing photos in datou_result : time spend for datou_step_exec : 1.4426300525665283 time spend to save output : 0.030596256256103516 total time spend for step 3 : 1.4732263088226318 step4:brightness Tue May 6 22:04:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746561687_2219837_1356446131_1532b8c7a37aca037128d7f388ac456d.jpg treat image : temp/1746561687_2219837_1356446129_d10af3d5b88cbd2795d0507ae83d0281.jpg treat image : temp/1746561687_2219837_1356446125_03c573ec8e24374181c40fcca07c2014.jpg treat image : temp/1746561687_2219837_1356446122_152194fe7b1174657f21592def1c13c3.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 4 time used for this insertion : 0.012708663940429688 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 4 time used for this insertion : 0.01064610481262207 save missing photos in datou_result : time spend for datou_step_exec : 6.653375625610352 time spend to save output : 0.029193639755249023 total time spend for step 4 : 6.682569265365601 step5:rle_unique_nms_with_priority Tue May 6 22:04:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 217 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 60 nb_hashtags : 9 time to prepare the origin masks : 52.90664529800415 time for calcul the mask position with numpy : 0.647860050201416 nb_pixel_total : 3912444 time to create 1 rle with new method : 1.201225757598877 time for calcul the mask position with numpy : 0.0516512393951416 nb_pixel_total : 38507 time to create 1 rle with old method : 0.04050922393798828 time for calcul the mask position with numpy : 0.054229021072387695 nb_pixel_total : 190879 time to create 1 rle with new method : 1.1111881732940674 time for calcul the mask position with numpy : 0.07655930519104004 nb_pixel_total : 657142 time to create 1 rle with new method : 0.8511316776275635 time for calcul the mask position with numpy : 0.0715339183807373 nb_pixel_total : 824485 time to create 1 rle with new method : 1.179535150527954 time for calcul the mask position with numpy : 0.05366659164428711 nb_pixel_total : 76829 time to create 1 rle with old method : 0.08437466621398926 time for calcul the mask position with numpy : 0.05515480041503906 nb_pixel_total : 698979 time to create 1 rle with new method : 0.8244500160217285 time for calcul the mask position with numpy : 0.05135297775268555 nb_pixel_total : 13764 time to create 1 rle with old method : 0.014883041381835938 time for calcul the mask position with numpy : 0.05118966102600098 nb_pixel_total : 55233 time to create 1 rle with old method : 0.058423519134521484 time for calcul the mask position with numpy : 0.05740618705749512 nb_pixel_total : 119213 time to create 1 rle with old method : 0.13692522048950195 time for calcul the mask position with numpy : 0.05105233192443848 nb_pixel_total : 11642 time to create 1 rle with old method : 0.012730598449707031 time for calcul the mask position with numpy : 0.05111575126647949 nb_pixel_total : 57330 time to create 1 rle with old method : 0.06992483139038086 time for calcul the mask position with numpy : 0.051247358322143555 nb_pixel_total : 33434 time to create 1 rle with old method : 0.03654313087463379 time for calcul the mask position with numpy : 0.052745819091796875 nb_pixel_total : 47494 time to create 1 rle with old method : 0.05122542381286621 time for calcul the mask position with numpy : 0.05319499969482422 nb_pixel_total : 25882 time to create 1 rle with old method : 0.028276920318603516 time for calcul the mask position with numpy : 0.06270790100097656 nb_pixel_total : 942072 time to create 1 rle with new method : 1.0587916374206543 time for calcul the mask position with numpy : 0.05456709861755371 nb_pixel_total : 65492 time to create 1 rle with old method : 0.0907435417175293 time for calcul the mask position with numpy : 0.0541234016418457 nb_pixel_total : 44027 time to create 1 rle with old method : 0.05086851119995117 time for calcul the mask position with numpy : 0.05747556686401367 nb_pixel_total : 114172 time to create 1 rle with old method : 0.1333768367767334 time for calcul the mask position with numpy : 0.053305625915527344 nb_pixel_total : 136504 time to create 1 rle with old method : 0.14738011360168457 time for calcul the mask position with numpy : 0.0541377067565918 nb_pixel_total : 222761 time to create 1 rle with new method : 0.9272153377532959 time for calcul the mask position with numpy : 0.05496096611022949 nb_pixel_total : 110844 time to create 1 rle with old method : 0.1185915470123291 time for calcul the mask position with numpy : 0.052873849868774414 nb_pixel_total : 184248 time to create 1 rle with new method : 0.9075171947479248 time for calcul the mask position with numpy : 0.05351972579956055 nb_pixel_total : 267926 time to create 1 rle with new method : 1.1392316818237305 time for calcul the mask position with numpy : 0.05759000778198242 nb_pixel_total : 150616 time to create 1 rle with new method : 1.1085877418518066 time for calcul the mask position with numpy : 0.051971435546875 nb_pixel_total : 54210 time to create 1 rle with old method : 0.07994985580444336 time for calcul the mask position with numpy : 0.06023049354553223 nb_pixel_total : 324224 time to create 1 rle with new method : 0.9561493396759033 time for calcul the mask position with numpy : 0.053131818771362305 nb_pixel_total : 87684 time to create 1 rle with old method : 0.09509539604187012 time for calcul the mask position with numpy : 0.05163407325744629 nb_pixel_total : 7963 time to create 1 rle with old method : 0.009038686752319336 time for calcul the mask position with numpy : 0.05966973304748535 nb_pixel_total : 994859 time to create 1 rle with new method : 2.0157742500305176 time for calcul the mask position with numpy : 0.05245709419250488 nb_pixel_total : 10891 time to create 1 rle with old method : 0.012272834777832031 time for calcul the mask position with numpy : 0.055437326431274414 nb_pixel_total : 4350 time to create 1 rle with old method : 0.009351015090942383 time for calcul the mask position with numpy : 0.05574607849121094 nb_pixel_total : 228672 time to create 1 rle with new method : 1.2707109451293945 time for calcul the mask position with numpy : 0.061197519302368164 nb_pixel_total : 127277 time to create 1 rle with old method : 0.16730189323425293 time for calcul the mask position with numpy : 0.05164504051208496 nb_pixel_total : 4251 time to create 1 rle with old method : 0.004900693893432617 time for calcul the mask position with numpy : 0.05604863166809082 nb_pixel_total : 112472 time to create 1 rle with old method : 0.12265896797180176 time for calcul the mask position with numpy : 0.0532071590423584 nb_pixel_total : 12284 time to create 1 rle with old method : 0.01401829719543457 time for calcul the mask position with numpy : 0.0539698600769043 nb_pixel_total : 41487 time to create 1 rle with old method : 0.04913473129272461 time for calcul the mask position with numpy : 0.052527427673339844 nb_pixel_total : 57234 time to create 1 rle with old method : 0.06368327140808105 time for calcul the mask position with numpy : 0.05645442008972168 nb_pixel_total : 541792 time to create 1 rle with new method : 1.25173020362854 time for calcul the mask position with numpy : 0.05492377281188965 nb_pixel_total : 47720 time to create 1 rle with old method : 0.05165553092956543 time for calcul the mask position with numpy : 0.05124950408935547 nb_pixel_total : 5182 time to create 1 rle with old method : 0.005851030349731445 time for calcul the mask position with numpy : 0.05554986000061035 nb_pixel_total : 313502 time to create 1 rle with new method : 0.7903420925140381 time for calcul the mask position with numpy : 0.05133056640625 nb_pixel_total : 1111 time to create 1 rle with old method : 0.001359701156616211 time for calcul the mask position with numpy : 0.05187034606933594 nb_pixel_total : 19999 time to create 1 rle with old method : 0.022032737731933594 time for calcul the mask position with numpy : 0.05204343795776367 nb_pixel_total : 75697 time to create 1 rle with old method : 0.08237290382385254 time for calcul the mask position with numpy : 0.05231356620788574 nb_pixel_total : 149652 time to create 1 rle with old method : 0.16203737258911133 time for calcul the mask position with numpy : 0.05219101905822754 nb_pixel_total : 4606 time to create 1 rle with old method : 0.005220651626586914 time for calcul the mask position with numpy : 0.05544853210449219 nb_pixel_total : 47711 time to create 1 rle with old method : 0.056725502014160156 time for calcul the mask position with numpy : 0.056397438049316406 nb_pixel_total : 51469 time to create 1 rle with old method : 0.05745196342468262 time for calcul the mask position with numpy : 0.058348655700683594 nb_pixel_total : 189076 time to create 1 rle with new method : 1.1071326732635498 time for calcul the mask position with numpy : 0.05183768272399902 nb_pixel_total : 6737 time to create 1 rle with old method : 0.007789134979248047 time for calcul the mask position with numpy : 0.05384373664855957 nb_pixel_total : 279364 time to create 1 rle with new method : 0.8158459663391113 time for calcul the mask position with numpy : 0.053342580795288086 nb_pixel_total : 6705 time to create 1 rle with old method : 0.0075359344482421875 time for calcul the mask position with numpy : 0.06024503707885742 nb_pixel_total : 72178 time to create 1 rle with old method : 0.07992267608642578 time for calcul the mask position with numpy : 0.05386781692504883 nb_pixel_total : 29549 time to create 1 rle with old method : 0.033364295959472656 time for calcul the mask position with numpy : 0.05392098426818848 nb_pixel_total : 24123 time to create 1 rle with old method : 0.0267484188079834 time for calcul the mask position with numpy : 0.05362653732299805 nb_pixel_total : 274 time to create 1 rle with old method : 0.0003368854522705078 time for calcul the mask position with numpy : 0.054120540618896484 nb_pixel_total : 5 time to create 1 rle with old method : 4.00543212890625e-05 time for calcul the mask position with numpy : 0.05242133140563965 nb_pixel_total : 8332 time to create 1 rle with old method : 0.009174108505249023 time for calcul the mask position with numpy : 0.053550004959106445 nb_pixel_total : 34639 time to create 1 rle with old method : 0.03834247589111328 create new chi : 25.578683376312256 time to delete rle : 0.0315093994140625 batch 1 Loaded 61 chid ids of type : 4855 Number RLEs to save : 53344 TO DO : save crop sub photo not yet done ! save time : 2.9074325561523438 nb_obj : 87 nb_hashtags : 11 time to prepare the origin masks : 64.7750129699707 time for calcul the mask position with numpy : 0.22597956657409668 nb_pixel_total : 4647889 time to create 1 rle with new method : 0.7109763622283936 time for calcul the mask position with numpy : 0.05449867248535156 nb_pixel_total : 36871 time to create 1 rle with old method : 0.04129648208618164 time for calcul the mask position with numpy : 0.07036995887756348 nb_pixel_total : 337547 time to create 1 rle with new method : 0.6284828186035156 time for calcul the mask position with numpy : 0.056685686111450195 nb_pixel_total : 754 time to create 1 rle with old method : 0.0012307167053222656 time for calcul the mask position with numpy : 0.05481219291687012 nb_pixel_total : 29252 time to create 1 rle with old method : 0.0341191291809082 time for calcul the mask position with numpy : 0.05488133430480957 nb_pixel_total : 32701 time to create 1 rle with old method : 0.03630948066711426 time for calcul the mask position with numpy : 0.0721290111541748 nb_pixel_total : 749510 time to create 1 rle with new method : 0.6361486911773682 time for calcul the mask position with numpy : 0.05401468276977539 nb_pixel_total : 23244 time to create 1 rle with old method : 0.025261640548706055 time for calcul the mask position with numpy : 0.05309152603149414 nb_pixel_total : 17013 time to create 1 rle with old method : 0.019121170043945312 time for calcul the mask position with numpy : 0.054688215255737305 nb_pixel_total : 106100 time to create 1 rle with old method : 0.11662983894348145 time for calcul the mask position with numpy : 0.054442644119262695 nb_pixel_total : 4610 time to create 1 rle with old method : 0.00526118278503418 time for calcul the mask position with numpy : 0.06067776679992676 nb_pixel_total : 50777 time to create 1 rle with old method : 0.07696413993835449 time for calcul the mask position with numpy : 0.057953596115112305 nb_pixel_total : 26452 time to create 1 rle with old method : 0.042102813720703125 time for calcul the mask position with numpy : 0.06030440330505371 nb_pixel_total : 2364 time to create 1 rle with old method : 0.002905607223510742 time for calcul the mask position with numpy : 0.05578351020812988 nb_pixel_total : 46909 time to create 1 rle with old method : 0.05247020721435547 time for calcul the mask position with numpy : 0.055092811584472656 nb_pixel_total : 8550 time to create 1 rle with old method : 0.009778976440429688 time for calcul the mask position with numpy : 0.05564403533935547 nb_pixel_total : 43842 time to create 1 rle with old method : 0.04927420616149902 time for calcul the mask position with numpy : 0.07729887962341309 nb_pixel_total : 934668 time to create 1 rle with new method : 0.8113839626312256 time for calcul the mask position with numpy : 0.054996490478515625 nb_pixel_total : 45291 time to create 1 rle with old method : 0.049138784408569336 time for calcul the mask position with numpy : 0.05474090576171875 nb_pixel_total : 33963 time to create 1 rle with old method : 0.038771867752075195 time for calcul the mask position with numpy : 0.05829119682312012 nb_pixel_total : 2557 time to create 1 rle with old method : 0.006325960159301758 time for calcul the mask position with numpy : 0.06575655937194824 nb_pixel_total : 17697 time to create 1 rle with old method : 0.026348114013671875 time for calcul the mask position with numpy : 0.05249309539794922 nb_pixel_total : 414 time to create 1 rle with old method : 0.000701904296875 time for calcul the mask position with numpy : 0.05253028869628906 nb_pixel_total : 27633 time to create 1 rle with old method : 0.02895808219909668 time for calcul the mask position with numpy : 0.05303835868835449 nb_pixel_total : 29628 time to create 1 rle with old method : 0.03221249580383301 time for calcul the mask position with numpy : 0.05429816246032715 nb_pixel_total : 137694 time to create 1 rle with old method : 0.14644122123718262 time for calcul the mask position with numpy : 0.05370616912841797 nb_pixel_total : 25928 time to create 1 rle with old method : 0.0282895565032959 time for calcul the mask position with numpy : 0.05140233039855957 nb_pixel_total : 4 time to create 1 rle with old method : 4.458427429199219e-05 time for calcul the mask position with numpy : 0.05783486366271973 nb_pixel_total : 364667 time to create 1 rle with new method : 0.6412971019744873 time for calcul the mask position with numpy : 0.0522913932800293 nb_pixel_total : 65301 time to create 1 rle with old method : 0.06777453422546387 time for calcul the mask position with numpy : 0.051972150802612305 nb_pixel_total : 1558 time to create 1 rle with old method : 0.001920938491821289 time for calcul the mask position with numpy : 0.05262017250061035 nb_pixel_total : 148005 time to create 1 rle with old method : 0.15472197532653809 time for calcul the mask position with numpy : 0.05248117446899414 nb_pixel_total : 9625 time to create 1 rle with old method : 0.010504722595214844 time for calcul the mask position with numpy : 0.0530238151550293 nb_pixel_total : 120909 time to create 1 rle with old method : 0.16754937171936035 time for calcul the mask position with numpy : 0.050931692123413086 nb_pixel_total : 10993 time to create 1 rle with old method : 0.011855840682983398 time for calcul the mask position with numpy : 0.05136585235595703 nb_pixel_total : 54755 time to create 1 rle with old method : 0.05664324760437012 time for calcul the mask position with numpy : 0.05018949508666992 nb_pixel_total : 805 time to create 1 rle with old method : 0.0009703636169433594 time for calcul the mask position with numpy : 0.05131864547729492 nb_pixel_total : 32201 time to create 1 rle with old method : 0.034801483154296875 time for calcul the mask position with numpy : 0.05287051200866699 nb_pixel_total : 33582 time to create 1 rle with old method : 0.036493539810180664 time for calcul the mask position with numpy : 0.055051565170288086 nb_pixel_total : 424303 time to create 1 rle with new method : 0.597233772277832 time for calcul the mask position with numpy : 0.052756547927856445 nb_pixel_total : 27924 time to create 1 rle with old method : 0.029881954193115234 time for calcul the mask position with numpy : 0.05372476577758789 nb_pixel_total : 136047 time to create 1 rle with old method : 0.17015790939331055 time for calcul the mask position with numpy : 0.05536913871765137 nb_pixel_total : 254643 time to create 1 rle with new method : 0.718325138092041 time for calcul the mask position with numpy : 0.05185198783874512 nb_pixel_total : 9079 time to create 1 rle with old method : 0.01044464111328125 time for calcul the mask position with numpy : 0.05236363410949707 nb_pixel_total : 110380 time to create 1 rle with old method : 0.11255121231079102 time for calcul the mask position with numpy : 0.052968502044677734 nb_pixel_total : 42526 time to create 1 rle with old method : 0.04730367660522461 time for calcul the mask position with numpy : 0.05344343185424805 nb_pixel_total : 41450 time to create 1 rle with old method : 0.04369306564331055 time for calcul the mask position with numpy : 0.053488731384277344 nb_pixel_total : 14358 time to create 1 rle with old method : 0.015755891799926758 time for calcul the mask position with numpy : 0.053969621658325195 nb_pixel_total : 124552 time to create 1 rle with old method : 0.13197565078735352 time for calcul the mask position with numpy : 0.05280256271362305 nb_pixel_total : 125 time to create 1 rle with old method : 0.0003478527069091797 time for calcul the mask position with numpy : 0.05621790885925293 nb_pixel_total : 29464 time to create 1 rle with old method : 0.047387123107910156 time for calcul the mask position with numpy : 0.052906036376953125 nb_pixel_total : 42574 time to create 1 rle with old method : 0.04576230049133301 time for calcul the mask position with numpy : 0.05179190635681152 nb_pixel_total : 44584 time to create 1 rle with old method : 0.04691147804260254 time for calcul the mask position with numpy : 0.05275583267211914 nb_pixel_total : 74063 time to create 1 rle with old method : 0.07921242713928223 time for calcul the mask position with numpy : 0.0523381233215332 nb_pixel_total : 142364 time to create 1 rle with old method : 0.14734649658203125 time for calcul the mask position with numpy : 0.0508732795715332 nb_pixel_total : 28609 time to create 1 rle with old method : 0.02974677085876465 time for calcul the mask position with numpy : 0.05022287368774414 nb_pixel_total : 34949 time to create 1 rle with old method : 0.03573489189147949 time for calcul the mask position with numpy : 0.05190014839172363 nb_pixel_total : 63654 time to create 1 rle with old method : 0.06722903251647949 time for calcul the mask position with numpy : 0.050805091857910156 nb_pixel_total : 38854 time to create 1 rle with old method : 0.03969168663024902 time for calcul the mask position with numpy : 0.05291628837585449 nb_pixel_total : 78498 time to create 1 rle with old method : 0.08224606513977051 time for calcul the mask position with numpy : 0.05131125450134277 nb_pixel_total : 94816 time to create 1 rle with old method : 0.09868788719177246 time for calcul the mask position with numpy : 0.05294156074523926 nb_pixel_total : 14040 time to create 1 rle with old method : 0.014969587326049805 time for calcul the mask position with numpy : 0.05148649215698242 nb_pixel_total : 23790 time to create 1 rle with old method : 0.02516460418701172 time for calcul the mask position with numpy : 0.05356550216674805 nb_pixel_total : 185382 time to create 1 rle with new method : 0.9103200435638428 time for calcul the mask position with numpy : 0.05925107002258301 nb_pixel_total : 126079 time to create 1 rle with old method : 0.1397864818572998 time for calcul the mask position with numpy : 0.05558490753173828 nb_pixel_total : 84545 time to create 1 rle with old method : 0.0974569320678711 time for calcul the mask position with numpy : 0.05762600898742676 nb_pixel_total : 358121 time to create 1 rle with new method : 1.1892399787902832 time for calcul the mask position with numpy : 0.05360007286071777 nb_pixel_total : 15909 time to create 1 rle with old method : 0.017451047897338867 time for calcul the mask position with numpy : 0.05571866035461426 nb_pixel_total : 134016 time to create 1 rle with old method : 0.18956923484802246 time for calcul the mask position with numpy : 0.05634760856628418 nb_pixel_total : 7383 time to create 1 rle with old method : 0.008555412292480469 time for calcul the mask position with numpy : 0.0558929443359375 nb_pixel_total : 172405 time to create 1 rle with new method : 0.6662139892578125 time for calcul the mask position with numpy : 0.05317568778991699 nb_pixel_total : 6993 time to create 1 rle with old method : 0.00811910629272461 time for calcul the mask position with numpy : 0.05984616279602051 nb_pixel_total : 459864 time to create 1 rle with new method : 0.7382962703704834 time for calcul the mask position with numpy : 0.055658817291259766 nb_pixel_total : 313923 time to create 1 rle with new method : 0.8598964214324951 time for calcul the mask position with numpy : 0.05499887466430664 nb_pixel_total : 111275 time to create 1 rle with old method : 0.14762639999389648 time for calcul the mask position with numpy : 0.054647207260131836 nb_pixel_total : 1410 time to create 1 rle with old method : 0.0018284320831298828 time for calcul the mask position with numpy : 0.05993247032165527 nb_pixel_total : 144894 time to create 1 rle with old method : 0.16527986526489258 time for calcul the mask position with numpy : 0.0536496639251709 nb_pixel_total : 29587 time to create 1 rle with old method : 0.032148122787475586 time for calcul the mask position with numpy : 0.05320930480957031 nb_pixel_total : 6306 time to create 1 rle with old method : 0.0071582794189453125 time for calcul the mask position with numpy : 0.052712440490722656 nb_pixel_total : 25094 time to create 1 rle with old method : 0.027951478958129883 time for calcul the mask position with numpy : 0.05304431915283203 nb_pixel_total : 6635 time to create 1 rle with old method : 0.007413625717163086 time for calcul the mask position with numpy : 0.05505180358886719 nb_pixel_total : 362758 time to create 1 rle with new method : 1.1658742427825928 time for calcul the mask position with numpy : 0.05333518981933594 nb_pixel_total : 22708 time to create 1 rle with old method : 0.026501893997192383 time for calcul the mask position with numpy : 0.05485868453979492 nb_pixel_total : 155359 time to create 1 rle with new method : 1.2009789943695068 time for calcul the mask position with numpy : 0.05719757080078125 nb_pixel_total : 42390 time to create 1 rle with old method : 0.04939723014831543 time for calcul the mask position with numpy : 0.054046630859375 nb_pixel_total : 15283 time to create 1 rle with old method : 0.017233610153198242 time for calcul the mask position with numpy : 0.058374643325805664 nb_pixel_total : 32867 time to create 1 rle with old method : 0.04891777038574219 time for calcul the mask position with numpy : 0.05576610565185547 nb_pixel_total : 105 time to create 1 rle with old method : 0.00024962425231933594 create new chi : 20.81524348258972 time to delete rle : 0.015100955963134766 batch 1 Loaded 88 chid ids of type : 4855 Number RLEs to save : 61378 TO DO : save crop sub photo not yet done ! save time : 3.4458703994750977 nb_obj : 54 nb_hashtags : 10 time to prepare the origin masks : 49.777851581573486 time for calcul the mask position with numpy : 0.42214226722717285 nb_pixel_total : 5110165 time to create 1 rle with new method : 1.1771042346954346 time for calcul the mask position with numpy : 0.07649087905883789 nb_pixel_total : 433734 time to create 1 rle with new method : 4.310478925704956 time for calcul the mask position with numpy : 0.05217385292053223 nb_pixel_total : 356 time to create 1 rle with old method : 0.0008056163787841797 time for calcul the mask position with numpy : 0.06098294258117676 nb_pixel_total : 643137 time to create 1 rle with new method : 2.0099284648895264 time for calcul the mask position with numpy : 0.054207801818847656 nb_pixel_total : 67506 time to create 1 rle with old method : 0.075286865234375 time for calcul the mask position with numpy : 0.054419517517089844 nb_pixel_total : 52819 time to create 1 rle with old method : 0.05690574645996094 time for calcul the mask position with numpy : 0.05251002311706543 nb_pixel_total : 164931 time to create 1 rle with new method : 1.33138108253479 time for calcul the mask position with numpy : 0.05339503288269043 nb_pixel_total : 3274 time to create 1 rle with old method : 0.0041124820709228516 time for calcul the mask position with numpy : 0.05699419975280762 nb_pixel_total : 203077 time to create 1 rle with new method : 1.374863862991333 time for calcul the mask position with numpy : 0.05432868003845215 nb_pixel_total : 2647 time to create 1 rle with old method : 0.0034780502319335938 time for calcul the mask position with numpy : 0.05396842956542969 nb_pixel_total : 515 time to create 1 rle with old method : 0.0007822513580322266 time for calcul the mask position with numpy : 0.054612159729003906 nb_pixel_total : 15333 time to create 1 rle with old method : 0.017106056213378906 time for calcul the mask position with numpy : 0.054407596588134766 nb_pixel_total : 12167 time to create 1 rle with old method : 0.013706684112548828 time for calcul the mask position with numpy : 0.05423164367675781 nb_pixel_total : 622 time to create 1 rle with old method : 0.0009210109710693359 time for calcul the mask position with numpy : 0.05573296546936035 nb_pixel_total : 210093 time to create 1 rle with new method : 1.7954354286193848 time for calcul the mask position with numpy : 0.05799365043640137 nb_pixel_total : 165959 time to create 1 rle with new method : 1.3053483963012695 time for calcul the mask position with numpy : 0.05321669578552246 nb_pixel_total : 28362 time to create 1 rle with old method : 0.030747413635253906 time for calcul the mask position with numpy : 0.054472923278808594 nb_pixel_total : 206419 time to create 1 rle with new method : 1.1568152904510498 time for calcul the mask position with numpy : 0.055338382720947266 nb_pixel_total : 136725 time to create 1 rle with old method : 0.1513688564300537 time for calcul the mask position with numpy : 0.06024670600891113 nb_pixel_total : 415393 time to create 1 rle with new method : 1.0962610244750977 time for calcul the mask position with numpy : 0.05471062660217285 nb_pixel_total : 29114 time to create 1 rle with old method : 0.032807350158691406 time for calcul the mask position with numpy : 0.05521059036254883 nb_pixel_total : 10619 time to create 1 rle with old method : 0.012408733367919922 time for calcul the mask position with numpy : 0.05623912811279297 nb_pixel_total : 151804 time to create 1 rle with new method : 2.1864399909973145 time for calcul the mask position with numpy : 0.05155348777770996 nb_pixel_total : 29856 time to create 1 rle with old method : 0.031646728515625 time for calcul the mask position with numpy : 0.05208849906921387 nb_pixel_total : 121394 time to create 1 rle with old method : 0.13902926445007324 time for calcul the mask position with numpy : 0.05763816833496094 nb_pixel_total : 398830 time to create 1 rle with new method : 1.025024652481079 time for calcul the mask position with numpy : 0.051529645919799805 nb_pixel_total : 45315 time to create 1 rle with old method : 0.04912400245666504 time for calcul the mask position with numpy : 0.06679558753967285 nb_pixel_total : 752808 time to create 1 rle with new method : 1.6843726634979248 time for calcul the mask position with numpy : 0.05225038528442383 nb_pixel_total : 1337 time to create 1 rle with old method : 0.0015213489532470703 time for calcul the mask position with numpy : 0.05498528480529785 nb_pixel_total : 88023 time to create 1 rle with old method : 0.09491753578186035 time for calcul the mask position with numpy : 0.052358150482177734 nb_pixel_total : 31819 time to create 1 rle with old method : 0.033728837966918945 time for calcul the mask position with numpy : 0.05199766159057617 nb_pixel_total : 25403 time to create 1 rle with old method : 0.02666330337524414 time for calcul the mask position with numpy : 0.05207538604736328 nb_pixel_total : 58135 time to create 1 rle with old method : 0.06350255012512207 time for calcul the mask position with numpy : 0.05157971382141113 nb_pixel_total : 2335 time to create 1 rle with old method : 0.0028264522552490234 time for calcul the mask position with numpy : 0.05405855178833008 nb_pixel_total : 345368 time to create 1 rle with new method : 1.1279504299163818 time for calcul the mask position with numpy : 0.0557103157043457 nb_pixel_total : 240866 time to create 1 rle with new method : 1.0211467742919922 time for calcul the mask position with numpy : 0.051514387130737305 nb_pixel_total : 19790 time to create 1 rle with old method : 0.021748781204223633 time for calcul the mask position with numpy : 0.05294322967529297 nb_pixel_total : 85728 time to create 1 rle with old method : 0.09290003776550293 time for calcul the mask position with numpy : 0.05437922477722168 nb_pixel_total : 156767 time to create 1 rle with new method : 1.960038661956787 time for calcul the mask position with numpy : 0.052437782287597656 nb_pixel_total : 26938 time to create 1 rle with old method : 0.02920246124267578 time for calcul the mask position with numpy : 0.05298423767089844 nb_pixel_total : 35726 time to create 1 rle with old method : 0.039481163024902344 time for calcul the mask position with numpy : 0.05227303504943848 nb_pixel_total : 65564 time to create 1 rle with old method : 0.0719137191772461 time for calcul the mask position with numpy : 0.05321145057678223 nb_pixel_total : 139938 time to create 1 rle with old method : 0.15525269508361816 time for calcul the mask position with numpy : 0.05701470375061035 nb_pixel_total : 306017 time to create 1 rle with new method : 1.0866317749023438 time for calcul the mask position with numpy : 0.05273008346557617 nb_pixel_total : 38177 time to create 1 rle with old method : 0.03957414627075195 time for calcul the mask position with numpy : 0.053136348724365234 nb_pixel_total : 216605 time to create 1 rle with new method : 1.3424043655395508 time for calcul the mask position with numpy : 0.05300402641296387 nb_pixel_total : 35619 time to create 1 rle with old method : 0.03899121284484863 time for calcul the mask position with numpy : 0.05405402183532715 nb_pixel_total : 265556 time to create 1 rle with new method : 0.8178436756134033 time for calcul the mask position with numpy : 0.05556607246398926 nb_pixel_total : 47599 time to create 1 rle with old method : 0.0787196159362793 time for calcul the mask position with numpy : 0.05779314041137695 nb_pixel_total : 421173 time to create 1 rle with new method : 1.2332539558410645 time for calcul the mask position with numpy : 0.060463905334472656 nb_pixel_total : 747959 time to create 1 rle with new method : 1.0951800346374512 time for calcul the mask position with numpy : 0.05070137977600098 nb_pixel_total : 99969 time to create 1 rle with old method : 0.10460138320922852 time for calcul the mask position with numpy : 0.05091238021850586 nb_pixel_total : 32710 time to create 1 rle with old method : 0.03404951095581055 time for calcul the mask position with numpy : 0.0505826473236084 nb_pixel_total : 28396 time to create 1 rle with old method : 0.04824519157409668 time for calcul the mask position with numpy : 0.05017542839050293 nb_pixel_total : 2709 time to create 1 rle with old method : 0.0028352737426757812 create new chi : 36.188398361206055 time to delete rle : 0.011973142623901367 batch 1 Loaded 55 chid ids of type : 4855 Number RLEs to save : 47507 TO DO : save crop sub photo not yet done ! save time : 2.6782543659210205 nb_obj : 9 nb_hashtags : 2 time to prepare the origin masks : 18.219478607177734 time for calcul the mask position with numpy : 0.25818943977355957 nb_pixel_total : 1301544 time to create 1 rle with new method : 1.3932247161865234 time for calcul the mask position with numpy : 0.0676276683807373 nb_pixel_total : 49750 time to create 1 rle with old method : 0.05701804161071777 time for calcul the mask position with numpy : 0.03960061073303223 nb_pixel_total : 77375 time to create 1 rle with old method : 0.09143757820129395 time for calcul the mask position with numpy : 0.05573892593383789 nb_pixel_total : 1622138 time to create 1 rle with new method : 1.6024954319000244 time for calcul the mask position with numpy : 0.06470918655395508 nb_pixel_total : 268579 time to create 1 rle with new method : 1.2721374034881592 time for calcul the mask position with numpy : 0.07235980033874512 nb_pixel_total : 940365 time to create 1 rle with new method : 1.5631539821624756 time for calcul the mask position with numpy : 0.07037115097045898 nb_pixel_total : 972624 time to create 1 rle with new method : 1.6472148895263672 time for calcul the mask position with numpy : 0.06732773780822754 nb_pixel_total : 783305 time to create 1 rle with new method : 1.2121391296386719 time for calcul the mask position with numpy : 0.07639884948730469 nb_pixel_total : 741935 time to create 1 rle with new method : 1.480114459991455 time for calcul the mask position with numpy : 0.4933593273162842 nb_pixel_total : 6221585 time to create 1 rle with new method : 1.231083631515503 create new chi : 13.187175512313843 time to delete rle : 0.0036399364471435547 batch 1 Loaded 10 chid ids of type : 4855 Number RLEs to save : 30393 TO DO : save crop sub photo not yet done ! save time : 1.758683204650879 map_output_result : {1356446131: (0.7055205351543858, 'Should be the crop_list due to order', 0.708612584748593), 1356446129: (0.7055205351543858, 'Should be the crop_list due to order', 0.6124945365457665), 1356446125: (0.7055205351543858, 'Should be the crop_list due to order', 0.500975019323184), 1356446122: (0.7055205351543858, 'Should be the crop_list due to order', 1.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 4 /1356446131.Didn't retrieve data . /1356446129.Didn't retrieve data . /1356446125.Didn't retrieve data . /1356446122.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.012394428253173828 save_final save missing photos in datou_result : time spend for datou_step_exec : 293.70734906196594 time spend to save output : 0.01764392852783203 total time spend for step 5 : 293.7249929904938 step6:crop_condition Tue May 6 22:09:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 4 ! batch 1 Loaded 214 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 3015255 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562164_2219837 we have uploaded 12 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.5691745281219482 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 41 About to insert : list_path_to_insert length 41 new photo from crops ! About to upload 41 photos upload in portfolio : 3015255 init cache_photo without model_param we have 41 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562192_2219837 we have uploaded 41 photos in the portfolio 3015255 time of upload the photos Elapsed time : 19.478360176086426 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3015255 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562217_2219837 we have uploaded 14 photos in the portfolio 3015255 time of upload the photos Elapsed time : 4.811570167541504 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 3015255 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562224_2219837 we have uploaded 5 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.6209204196929932 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562229_2219837 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.896467924118042 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 3015255 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562237_2219837 we have uploaded 7 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.247586488723755 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 37 About to insert : list_path_to_insert length 37 new photo from crops ! About to upload 37 photos upload in portfolio : 3015255 init cache_photo without model_param we have 37 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562296_2219837 we have uploaded 37 photos in the portfolio 3015255 time of upload the photos Elapsed time : 13.405455112457275 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562312_2219837 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.7778983116149902 we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 3015255 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562317_2219837 we have uploaded 8 photos in the portfolio 3015255 time of upload the photos Elapsed time : 2.4430480003356934 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 25 About to insert : list_path_to_insert length 25 new photo from crops ! About to upload 25 photos upload in portfolio : 3015255 init cache_photo without model_param we have 25 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746562430_2219837 we have uploaded 25 photos in the portfolio 3015255 time of upload the photos Elapsed time : 9.236934661865234 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 153 /1356461810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356461831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462641Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356462763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356463478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464579Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464588Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464604Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356464623Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 463 time used for this insertion : 0.04876542091369629 save_final save missing photos in datou_result : time spend for datou_step_exec : 290.3148503303528 time spend to save output : 0.1161642074584961 total time spend for step 6 : 290.4310145378113 step7:ventilate_hashtags_in_portfolio Tue May 6 22:14:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22731303 get user id for portfolio 22731303 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731303 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731303 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731303 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731303 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22734147,22734148,22734150,22734151,22734152,22734158,22734161,22734162,22734163,22734165,22734166,22734167,22734170,22734171,22734172,22734174,22734175,22734176,22734177,22734179?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22731303 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 1 /22731303. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.013661384582519531 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.244617223739624 time spend to save output : 0.013880729675292969 total time spend for step 7 : 4.258497953414917 step8:final Tue May 6 22:14:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356446131: ('0.6386387749089113',), 1356446129: ('0.6386387749089113',), 1356446125: ('0.6386387749089113',), 1356446122: ('0.6386387749089113',)} new output for save of step final : {1356446131: ('0.6386387749089113',), 1356446129: ('0.6386387749089113',), 1356446125: ('0.6386387749089113',), 1356446122: ('0.6386387749089113',)} [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 4 /1356446131.Didn't retrieve data . /1356446129.Didn't retrieve data . /1356446125.Didn't retrieve data . /1356446122.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 12 time used for this insertion : 0.011819124221801758 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11831521987915039 time spend to save output : 0.012500286102294922 total time spend for step 8 : 0.1308155059814453 step9:velours_tree Tue May 6 22:14:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.8529562950134277 time spend to save output : 4.9114227294921875e-05 total time spend for step 9 : 1.8530054092407227 step10:send_mail_cod Tue May 6 22:14:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22731303_06-05-2025_22_14_07.pdf 22734117 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341171746562447 22734118 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341181746562448 22734119 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341191746562450 22734120 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341201746562451 22734121 change filename to text .change filename to text .imagette227341211746562451 22734122 imagette227341221746562452 22734123 imagette227341231746562452 22734124 imagette227341241746562452 22734125 imagette227341251746562452 22734126 imagette227341261746562452 22734127 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341271746562452 22734128 imagette227341281746562452 22734129 imagette227341291746562452 22734130 imagette227341301746562452 22734131 imagette227341311746562452 22734132 imagette227341321746562452 22734133 imagette227341331746562452 22734134 change filename to text .change filename to text .imagette227341341746562452 22734135 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341351746562452 22734137 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227341371746562453 22734138 imagette227341381746562455 22734139 imagette227341391746562455 22734140 imagette227341401746562455 22734141 imagette227341411746562455 22734142 imagette227341421746562455 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22731303 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22734147,22734148,22734150,22734151,22734152,22734158,22734161,22734162,22734163,22734165,22734166,22734167,22734170,22734171,22734172,22734174,22734175,22734176,22734177,22734179?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22731303 your option no_mail is active, we will not send the real mail to your client args[1356446131] : ((1356446131, 820.327219447486, 2107751945), (1356446131, -0.3230384088625607, 496442774), '0.6386387749089113') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446129] : ((1356446129, 2341.3283598206795, 492609224), (1356446129, -0.32245153695990864, 496442774), '0.6386387749089113') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446125] : ((1356446125, 1034.7216852982658, 2107751945), (1356446125, -0.34435336854008675, 496442774), '0.6386387749089113') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356446122] : ((1356446122, 130.86716434383777, 492688767), (1356446122, -0.8756705086045145, 501862349), '0.6386387749089113') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.6386387749089113 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731303_06-05-2025_22_14_07.pdf results_Auto_P22731303_06-05-2025_22_14_07.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731303_06-05-2025_22_14_07.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22731303','results_Auto_P22731303_06-05-2025_22_14_07.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731303_06-05-2025_22_14_07.pdf','pdf','','1.94','0.6386387749089113') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.01257467269897461 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.534398078918457 time spend to save output : 0.012848138809204102 total time spend for step 10 : 10.547246217727661 step11:split_time_score Tue May 6 22:14:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('46', 1), ('47', 3)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06052025 22731303 Nombre de photos uploadées : 4 / 23040 (0%) 06052025 22731303 Nombre de photos taguées (types de déchets): 0 / 4 (0%) 06052025 22731303 Nombre de photos taguées (volume) : 0 / 4 (0%) elapsed_time : load_data_split_time_score 6.4373016357421875e-06 elapsed_time : order_list_meta_photo_and_scores 1.3589859008789062e-05 ???? elapsed_time : fill_and_build_computed_from_old_data 0.0003914833068847656 elapsed_time : insert_dashboard_record_day_entry 0.022523164749145508 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731300 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731301 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731302 order by id desc limit 1 Qualite : 0.6386387749089113 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731303_06-05-2025_22_14_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731303 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731303 AND mptpi.`type`=4855 To do Qualite : 0.5753207720446533 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22731304_06-05-2025_21_53_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22731304 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22731304 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06052025': {'nb_upload': 4, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356446131, 1356446129, 1356446125, 1356446122] Looping around the photos to save general results len do output : 1 /22731303Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446131', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446129', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446125', None, None, None, None, None, '2799188') ('4741', None, None, None, None, None, None, None, '2799188') ('4741', '22731303', '1356446122', None, None, None, None, None, '2799188') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.012651920318603516 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.82303786277771 time spend to save output : 0.012901067733764648 total time spend for step 11 : 2.8359389305114746 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 4 set_done_treatment 466.78user 249.57system 12:55.93elapsed 92%CPU (0avgtext+0avgdata 6647348maxresident)k 1413496inputs+504520outputs (24571major+36315349minor)pagefaults 0swaps