python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 652770 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['2797355'] with mtr_portfolio_ids : ['22713613'] and first list_photo_ids : [] new path : /proc/652770/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 5 ; length of list_pids : 5 ; length of list_args : 5 time to download the photos : 1.1889293193817139 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue May 6 12:01:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 281 wait 20 seconds l 3637 free memory gpu now : 281 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 12:01:56.743765: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 12:01:56.799167: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 12:01:56.801083: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f55e8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 12:01:56.801129: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 12:01:56.813811: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 12:01:57.282714: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x250f08a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 12:01:57.282793: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 12:01:57.284313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:01:57.286624: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:01:57.317238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:01:57.347108: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:01:57.351577: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:01:57.386535: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:01:57.390328: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:01:57.446658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:01:57.447820: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:01:57.448225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:01:57.449151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 12:01:57.449168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 12:01:57.449177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 12:01:57.450953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2186 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 12:01:57.937372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:01:57.937539: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:01:57.937589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:01:57.937633: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:01:57.937675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:01:57.937718: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:01:57.937759: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:01:57.937802: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:01:57.939584: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:01:57.941057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:01:57.941180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:01:57.941219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:01:57.941275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:01:57.941306: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:01:57.941334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:01:57.941360: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:01:57.941386: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:01:57.942607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:01:57.942648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 12:01:57.942664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 12:01:57.942677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 12:01:57.943967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2186 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 12:02:04.730145: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 51380224 exceeds 10% of free system memory. 2025-05-06 12:02:12.907236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:02:13.321739: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:02:16.156274: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.156692: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.163254: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.163281: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.217359: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.217412: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.260571: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.260617: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.313473: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.313527: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:02:16.315933: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.14G (1220411392 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.315965: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-06 12:02:16.336645: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.337757: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.370247: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.370874: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.372651: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.373235: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.380036: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.380628: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.382563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.383181: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.390321: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.390942: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.392690: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.393277: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.422643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.423588: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.424486: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.425379: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.429566: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.430490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.447221: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.448079: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.448905: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.449719: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.462758: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.463718: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.464613: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.465503: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.470336: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.471240: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.476480: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.477450: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.489750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.490702: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.494924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.495530: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.496358: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.496924: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.507746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.508337: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.508918: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.509477: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.510038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.510596: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.550856: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.550934: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-06 12:02:16.551532: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.552115: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.559683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.560294: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.568447: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.569036: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.584780: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.585527: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.586233: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.586938: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.591179: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.591764: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.592323: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.592892: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.594595: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.604553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.605149: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.614691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.615552: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.616325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.616950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.617593: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:02:16.618253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.64G (1757282304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 4160.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 653823 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1445 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3831 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.0017583370208740234 nb_pixel_total : 65580 time to create 1 rle with old method : 0.07307147979736328 length of segment : 294 time for calcul the mask position with numpy : 0.004347324371337891 nb_pixel_total : 166791 time to create 1 rle with new method : 0.011362791061401367 length of segment : 406 time for calcul the mask position with numpy : 0.23464012145996094 nb_pixel_total : 1118196 time to create 1 rle with new method : 0.1629784107208252 length of segment : 1313 time for calcul the mask position with numpy : 0.014961719512939453 nb_pixel_total : 816561 time to create 1 rle with new method : 0.09449958801269531 length of segment : 1043 time for calcul the mask position with numpy : 0.0005648136138916016 nb_pixel_total : 24319 time to create 1 rle with old method : 0.027629852294921875 length of segment : 229 time for calcul the mask position with numpy : 0.00029349327087402344 nb_pixel_total : 287 time to create 1 rle with old method : 0.00074005126953125 length of segment : 24 time for calcul the mask position with numpy : 0.0003972053527832031 nb_pixel_total : 22216 time to create 1 rle with old method : 0.025661230087280273 length of segment : 218 time for calcul the mask position with numpy : 0.00022983551025390625 nb_pixel_total : 13500 time to create 1 rle with old method : 0.01575613021850586 length of segment : 129 time for calcul the mask position with numpy : 0.00012922286987304688 nb_pixel_total : 6164 time to create 1 rle with old method : 0.007338762283325195 length of segment : 84 time for calcul the mask position with numpy : 0.0003314018249511719 nb_pixel_total : 20476 time to create 1 rle with old method : 0.02482771873474121 length of segment : 134 time for calcul the mask position with numpy : 0.004686832427978516 nb_pixel_total : 304673 time to create 1 rle with new method : 0.020437002182006836 length of segment : 560 time for calcul the mask position with numpy : 0.009395837783813477 nb_pixel_total : 266817 time to create 1 rle with new method : 0.012217998504638672 length of segment : 473 time for calcul the mask position with numpy : 0.006096363067626953 nb_pixel_total : 388035 time to create 1 rle with new method : 0.015465497970581055 length of segment : 714 time for calcul the mask position with numpy : 0.017735004425048828 nb_pixel_total : 430641 time to create 1 rle with new method : 0.018661022186279297 length of segment : 445 time for calcul the mask position with numpy : 0.004345417022705078 nb_pixel_total : 185429 time to create 1 rle with new method : 0.005885124206542969 length of segment : 425 time for calcul the mask position with numpy : 0.005613803863525391 nb_pixel_total : 179222 time to create 1 rle with new method : 0.008777141571044922 length of segment : 654 time for calcul the mask position with numpy : 0.001622915267944336 nb_pixel_total : 40474 time to create 1 rle with old method : 0.046442508697509766 length of segment : 329 time for calcul the mask position with numpy : 0.017758846282958984 nb_pixel_total : 472151 time to create 1 rle with new method : 0.07413768768310547 length of segment : 806 time for calcul the mask position with numpy : 0.0020821094512939453 nb_pixel_total : 55935 time to create 1 rle with old method : 0.0727682113647461 length of segment : 319 time for calcul the mask position with numpy : 0.007299184799194336 nb_pixel_total : 205315 time to create 1 rle with new method : 0.013521194458007812 length of segment : 864 time for calcul the mask position with numpy : 0.0061664581298828125 nb_pixel_total : 241144 time to create 1 rle with new method : 0.011159181594848633 length of segment : 493 time for calcul the mask position with numpy : 0.009397029876708984 nb_pixel_total : 389794 time to create 1 rle with new method : 0.01410675048828125 length of segment : 739 time for calcul the mask position with numpy : 0.002475261688232422 nb_pixel_total : 47481 time to create 1 rle with old method : 0.05722236633300781 length of segment : 341 time for calcul the mask position with numpy : 0.0025415420532226562 nb_pixel_total : 139167 time to create 1 rle with old method : 0.15603971481323242 length of segment : 506 time for calcul the mask position with numpy : 0.0019903182983398438 nb_pixel_total : 54343 time to create 1 rle with old method : 0.06635403633117676 length of segment : 293 time for calcul the mask position with numpy : 0.0009417533874511719 nb_pixel_total : 16753 time to create 1 rle with old method : 0.019429445266723633 length of segment : 157 time for calcul the mask position with numpy : 0.0014505386352539062 nb_pixel_total : 38819 time to create 1 rle with old method : 0.06317758560180664 length of segment : 198 time for calcul the mask position with numpy : 0.005406856536865234 nb_pixel_total : 208784 time to create 1 rle with new method : 0.01016998291015625 length of segment : 379 time for calcul the mask position with numpy : 0.0047740936279296875 nb_pixel_total : 200300 time to create 1 rle with new method : 0.008983850479125977 length of segment : 552 time for calcul the mask position with numpy : 0.00112152099609375 nb_pixel_total : 35602 time to create 1 rle with old method : 0.04001760482788086 length of segment : 258 time for calcul the mask position with numpy : 0.0008451938629150391 nb_pixel_total : 21609 time to create 1 rle with old method : 0.026567935943603516 length of segment : 159 time for calcul the mask position with numpy : 0.0011322498321533203 nb_pixel_total : 39826 time to create 1 rle with old method : 0.04661417007446289 length of segment : 250 time for calcul the mask position with numpy : 0.002881288528442383 nb_pixel_total : 76151 time to create 1 rle with old method : 0.08791208267211914 length of segment : 224 time for calcul the mask position with numpy : 0.002391338348388672 nb_pixel_total : 77114 time to create 1 rle with old method : 0.08971500396728516 length of segment : 308 time for calcul the mask position with numpy : 0.007900476455688477 nb_pixel_total : 326335 time to create 1 rle with new method : 0.014641523361206055 length of segment : 576 time for calcul the mask position with numpy : 0.001672506332397461 nb_pixel_total : 61476 time to create 1 rle with old method : 0.06962728500366211 length of segment : 214 time for calcul the mask position with numpy : 0.002267599105834961 nb_pixel_total : 65326 time to create 1 rle with old method : 0.07828998565673828 length of segment : 324 time for calcul the mask position with numpy : 0.029477357864379883 nb_pixel_total : 843110 time to create 1 rle with new method : 0.3265659809112549 length of segment : 985 time for calcul the mask position with numpy : 0.0030150413513183594 nb_pixel_total : 112118 time to create 1 rle with old method : 0.12641143798828125 length of segment : 376 time for calcul the mask position with numpy : 0.0018656253814697266 nb_pixel_total : 83649 time to create 1 rle with old method : 0.09317326545715332 length of segment : 446 time for calcul the mask position with numpy : 0.007439136505126953 nb_pixel_total : 176047 time to create 1 rle with new method : 0.012174606323242188 length of segment : 420 time for calcul the mask position with numpy : 0.0008454322814941406 nb_pixel_total : 17708 time to create 1 rle with old method : 0.020242691040039062 length of segment : 219 time for calcul the mask position with numpy : 0.24629545211791992 nb_pixel_total : 1860193 time to create 1 rle with new method : 0.2023458480834961 length of segment : 1436 time for calcul the mask position with numpy : 0.0032935142517089844 nb_pixel_total : 103406 time to create 1 rle with old method : 0.12302637100219727 length of segment : 376 time for calcul the mask position with numpy : 0.0010044574737548828 nb_pixel_total : 25062 time to create 1 rle with old method : 0.02894902229309082 length of segment : 219 time for calcul the mask position with numpy : 0.0007431507110595703 nb_pixel_total : 15966 time to create 1 rle with old method : 0.018870830535888672 length of segment : 129 time for calcul the mask position with numpy : 0.00096893310546875 nb_pixel_total : 41924 time to create 1 rle with old method : 0.05161452293395996 length of segment : 109 time for calcul the mask position with numpy : 0.0006103515625 nb_pixel_total : 24637 time to create 1 rle with old method : 0.028335094451904297 length of segment : 238 time for calcul the mask position with numpy : 0.006529569625854492 nb_pixel_total : 146482 time to create 1 rle with old method : 0.16365981101989746 length of segment : 389 time for calcul the mask position with numpy : 0.0007345676422119141 nb_pixel_total : 12559 time to create 1 rle with old method : 0.02094721794128418 length of segment : 132 time for calcul the mask position with numpy : 0.0002884864807128906 nb_pixel_total : 3401 time to create 1 rle with old method : 0.004436492919921875 length of segment : 81 time for calcul the mask position with numpy : 0.00043654441833496094 nb_pixel_total : 5837 time to create 1 rle with old method : 0.0071332454681396484 length of segment : 136 time for calcul the mask position with numpy : 0.007237911224365234 nb_pixel_total : 217071 time to create 1 rle with new method : 0.011064529418945312 length of segment : 646 time for calcul the mask position with numpy : 0.00045299530029296875 nb_pixel_total : 12077 time to create 1 rle with old method : 0.014012813568115234 length of segment : 137 time for calcul the mask position with numpy : 0.0018112659454345703 nb_pixel_total : 67954 time to create 1 rle with old method : 0.10393095016479492 length of segment : 326 time for calcul the mask position with numpy : 0.0013070106506347656 nb_pixel_total : 41758 time to create 1 rle with old method : 0.046683311462402344 length of segment : 258 time for calcul the mask position with numpy : 0.0007653236389160156 nb_pixel_total : 15190 time to create 1 rle with old method : 0.017664194107055664 length of segment : 186 time for calcul the mask position with numpy : 0.00021028518676757812 nb_pixel_total : 5530 time to create 1 rle with old method : 0.006829023361206055 length of segment : 125 time for calcul the mask position with numpy : 0.002261638641357422 nb_pixel_total : 132384 time to create 1 rle with old method : 0.15075135231018066 length of segment : 395 time for calcul the mask position with numpy : 0.5364871025085449 nb_pixel_total : 3746400 time to create 1 rle with new method : 0.6859290599822998 length of segment : 3093 time for calcul the mask position with numpy : 0.009485483169555664 nb_pixel_total : 357607 time to create 1 rle with new method : 0.02161264419555664 length of segment : 1409 time for calcul the mask position with numpy : 0.20404481887817383 nb_pixel_total : 3020020 time to create 1 rle with new method : 0.7574343681335449 length of segment : 3495 time for calcul the mask position with numpy : 0.3773646354675293 nb_pixel_total : 3884981 time to create 1 rle with new method : 1.2558507919311523 length of segment : 3442 time for calcul the mask position with numpy : 0.9743866920471191 nb_pixel_total : 5582610 time to create 1 rle with new method : 1.2278590202331543 length of segment : 4011 time for calcul the mask position with numpy : 0.39475107192993164 nb_pixel_total : 4350957 time to create 1 rle with new method : 0.8985230922698975 length of segment : 5104 time for calcul the mask position with numpy : 0.005781650543212891 nb_pixel_total : 358543 time to create 1 rle with new method : 0.011664628982543945 length of segment : 328 time for calcul the mask position with numpy : 0.02835869789123535 nb_pixel_total : 2418619 time to create 1 rle with new method : 0.1654832363128662 length of segment : 1007 time spent for convertir_results : 29.435933828353882 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 67 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 110.07182621955872 time spend to save output : 0.10492372512817383 total time spend for step 1 : 110.17674994468689 step2:mask_detect Tue May 6 12:03:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3831 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-05-06 12:03:25.519736: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-05-06 12:03:25.555182: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-05-06 12:03:25.557473: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f55e4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-05-06 12:03:25.557533: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-05-06 12:03:25.563359: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-05-06 12:03:25.855430: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x25fd0f60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-05-06 12:03:25.855487: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-05-06 12:03:25.856865: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:03:25.859342: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:03:25.893513: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:03:25.913673: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:03:25.917935: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:03:25.951560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:03:25.956895: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:03:26.015526: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:03:26.016743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:03:26.017176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:03:26.018215: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 12:03:26.018235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 12:03:26.018244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 12:03:26.019926: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-05-06 12:03:26.207730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:03:26.207884: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:03:26.207915: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:03:26.207942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:03:26.207969: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:03:26.207995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:03:26.208021: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:03:26.208048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:03:26.209623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:03:26.211109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-05-06 12:03:26.211165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-05-06 12:03:26.211194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:03:26.211225: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-05-06 12:03:26.211252: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-05-06 12:03:26.211277: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-05-06 12:03:26.211303: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-05-06 12:03:26.211330: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:03:26.212813: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-05-06 12:03:26.212853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-05-06 12:03:26.212868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-05-06 12:03:26.212881: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-05-06 12:03:26.214433: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3379 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-05-06 12:03:39.583915: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-05-06 12:03:40.020625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-05-06 12:03:41.798175: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.526933: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-05-06 12:03:42.586531: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.586681: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.588081: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.588126: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.596077: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.596166: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.602894: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.602987: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.603644: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.603661: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.667972: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.668057: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.669191: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.669213: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.678804: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.678871: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.679611: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.679641: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 417.62MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-05-06 12:03:42.701733: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.702563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.703348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.704150: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.708815: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.709585: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.710361: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.711188: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.713390: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.713423: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-05-06 12:03:42.723167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.723960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.734349: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.735203: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.736001: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.736772: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.737583: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-05-06 12:03:42.738360: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.80G (3008233472 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 5 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 29 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 1.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 49 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 45 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 5.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (3120, 4160, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 4160.00000 nb d'objets trouves : 7 Detection mask done ! Trying to reset tf kernel 659073 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2638 tf kernel not reseted sub process len(results) : 5 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 5 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3831 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0022656917572021484 nb_pixel_total : 54304 time to create 1 rle with old method : 0.06483864784240723 length of segment : 422 time for calcul the mask position with numpy : 0.001699686050415039 nb_pixel_total : 109581 time to create 1 rle with old method : 0.13781118392944336 length of segment : 368 time for calcul the mask position with numpy : 0.018710851669311523 nb_pixel_total : 741736 time to create 1 rle with new method : 0.07047152519226074 length of segment : 965 time for calcul the mask position with numpy : 0.0036907196044921875 nb_pixel_total : 190450 time to create 1 rle with new method : 0.008111953735351562 length of segment : 451 time for calcul the mask position with numpy : 0.0033800601959228516 nb_pixel_total : 166304 time to create 1 rle with new method : 0.009884357452392578 length of segment : 529 time for calcul the mask position with numpy : 0.4101572036743164 nb_pixel_total : 2563139 time to create 1 rle with new method : 0.5679099559783936 length of segment : 1026 time for calcul the mask position with numpy : 0.002150297164916992 nb_pixel_total : 133892 time to create 1 rle with old method : 0.15362882614135742 length of segment : 485 time for calcul the mask position with numpy : 0.0009632110595703125 nb_pixel_total : 42133 time to create 1 rle with old method : 0.04894542694091797 length of segment : 255 time for calcul the mask position with numpy : 0.0004150867462158203 nb_pixel_total : 18811 time to create 1 rle with old method : 0.021889925003051758 length of segment : 370 time for calcul the mask position with numpy : 0.0031816959381103516 nb_pixel_total : 181159 time to create 1 rle with new method : 0.008471488952636719 length of segment : 766 time for calcul the mask position with numpy : 0.0028688907623291016 nb_pixel_total : 178323 time to create 1 rle with new method : 0.007877588272094727 length of segment : 663 time for calcul the mask position with numpy : 0.012745141983032227 nb_pixel_total : 550081 time to create 1 rle with new method : 0.07355165481567383 length of segment : 900 time for calcul the mask position with numpy : 0.03751373291015625 nb_pixel_total : 2345709 time to create 1 rle with new method : 0.4727182388305664 length of segment : 1486 time for calcul the mask position with numpy : 0.0011332035064697266 nb_pixel_total : 49345 time to create 1 rle with old method : 0.05745744705200195 length of segment : 288 time for calcul the mask position with numpy : 0.001298666000366211 nb_pixel_total : 88762 time to create 1 rle with old method : 0.09992671012878418 length of segment : 201 time for calcul the mask position with numpy : 0.0020394325256347656 nb_pixel_total : 142354 time to create 1 rle with old method : 0.18483567237854004 length of segment : 407 time for calcul the mask position with numpy : 0.0004177093505859375 nb_pixel_total : 20087 time to create 1 rle with old method : 0.02301621437072754 length of segment : 225 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 5579 time to create 1 rle with old method : 0.006591796875 length of segment : 197 time for calcul the mask position with numpy : 0.0041921138763427734 nb_pixel_total : 271503 time to create 1 rle with new method : 0.011630773544311523 length of segment : 590 time for calcul the mask position with numpy : 0.002489328384399414 nb_pixel_total : 66777 time to create 1 rle with old method : 0.08336257934570312 length of segment : 390 time for calcul the mask position with numpy : 0.000986337661743164 nb_pixel_total : 66476 time to create 1 rle with old method : 0.0759575366973877 length of segment : 355 time for calcul the mask position with numpy : 0.0018346309661865234 nb_pixel_total : 96067 time to create 1 rle with old method : 0.11556577682495117 length of segment : 392 time for calcul the mask position with numpy : 0.005489349365234375 nb_pixel_total : 372075 time to create 1 rle with new method : 0.014751434326171875 length of segment : 751 time for calcul the mask position with numpy : 0.0008261203765869141 nb_pixel_total : 46039 time to create 1 rle with old method : 0.05234527587890625 length of segment : 257 time for calcul the mask position with numpy : 0.009404420852661133 nb_pixel_total : 591499 time to create 1 rle with new method : 0.02176976203918457 length of segment : 956 time for calcul the mask position with numpy : 0.000865936279296875 nb_pixel_total : 46643 time to create 1 rle with old method : 0.05271148681640625 length of segment : 227 time for calcul the mask position with numpy : 0.0006895065307617188 nb_pixel_total : 40661 time to create 1 rle with old method : 0.046462059020996094 length of segment : 224 time for calcul the mask position with numpy : 0.0005371570587158203 nb_pixel_total : 32370 time to create 1 rle with old method : 0.03746438026428223 length of segment : 153 time for calcul the mask position with numpy : 0.0005309581756591797 nb_pixel_total : 33101 time to create 1 rle with old method : 0.0380246639251709 length of segment : 157 time for calcul the mask position with numpy : 0.0040781497955322266 nb_pixel_total : 292286 time to create 1 rle with new method : 0.010538578033447266 length of segment : 444 time for calcul the mask position with numpy : 0.003293275833129883 nb_pixel_total : 190747 time to create 1 rle with new method : 0.008100748062133789 length of segment : 347 time for calcul the mask position with numpy : 0.004093170166015625 nb_pixel_total : 209391 time to create 1 rle with new method : 0.010762453079223633 length of segment : 519 time for calcul the mask position with numpy : 0.0009942054748535156 nb_pixel_total : 39632 time to create 1 rle with old method : 0.04522299766540527 length of segment : 240 time for calcul the mask position with numpy : 0.0044667720794677734 nb_pixel_total : 218710 time to create 1 rle with new method : 0.011749982833862305 length of segment : 764 time for calcul the mask position with numpy : 0.0010273456573486328 nb_pixel_total : 39395 time to create 1 rle with old method : 0.04499006271362305 length of segment : 272 time for calcul the mask position with numpy : 0.0008287429809570312 nb_pixel_total : 26340 time to create 1 rle with old method : 0.030500173568725586 length of segment : 187 time for calcul the mask position with numpy : 0.011069297790527344 nb_pixel_total : 654309 time to create 1 rle with new method : 0.02352619171142578 length of segment : 764 time for calcul the mask position with numpy : 0.01871800422668457 nb_pixel_total : 955445 time to create 1 rle with new method : 0.3137338161468506 length of segment : 1103 time for calcul the mask position with numpy : 0.0013048648834228516 nb_pixel_total : 55184 time to create 1 rle with old method : 0.07607293128967285 length of segment : 403 time for calcul the mask position with numpy : 0.011697053909301758 nb_pixel_total : 628248 time to create 1 rle with new method : 0.29196810722351074 length of segment : 1123 time for calcul the mask position with numpy : 0.014923095703125 nb_pixel_total : 904824 time to create 1 rle with new method : 0.044053077697753906 length of segment : 945 time for calcul the mask position with numpy : 0.01049184799194336 nb_pixel_total : 529568 time to create 1 rle with new method : 0.026464462280273438 length of segment : 909 time for calcul the mask position with numpy : 0.0018336772918701172 nb_pixel_total : 89429 time to create 1 rle with old method : 0.10642337799072266 length of segment : 299 time for calcul the mask position with numpy : 0.0008885860443115234 nb_pixel_total : 23526 time to create 1 rle with old method : 0.030326128005981445 length of segment : 183 time for calcul the mask position with numpy : 0.008798837661743164 nb_pixel_total : 431314 time to create 1 rle with new method : 0.02432394027709961 length of segment : 776 time for calcul the mask position with numpy : 0.007279872894287109 nb_pixel_total : 419731 time to create 1 rle with new method : 0.01871514320373535 length of segment : 636 time for calcul the mask position with numpy : 0.003161191940307617 nb_pixel_total : 183774 time to create 1 rle with new method : 0.00758814811706543 length of segment : 471 time for calcul the mask position with numpy : 0.001310586929321289 nb_pixel_total : 72874 time to create 1 rle with old method : 0.08908629417419434 length of segment : 312 time for calcul the mask position with numpy : 0.0017707347869873047 nb_pixel_total : 66618 time to create 1 rle with old method : 0.07601308822631836 length of segment : 479 time for calcul the mask position with numpy : 0.001695394515991211 nb_pixel_total : 81274 time to create 1 rle with old method : 0.09231972694396973 length of segment : 440 time for calcul the mask position with numpy : 0.001028299331665039 nb_pixel_total : 34055 time to create 1 rle with old method : 0.03969573974609375 length of segment : 290 time for calcul the mask position with numpy : 0.0028214454650878906 nb_pixel_total : 131954 time to create 1 rle with old method : 0.15029048919677734 length of segment : 441 time for calcul the mask position with numpy : 0.0009810924530029297 nb_pixel_total : 25829 time to create 1 rle with old method : 0.0323636531829834 length of segment : 198 time for calcul the mask position with numpy : 0.005208015441894531 nb_pixel_total : 285099 time to create 1 rle with new method : 0.015282154083251953 length of segment : 852 time for calcul the mask position with numpy : 0.001889944076538086 nb_pixel_total : 76296 time to create 1 rle with old method : 0.09132099151611328 length of segment : 428 time for calcul the mask position with numpy : 0.013708829879760742 nb_pixel_total : 584928 time to create 1 rle with new method : 0.03139638900756836 length of segment : 864 time for calcul the mask position with numpy : 0.0007731914520263672 nb_pixel_total : 46736 time to create 1 rle with old method : 0.05347752571105957 length of segment : 239 time for calcul the mask position with numpy : 0.0038709640502929688 nb_pixel_total : 231561 time to create 1 rle with new method : 0.011039018630981445 length of segment : 849 time for calcul the mask position with numpy : 0.0016036033630371094 nb_pixel_total : 71552 time to create 1 rle with old method : 0.08251595497131348 length of segment : 314 time for calcul the mask position with numpy : 0.0006067752838134766 nb_pixel_total : 33097 time to create 1 rle with old method : 0.0387420654296875 length of segment : 230 time for calcul the mask position with numpy : 0.014175891876220703 nb_pixel_total : 668539 time to create 1 rle with new method : 0.03749799728393555 length of segment : 955 time for calcul the mask position with numpy : 0.0008449554443359375 nb_pixel_total : 40723 time to create 1 rle with old method : 0.05281949043273926 length of segment : 197 time for calcul the mask position with numpy : 0.0071561336517333984 nb_pixel_total : 343694 time to create 1 rle with new method : 0.020214319229125977 length of segment : 590 time for calcul the mask position with numpy : 0.0033419132232666016 nb_pixel_total : 141080 time to create 1 rle with old method : 0.15946578979492188 length of segment : 452 time for calcul the mask position with numpy : 0.0028829574584960938 nb_pixel_total : 153968 time to create 1 rle with new method : 0.008386611938476562 length of segment : 486 time for calcul the mask position with numpy : 0.007692098617553711 nb_pixel_total : 370950 time to create 1 rle with new method : 0.023058176040649414 length of segment : 948 time for calcul the mask position with numpy : 0.0018951892852783203 nb_pixel_total : 105337 time to create 1 rle with old method : 0.11925506591796875 length of segment : 331 time for calcul the mask position with numpy : 0.0005223751068115234 nb_pixel_total : 20454 time to create 1 rle with old method : 0.023781538009643555 length of segment : 165 time for calcul the mask position with numpy : 0.0017197132110595703 nb_pixel_total : 74586 time to create 1 rle with old method : 0.0858008861541748 length of segment : 324 time for calcul the mask position with numpy : 0.0009980201721191406 nb_pixel_total : 47144 time to create 1 rle with old method : 0.05426740646362305 length of segment : 161 time for calcul the mask position with numpy : 0.0009272098541259766 nb_pixel_total : 15978 time to create 1 rle with old method : 0.01894974708557129 length of segment : 246 time for calcul the mask position with numpy : 0.0014348030090332031 nb_pixel_total : 70747 time to create 1 rle with old method : 0.08075332641601562 length of segment : 276 time for calcul the mask position with numpy : 0.0020062923431396484 nb_pixel_total : 88114 time to create 1 rle with old method : 0.11461234092712402 length of segment : 661 time for calcul the mask position with numpy : 0.0009844303131103516 nb_pixel_total : 39586 time to create 1 rle with old method : 0.045174598693847656 length of segment : 282 time for calcul the mask position with numpy : 0.003926753997802734 nb_pixel_total : 117649 time to create 1 rle with old method : 0.16031217575073242 length of segment : 619 time for calcul the mask position with numpy : 0.004189491271972656 nb_pixel_total : 239068 time to create 1 rle with new method : 0.011592388153076172 length of segment : 710 time for calcul the mask position with numpy : 0.0038475990295410156 nb_pixel_total : 199346 time to create 1 rle with new method : 0.010882377624511719 length of segment : 382 time for calcul the mask position with numpy : 0.0013778209686279297 nb_pixel_total : 83227 time to create 1 rle with old method : 0.09835195541381836 length of segment : 433 time for calcul the mask position with numpy : 0.0006821155548095703 nb_pixel_total : 23562 time to create 1 rle with old method : 0.027089595794677734 length of segment : 186 time for calcul the mask position with numpy : 0.0010738372802734375 nb_pixel_total : 58062 time to create 1 rle with old method : 0.07093477249145508 length of segment : 216 time for calcul the mask position with numpy : 0.0009248256683349609 nb_pixel_total : 30010 time to create 1 rle with old method : 0.03367805480957031 length of segment : 258 time for calcul the mask position with numpy : 0.009263277053833008 nb_pixel_total : 323370 time to create 1 rle with new method : 0.02855062484741211 length of segment : 1669 time for calcul the mask position with numpy : 0.008052587509155273 nb_pixel_total : 325855 time to create 1 rle with new method : 0.018298625946044922 length of segment : 608 time for calcul the mask position with numpy : 0.0020437240600585938 nb_pixel_total : 146464 time to create 1 rle with old method : 0.16870975494384766 length of segment : 214 time for calcul the mask position with numpy : 0.0017364025115966797 nb_pixel_total : 41719 time to create 1 rle with old method : 0.05035090446472168 length of segment : 217 time for calcul the mask position with numpy : 0.004750728607177734 nb_pixel_total : 283310 time to create 1 rle with new method : 0.010854482650756836 length of segment : 308 time for calcul the mask position with numpy : 0.0041675567626953125 nb_pixel_total : 275211 time to create 1 rle with new method : 0.009887933731079102 length of segment : 499 time for calcul the mask position with numpy : 0.0007128715515136719 nb_pixel_total : 35173 time to create 1 rle with old method : 0.0394740104675293 length of segment : 206 time for calcul the mask position with numpy : 0.0037174224853515625 nb_pixel_total : 203405 time to create 1 rle with new method : 0.009506702423095703 length of segment : 455 time for calcul the mask position with numpy : 0.013946771621704102 nb_pixel_total : 490392 time to create 1 rle with new method : 0.3318006992340088 length of segment : 761 time for calcul the mask position with numpy : 0.009377479553222656 nb_pixel_total : 448430 time to create 1 rle with new method : 0.06126666069030762 length of segment : 1031 time for calcul the mask position with numpy : 0.0010142326354980469 nb_pixel_total : 42785 time to create 1 rle with old method : 0.04835247993469238 length of segment : 352 time for calcul the mask position with numpy : 0.0009789466857910156 nb_pixel_total : 34626 time to create 1 rle with old method : 0.04033541679382324 length of segment : 235 time for calcul the mask position with numpy : 0.003961801528930664 nb_pixel_total : 189494 time to create 1 rle with new method : 0.010810613632202148 length of segment : 436 time for calcul the mask position with numpy : 0.009051799774169922 nb_pixel_total : 487344 time to create 1 rle with new method : 0.02263951301574707 length of segment : 722 time for calcul the mask position with numpy : 0.004095315933227539 nb_pixel_total : 195427 time to create 1 rle with new method : 0.012235403060913086 length of segment : 459 time for calcul the mask position with numpy : 0.0005183219909667969 nb_pixel_total : 30104 time to create 1 rle with old method : 0.03388643264770508 length of segment : 198 time for calcul the mask position with numpy : 0.0007326602935791016 nb_pixel_total : 32998 time to create 1 rle with old method : 0.040258169174194336 length of segment : 189 time for calcul the mask position with numpy : 0.0017786026000976562 nb_pixel_total : 73892 time to create 1 rle with old method : 0.08664321899414062 length of segment : 293 time for calcul the mask position with numpy : 0.0023717880249023438 nb_pixel_total : 100008 time to create 1 rle with old method : 0.11161923408508301 length of segment : 334 time for calcul the mask position with numpy : 0.0009715557098388672 nb_pixel_total : 45271 time to create 1 rle with old method : 0.06035304069519043 length of segment : 314 time for calcul the mask position with numpy : 0.0009124279022216797 nb_pixel_total : 37278 time to create 1 rle with old method : 0.04269146919250488 length of segment : 189 time for calcul the mask position with numpy : 0.0014119148254394531 nb_pixel_total : 84149 time to create 1 rle with old method : 0.10469985008239746 length of segment : 219 time for calcul the mask position with numpy : 0.00028514862060546875 nb_pixel_total : 11437 time to create 1 rle with old method : 0.014798879623413086 length of segment : 99 time for calcul the mask position with numpy : 0.004393577575683594 nb_pixel_total : 204260 time to create 1 rle with new method : 0.010578632354736328 length of segment : 507 time for calcul the mask position with numpy : 0.007624149322509766 nb_pixel_total : 393269 time to create 1 rle with new method : 0.02046513557434082 length of segment : 1093 time for calcul the mask position with numpy : 0.0009884834289550781 nb_pixel_total : 46140 time to create 1 rle with old method : 0.05509185791015625 length of segment : 227 time for calcul the mask position with numpy : 0.007200956344604492 nb_pixel_total : 344581 time to create 1 rle with new method : 0.020016908645629883 length of segment : 693 time for calcul the mask position with numpy : 0.0011017322540283203 nb_pixel_total : 26527 time to create 1 rle with old method : 0.03394722938537598 length of segment : 170 time for calcul the mask position with numpy : 0.0016109943389892578 nb_pixel_total : 84219 time to create 1 rle with old method : 0.09764671325683594 length of segment : 304 time for calcul the mask position with numpy : 0.013096809387207031 nb_pixel_total : 352424 time to create 1 rle with new method : 0.04873323440551758 length of segment : 1214 time for calcul the mask position with numpy : 0.0006589889526367188 nb_pixel_total : 23320 time to create 1 rle with old method : 0.026852846145629883 length of segment : 186 time for calcul the mask position with numpy : 0.032034873962402344 nb_pixel_total : 935307 time to create 1 rle with new method : 0.6703760623931885 length of segment : 821 time for calcul the mask position with numpy : 0.011151552200317383 nb_pixel_total : 663738 time to create 1 rle with new method : 0.035028696060180664 length of segment : 893 time for calcul the mask position with numpy : 0.004001617431640625 nb_pixel_total : 236443 time to create 1 rle with new method : 0.010756969451904297 length of segment : 767 time for calcul the mask position with numpy : 0.01833367347717285 nb_pixel_total : 602399 time to create 1 rle with new method : 0.49814558029174805 length of segment : 770 time for calcul the mask position with numpy : 0.010050058364868164 nb_pixel_total : 515822 time to create 1 rle with new method : 0.051587820053100586 length of segment : 1651 time for calcul the mask position with numpy : 0.002454042434692383 nb_pixel_total : 124936 time to create 1 rle with old method : 0.15050697326660156 length of segment : 320 time for calcul the mask position with numpy : 0.0011644363403320312 nb_pixel_total : 59769 time to create 1 rle with old method : 0.06952190399169922 length of segment : 325 time for calcul the mask position with numpy : 0.0011060237884521484 nb_pixel_total : 55521 time to create 1 rle with old method : 0.06447052955627441 length of segment : 195 time for calcul the mask position with numpy : 0.001352071762084961 nb_pixel_total : 62582 time to create 1 rle with old method : 0.07236671447753906 length of segment : 316 time for calcul the mask position with numpy : 0.0027823448181152344 nb_pixel_total : 131805 time to create 1 rle with old method : 0.17104172706604004 length of segment : 311 time for calcul the mask position with numpy : 0.0017442703247070312 nb_pixel_total : 73163 time to create 1 rle with old method : 0.0860898494720459 length of segment : 272 time for calcul the mask position with numpy : 0.5035302639007568 nb_pixel_total : 5853333 time to create 1 rle with new method : 1.5619475841522217 length of segment : 4386 time for calcul the mask position with numpy : 0.0004551410675048828 nb_pixel_total : 22225 time to create 1 rle with old method : 0.025767087936401367 length of segment : 225 time for calcul the mask position with numpy : 0.00012803077697753906 nb_pixel_total : 5706 time to create 1 rle with old method : 0.006418943405151367 length of segment : 168 time for calcul the mask position with numpy : 0.001132965087890625 nb_pixel_total : 94913 time to create 1 rle with old method : 0.10625767707824707 length of segment : 211 time for calcul the mask position with numpy : 0.0008211135864257812 nb_pixel_total : 74200 time to create 1 rle with old method : 0.08352327346801758 length of segment : 180 time for calcul the mask position with numpy : 0.4374573230743408 nb_pixel_total : 4401242 time to create 1 rle with new method : 1.153456449508667 length of segment : 1725 time for calcul the mask position with numpy : 0.0012211799621582031 nb_pixel_total : 114559 time to create 1 rle with old method : 0.1272430419921875 length of segment : 236 time for calcul the mask position with numpy : 0.026091575622558594 nb_pixel_total : 1335409 time to create 1 rle with new method : 0.7115731239318848 length of segment : 4042 time for calcul the mask position with numpy : 0.0009326934814453125 nb_pixel_total : 49565 time to create 1 rle with old method : 0.05614924430847168 length of segment : 253 time for calcul the mask position with numpy : 0.0003447532653808594 nb_pixel_total : 19568 time to create 1 rle with old method : 0.02314448356628418 length of segment : 68 time for calcul the mask position with numpy : 0.0011410713195800781 nb_pixel_total : 92911 time to create 1 rle with old method : 0.10567975044250488 length of segment : 196 time for calcul the mask position with numpy : 0.00045800209045410156 nb_pixel_total : 28842 time to create 1 rle with old method : 0.03523373603820801 length of segment : 174 time for calcul the mask position with numpy : 0.001323699951171875 nb_pixel_total : 129164 time to create 1 rle with old method : 0.14789795875549316 length of segment : 266 time for calcul the mask position with numpy : 0.0011811256408691406 nb_pixel_total : 100145 time to create 1 rle with old method : 0.11431097984313965 length of segment : 222 time for calcul the mask position with numpy : 0.24414992332458496 nb_pixel_total : 6312967 time to create 1 rle with new method : 1.5436573028564453 length of segment : 4373 time for calcul the mask position with numpy : 0.02932119369506836 nb_pixel_total : 1229970 time to create 1 rle with new method : 0.06162309646606445 length of segment : 914 time for calcul the mask position with numpy : 0.003397703170776367 nb_pixel_total : 310157 time to create 1 rle with new method : 0.0080718994140625 length of segment : 933 time for calcul the mask position with numpy : 0.0011096000671386719 nb_pixel_total : 92849 time to create 1 rle with old method : 0.1060037612915039 length of segment : 304 time for calcul the mask position with numpy : 0.0005562305450439453 nb_pixel_total : 31163 time to create 1 rle with old method : 0.03573727607727051 length of segment : 354 time for calcul the mask position with numpy : 0.00483393669128418 nb_pixel_total : 145772 time to create 1 rle with old method : 0.18934106826782227 length of segment : 917 time for calcul the mask position with numpy : 0.003365039825439453 nb_pixel_total : 333585 time to create 1 rle with new method : 0.007760047912597656 length of segment : 736 time spent for convertir_results : 40.81067442893982 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 211 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 112.81856083869934 time spend to save output : 0.14123964309692383 total time spend for step 2 : 112.95980048179626 step3:blur_detection Tue May 6 12:05:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (3120, 4160) 2243.3896209577115 (3120, 4160) 533.6080237762293 (3120, 4160) 2704.1605869591826 (3120, 4160) 1298.6623445949415 (3120, 4160) 500.26206196950665 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.012134552001953125 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.012602567672729492 save missing photos in datou_result : time spend for datou_step_exec : 2.2916438579559326 time spend to save output : 0.028850078582763672 total time spend for step 3 : 2.3204939365386963 step4:brightness Tue May 6 12:05:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1746525690_652770_1356331931_a13546e12ef85d2f6f8bc2661a4eb503.jpg treat image : temp/1746525690_652770_1356331923_f9041307201d4d4d58c62c9b942b5644.jpg treat image : temp/1746525690_652770_1356331918_05b72b2ea6bfd9a47f415decb8a9da42.jpg treat image : temp/1746525690_652770_1356331909_dfb6fa9867a3daa9da40c589f865d49f.jpg treat image : temp/1746525690_652770_1356331907_3b89b5d7a9d3d19adb6e75f923849bc2.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 5 time used for this insertion : 0.010644912719726562 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 5 time used for this insertion : 0.011108875274658203 save missing photos in datou_result : time spend for datou_step_exec : 8.641755104064941 time spend to save output : 0.02629566192626953 total time spend for step 4 : 8.668050765991211 step5:rle_unique_nms_with_priority Tue May 6 12:05:25 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 211 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 38 nb_hashtags : 8 time to prepare the origin masks : 34.127471685409546 time for calcul the mask position with numpy : 0.3972926139831543 nb_pixel_total : 3765460 time to create 1 rle with new method : 1.801344633102417 time for calcul the mask position with numpy : 0.05496931076049805 nb_pixel_total : 76634 time to create 1 rle with old method : 0.08808302879333496 time for calcul the mask position with numpy : 0.09189701080322266 nb_pixel_total : 1115722 time to create 1 rle with new method : 2.265364170074463 time for calcul the mask position with numpy : 0.06373357772827148 nb_pixel_total : 487446 time to create 1 rle with new method : 1.3669655323028564 time for calcul the mask position with numpy : 0.32370901107788086 nb_pixel_total : 2213108 time to create 1 rle with new method : 0.9941227436065674 time for calcul the mask position with numpy : 0.05474114418029785 nb_pixel_total : 42116 time to create 1 rle with old method : 0.04695630073547363 time for calcul the mask position with numpy : 0.055724382400512695 nb_pixel_total : 193329 time to create 1 rle with new method : 1.6081316471099854 time for calcul the mask position with numpy : 0.054863691329956055 nb_pixel_total : 177609 time to create 1 rle with new method : 2.1746280193328857 time for calcul the mask position with numpy : 0.0537714958190918 nb_pixel_total : 3546 time to create 1 rle with old method : 0.004098415374755859 time for calcul the mask position with numpy : 0.05406808853149414 nb_pixel_total : 25391 time to create 1 rle with old method : 0.029144763946533203 time for calcul the mask position with numpy : 0.05866241455078125 nb_pixel_total : 456302 time to create 1 rle with new method : 0.957240104675293 time for calcul the mask position with numpy : 0.05719470977783203 nb_pixel_total : 180775 time to create 1 rle with new method : 1.2645816802978516 time for calcul the mask position with numpy : 0.05352592468261719 nb_pixel_total : 13869 time to create 1 rle with old method : 0.015470027923583984 time for calcul the mask position with numpy : 0.059407711029052734 nb_pixel_total : 65385 time to create 1 rle with old method : 0.08225274085998535 time for calcul the mask position with numpy : 0.055895328521728516 nb_pixel_total : 3762 time to create 1 rle with old method : 0.004261493682861328 time for calcul the mask position with numpy : 0.0534665584564209 nb_pixel_total : 201 time to create 1 rle with old method : 0.00029754638671875 time for calcul the mask position with numpy : 0.054444313049316406 nb_pixel_total : 141978 time to create 1 rle with old method : 0.15702176094055176 time for calcul the mask position with numpy : 0.05561399459838867 nb_pixel_total : 190058 time to create 1 rle with new method : 1.268923044204712 time for calcul the mask position with numpy : 0.05704045295715332 nb_pixel_total : 5341 time to create 1 rle with old method : 0.006188631057739258 time for calcul the mask position with numpy : 0.053871870040893555 nb_pixel_total : 21743 time to create 1 rle with old method : 0.02530670166015625 time for calcul the mask position with numpy : 0.054506778717041016 nb_pixel_total : 49631 time to create 1 rle with old method : 0.05556344985961914 time for calcul the mask position with numpy : 0.05578756332397461 nb_pixel_total : 166580 time to create 1 rle with new method : 1.0995466709136963 time for calcul the mask position with numpy : 0.05831503868103027 nb_pixel_total : 267952 time to create 1 rle with new method : 1.2649996280670166 time for calcul the mask position with numpy : 0.052722930908203125 nb_pixel_total : 53924 time to create 1 rle with old method : 0.061562299728393555 time for calcul the mask position with numpy : 0.057892560958862305 nb_pixel_total : 5519 time to create 1 rle with old method : 0.0063686370849609375 time for calcul the mask position with numpy : 0.05358386039733887 nb_pixel_total : 25983 time to create 1 rle with old method : 0.02921009063720703 time for calcul the mask position with numpy : 0.05386710166931152 nb_pixel_total : 15268 time to create 1 rle with old method : 0.017195940017700195 time for calcul the mask position with numpy : 0.05435824394226074 nb_pixel_total : 133800 time to create 1 rle with old method : 0.14938139915466309 time for calcul the mask position with numpy : 0.05713081359863281 nb_pixel_total : 72146 time to create 1 rle with old method : 0.08227133750915527 time for calcul the mask position with numpy : 0.05383610725402832 nb_pixel_total : 68041 time to create 1 rle with old method : 0.07509756088256836 time for calcul the mask position with numpy : 0.14485597610473633 nb_pixel_total : 2561171 time to create 1 rle with new method : 0.7712967395782471 time for calcul the mask position with numpy : 0.053939104080200195 nb_pixel_total : 46238 time to create 1 rle with old method : 0.05157041549682617 time for calcul the mask position with numpy : 0.05584311485290527 nb_pixel_total : 166026 time to create 1 rle with new method : 1.5075006484985352 time for calcul the mask position with numpy : 0.05483531951904297 nb_pixel_total : 109302 time to create 1 rle with old method : 0.12166380882263184 time for calcul the mask position with numpy : 0.05357694625854492 nb_pixel_total : 11 time to create 1 rle with old method : 5.745887756347656e-05 time for calcul the mask position with numpy : 0.05519604682922363 nb_pixel_total : 45838 time to create 1 rle with old method : 0.05291914939880371 time for calcul the mask position with numpy : 0.05338406562805176 nb_pixel_total : 278 time to create 1 rle with old method : 0.00040912628173828125 time for calcul the mask position with numpy : 0.054573774337768555 nb_pixel_total : 420 time to create 1 rle with old method : 0.0009436607360839844 time for calcul the mask position with numpy : 0.06114983558654785 nb_pixel_total : 11297 time to create 1 rle with old method : 0.012847423553466797 create new chi : 23.073750495910645 time to delete rle : 0.027161598205566406 batch 1 Loaded 39 chid ids of type : 4855 Number RLEs to save : 30640 TO DO : save crop sub photo not yet done ! save time : 1.7506434917449951 nb_obj : 71 nb_hashtags : 10 time to prepare the origin masks : 51.64623498916626 time for calcul the mask position with numpy : 0.3788011074066162 nb_pixel_total : 2911342 time to create 1 rle with new method : 1.5261147022247314 time for calcul the mask position with numpy : 0.05407381057739258 nb_pixel_total : 9665 time to create 1 rle with old method : 0.011547088623046875 time for calcul the mask position with numpy : 0.054988861083984375 nb_pixel_total : 28524 time to create 1 rle with old method : 0.03227496147155762 time for calcul the mask position with numpy : 0.054935455322265625 nb_pixel_total : 72564 time to create 1 rle with old method : 0.0808563232421875 time for calcul the mask position with numpy : 0.06059002876281738 nb_pixel_total : 46986 time to create 1 rle with old method : 0.060422658920288086 time for calcul the mask position with numpy : 0.05374026298522949 nb_pixel_total : 20464 time to create 1 rle with old method : 0.023037433624267578 time for calcul the mask position with numpy : 0.05652952194213867 nb_pixel_total : 268117 time to create 1 rle with new method : 0.7732353210449219 time for calcul the mask position with numpy : 0.05560898780822754 nb_pixel_total : 291567 time to create 1 rle with new method : 0.7481884956359863 time for calcul the mask position with numpy : 0.05513739585876465 nb_pixel_total : 141601 time to create 1 rle with old method : 0.1611649990081787 time for calcul the mask position with numpy : 0.05373525619506836 nb_pixel_total : 379 time to create 1 rle with old method : 0.0006175041198730469 time for calcul the mask position with numpy : 0.05414175987243652 nb_pixel_total : 39497 time to create 1 rle with old method : 0.04465007781982422 time for calcul the mask position with numpy : 0.05336761474609375 nb_pixel_total : 4672 time to create 1 rle with old method : 0.005506992340087891 time for calcul the mask position with numpy : 0.05466580390930176 nb_pixel_total : 111312 time to create 1 rle with old method : 0.12359881401062012 time for calcul the mask position with numpy : 0.06872439384460449 nb_pixel_total : 523123 time to create 1 rle with new method : 1.4201972484588623 time for calcul the mask position with numpy : 0.06238055229187012 nb_pixel_total : 341375 time to create 1 rle with new method : 0.53912353515625 time for calcul the mask position with numpy : 0.05952620506286621 nb_pixel_total : 584530 time to create 1 rle with new method : 0.9900939464569092 time for calcul the mask position with numpy : 0.05598282814025879 nb_pixel_total : 194801 time to create 1 rle with new method : 1.1647956371307373 time for calcul the mask position with numpy : 0.05438041687011719 nb_pixel_total : 26289 time to create 1 rle with old method : 0.029742717742919922 time for calcul the mask position with numpy : 0.05714821815490723 nb_pixel_total : 240393 time to create 1 rle with new method : 2.170346736907959 time for calcul the mask position with numpy : 0.05371689796447754 nb_pixel_total : 16695 time to create 1 rle with old method : 0.01867532730102539 time for calcul the mask position with numpy : 0.0545501708984375 nb_pixel_total : 52571 time to create 1 rle with old method : 0.06016659736633301 time for calcul the mask position with numpy : 0.05571603775024414 nb_pixel_total : 1780 time to create 1 rle with old method : 0.0023317337036132812 time for calcul the mask position with numpy : 0.05428004264831543 nb_pixel_total : 40347 time to create 1 rle with old method : 0.04505467414855957 time for calcul the mask position with numpy : 0.05614113807678223 nb_pixel_total : 7000 time to create 1 rle with old method : 0.008271217346191406 time for calcul the mask position with numpy : 0.055428266525268555 nb_pixel_total : 186444 time to create 1 rle with new method : 1.1335382461547852 time for calcul the mask position with numpy : 0.07852578163146973 nb_pixel_total : 954288 time to create 1 rle with new method : 1.9493029117584229 time for calcul the mask position with numpy : 0.054376840591430664 nb_pixel_total : 46 time to create 1 rle with old method : 0.00011396408081054688 time for calcul the mask position with numpy : 0.05476641654968262 nb_pixel_total : 538 time to create 1 rle with old method : 0.0006871223449707031 time for calcul the mask position with numpy : 0.05478811264038086 nb_pixel_total : 39253 time to create 1 rle with old method : 0.044251441955566406 time for calcul the mask position with numpy : 0.054387807846069336 nb_pixel_total : 1735 time to create 1 rle with old method : 0.0024340152740478516 time for calcul the mask position with numpy : 0.05538034439086914 nb_pixel_total : 37038 time to create 1 rle with old method : 0.04238009452819824 time for calcul the mask position with numpy : 0.0547642707824707 nb_pixel_total : 55803 time to create 1 rle with old method : 0.062957763671875 time for calcul the mask position with numpy : 0.06387639045715332 nb_pixel_total : 449628 time to create 1 rle with new method : 1.9098951816558838 time for calcul the mask position with numpy : 0.054747819900512695 nb_pixel_total : 47214 time to create 1 rle with old method : 0.053923606872558594 time for calcul the mask position with numpy : 0.057521820068359375 nb_pixel_total : 387453 time to create 1 rle with new method : 1.571725606918335 time for calcul the mask position with numpy : 0.05419301986694336 nb_pixel_total : 54870 time to create 1 rle with old method : 0.06432747840881348 time for calcul the mask position with numpy : 0.0554041862487793 nb_pixel_total : 109784 time to create 1 rle with old method : 0.13603544235229492 time for calcul the mask position with numpy : 0.05553150177001953 nb_pixel_total : 205541 time to create 1 rle with new method : 0.9671657085418701 time for calcul the mask position with numpy : 0.054831743240356445 nb_pixel_total : 179218 time to create 1 rle with new method : 1.0953295230865479 time for calcul the mask position with numpy : 0.05466580390930176 nb_pixel_total : 26448 time to create 1 rle with old method : 0.029986858367919922 time for calcul the mask position with numpy : 0.056993722915649414 nb_pixel_total : 73592 time to create 1 rle with old method : 0.09419727325439453 time for calcul the mask position with numpy : 0.06245303153991699 nb_pixel_total : 218630 time to create 1 rle with new method : 1.0710875988006592 time for calcul the mask position with numpy : 0.05622243881225586 nb_pixel_total : 27692 time to create 1 rle with old method : 0.0316615104675293 time for calcul the mask position with numpy : 0.05527329444885254 nb_pixel_total : 18585 time to create 1 rle with old method : 0.021770477294921875 time for calcul the mask position with numpy : 0.05748629570007324 nb_pixel_total : 154096 time to create 1 rle with new method : 1.5676229000091553 time for calcul the mask position with numpy : 0.061324357986450195 nb_pixel_total : 47418 time to create 1 rle with old method : 0.06085968017578125 time for calcul the mask position with numpy : 0.054691314697265625 nb_pixel_total : 105048 time to create 1 rle with old method : 0.11651968955993652 time for calcul the mask position with numpy : 0.053954362869262695 nb_pixel_total : 24451 time to create 1 rle with old method : 0.027650117874145508 time for calcul the mask position with numpy : 0.06055712699890137 nb_pixel_total : 472047 time to create 1 rle with new method : 0.7534694671630859 time for calcul the mask position with numpy : 0.054418087005615234 nb_pixel_total : 25414 time to create 1 rle with old method : 0.02884984016418457 time for calcul the mask position with numpy : 0.055806636810302734 nb_pixel_total : 33030 time to create 1 rle with old method : 0.036644697189331055 time for calcul the mask position with numpy : 0.05517077445983887 nb_pixel_total : 23414 time to create 1 rle with old method : 0.02626180648803711 time for calcul the mask position with numpy : 0.06260180473327637 nb_pixel_total : 903145 time to create 1 rle with new method : 0.881605863571167 time for calcul the mask position with numpy : 0.055048465728759766 nb_pixel_total : 45621 time to create 1 rle with old method : 0.051444053649902344 time for calcul the mask position with numpy : 0.05793929100036621 nb_pixel_total : 303888 time to create 1 rle with new method : 1.9484410285949707 time for calcul the mask position with numpy : 0.055750131607055664 nb_pixel_total : 39465 time to create 1 rle with old method : 0.04519367218017578 time for calcul the mask position with numpy : 0.05657649040222168 nb_pixel_total : 221695 time to create 1 rle with new method : 1.2331373691558838 time for calcul the mask position with numpy : 0.059442996978759766 nb_pixel_total : 264864 time to create 1 rle with new method : 1.7464909553527832 time for calcul the mask position with numpy : 0.054892778396606445 nb_pixel_total : 1106 time to create 1 rle with old method : 0.0015902519226074219 time for calcul the mask position with numpy : 0.058007001876831055 nb_pixel_total : 389375 time to create 1 rle with new method : 1.3636815547943115 time for calcul the mask position with numpy : 0.05624127388000488 nb_pixel_total : 81162 time to create 1 rle with old method : 0.09359622001647949 time for calcul the mask position with numpy : 0.05405473709106445 nb_pixel_total : 47976 time to create 1 rle with old method : 0.06432485580444336 time for calcul the mask position with numpy : 0.06182527542114258 nb_pixel_total : 209055 time to create 1 rle with new method : 1.2776875495910645 time for calcul the mask position with numpy : 0.054701805114746094 nb_pixel_total : 33702 time to create 1 rle with old method : 0.03839755058288574 time for calcul the mask position with numpy : 0.05515742301940918 nb_pixel_total : 64477 time to create 1 rle with old method : 0.07072257995605469 time for calcul the mask position with numpy : 0.0532376766204834 nb_pixel_total : 1589 time to create 1 rle with old method : 0.0023005008697509766 time for calcul the mask position with numpy : 0.05449199676513672 nb_pixel_total : 185349 time to create 1 rle with new method : 0.8648338317871094 time for calcul the mask position with numpy : 0.055538177490234375 nb_pixel_total : 89166 time to create 1 rle with old method : 0.09798455238342285 time for calcul the mask position with numpy : 0.05295372009277344 nb_pixel_total : 73492 time to create 1 rle with old method : 0.0800013542175293 time for calcul the mask position with numpy : 0.053098201751708984 nb_pixel_total : 15876 time to create 1 rle with old method : 0.017366409301757812 time for calcul the mask position with numpy : 0.0536198616027832 nb_pixel_total : 2204 time to create 1 rle with old method : 0.002784252166748047 time for calcul the mask position with numpy : 0.05889630317687988 nb_pixel_total : 71681 time to create 1 rle with old method : 0.09232568740844727 create new chi : 38.45407843589783 time to delete rle : 0.009991645812988281 batch 1 Loaded 72 chid ids of type : 4855 Number RLEs to save : 57400 TO DO : save crop sub photo not yet done ! save time : 3.1210687160491943 nb_obj : 71 nb_hashtags : 9 time to prepare the origin masks : 55.30959486961365 time for calcul the mask position with numpy : 0.6697244644165039 nb_pixel_total : 3784983 time to create 1 rle with new method : 0.7675826549530029 time for calcul the mask position with numpy : 0.30602502822875977 nb_pixel_total : 1853829 time to create 1 rle with new method : 2.0641510486602783 time for calcul the mask position with numpy : 0.06304764747619629 nb_pixel_total : 840505 time to create 1 rle with new method : 1.2326979637145996 time for calcul the mask position with numpy : 0.053847312927246094 nb_pixel_total : 41674 time to create 1 rle with old method : 0.04816627502441406 time for calcul the mask position with numpy : 0.05541658401489258 nb_pixel_total : 13647 time to create 1 rle with old method : 0.01609945297241211 time for calcul the mask position with numpy : 0.05334663391113281 nb_pixel_total : 12314 time to create 1 rle with old method : 0.014512062072753906 time for calcul the mask position with numpy : 0.05366706848144531 nb_pixel_total : 50619 time to create 1 rle with old method : 0.057076215744018555 time for calcul the mask position with numpy : 0.05417752265930176 nb_pixel_total : 86056 time to create 1 rle with old method : 0.09430456161499023 time for calcul the mask position with numpy : 0.05325746536254883 nb_pixel_total : 1744 time to create 1 rle with old method : 0.0021851062774658203 time for calcul the mask position with numpy : 0.05324745178222656 nb_pixel_total : 350 time to create 1 rle with old method : 0.00043654441833496094 time for calcul the mask position with numpy : 0.05584573745727539 nb_pixel_total : 362410 time to create 1 rle with new method : 1.6872611045837402 time for calcul the mask position with numpy : 0.06343340873718262 nb_pixel_total : 202573 time to create 1 rle with new method : 1.4500937461853027 time for calcul the mask position with numpy : 0.05363821983337402 nb_pixel_total : 12651 time to create 1 rle with old method : 0.01413583755493164 time for calcul the mask position with numpy : 0.05587410926818848 nb_pixel_total : 397221 time to create 1 rle with new method : 1.3538613319396973 time for calcul the mask position with numpy : 0.053592681884765625 nb_pixel_total : 24548 time to create 1 rle with old method : 0.029205799102783203 time for calcul the mask position with numpy : 0.05502200126647949 nb_pixel_total : 332804 time to create 1 rle with new method : 1.784282922744751 time for calcul the mask position with numpy : 0.0587458610534668 nb_pixel_total : 216674 time to create 1 rle with new method : 1.7559683322906494 time for calcul the mask position with numpy : 0.05665707588195801 nb_pixel_total : 17628 time to create 1 rle with old method : 0.019813060760498047 time for calcul the mask position with numpy : 0.05994415283203125 nb_pixel_total : 188714 time to create 1 rle with new method : 1.1770761013031006 time for calcul the mask position with numpy : 0.05348038673400879 nb_pixel_total : 19578 time to create 1 rle with old method : 0.022073030471801758 time for calcul the mask position with numpy : 0.05459284782409668 nb_pixel_total : 325497 time to create 1 rle with new method : 1.4286973476409912 time for calcul the mask position with numpy : 0.0534515380859375 nb_pixel_total : 74335 time to create 1 rle with old method : 0.08311343193054199 time for calcul the mask position with numpy : 0.0532529354095459 nb_pixel_total : 3109 time to create 1 rle with old method : 0.0037348270416259766 time for calcul the mask position with numpy : 0.053206443786621094 nb_pixel_total : 1452 time to create 1 rle with old method : 0.0017147064208984375 time for calcul the mask position with numpy : 0.05338907241821289 nb_pixel_total : 23509 time to create 1 rle with old method : 0.026363372802734375 time for calcul the mask position with numpy : 0.053380727767944336 nb_pixel_total : 30806 time to create 1 rle with old method : 0.034433603286743164 time for calcul the mask position with numpy : 0.05720877647399902 nb_pixel_total : 14370 time to create 1 rle with old method : 0.016155481338500977 time for calcul the mask position with numpy : 0.052941083908081055 nb_pixel_total : 74896 time to create 1 rle with old method : 0.08494949340820312 time for calcul the mask position with numpy : 0.05315518379211426 nb_pixel_total : 572 time to create 1 rle with old method : 0.0008127689361572266 time for calcul the mask position with numpy : 0.054105281829833984 nb_pixel_total : 5554 time to create 1 rle with old method : 0.006322622299194336 time for calcul the mask position with numpy : 0.05362343788146973 nb_pixel_total : 103203 time to create 1 rle with old method : 0.11430168151855469 time for calcul the mask position with numpy : 0.05340147018432617 nb_pixel_total : 659 time to create 1 rle with old method : 0.0008051395416259766 time for calcul the mask position with numpy : 0.05366063117980957 nb_pixel_total : 111845 time to create 1 rle with old method : 0.12265968322753906 time for calcul the mask position with numpy : 0.05328369140625 nb_pixel_total : 3391 time to create 1 rle with old method : 0.00382232666015625 time for calcul the mask position with numpy : 0.05431032180786133 nb_pixel_total : 326156 time to create 1 rle with new method : 1.6387097835540771 time for calcul the mask position with numpy : 0.05362415313720703 nb_pixel_total : 65116 time to create 1 rle with old method : 0.2050161361694336 time for calcul the mask position with numpy : 0.05525708198547363 nb_pixel_total : 3251 time to create 1 rle with old method : 0.003931760787963867 time for calcul the mask position with numpy : 0.053762197494506836 nb_pixel_total : 184 time to create 1 rle with old method : 0.00054931640625 time for calcul the mask position with numpy : 0.059098243713378906 nb_pixel_total : 991 time to create 1 rle with old method : 0.0014376640319824219 time for calcul the mask position with numpy : 0.053658246994018555 nb_pixel_total : 61080 time to create 1 rle with old method : 0.06781911849975586 time for calcul the mask position with numpy : 0.05374860763549805 nb_pixel_total : 42549 time to create 1 rle with old method : 0.0477602481842041 time for calcul the mask position with numpy : 0.053726911544799805 nb_pixel_total : 99655 time to create 1 rle with old method : 0.11303400993347168 time for calcul the mask position with numpy : 0.0534820556640625 nb_pixel_total : 22559 time to create 1 rle with old method : 0.0251009464263916 time for calcul the mask position with numpy : 0.05446887016296387 nb_pixel_total : 145906 time to create 1 rle with old method : 0.166090726852417 time for calcul the mask position with numpy : 0.05340433120727539 nb_pixel_total : 239 time to create 1 rle with old method : 0.0005922317504882812 time for calcul the mask position with numpy : 0.053693532943725586 nb_pixel_total : 72314 time to create 1 rle with old method : 0.08319520950317383 time for calcul the mask position with numpy : 0.05337643623352051 nb_pixel_total : 2973 time to create 1 rle with old method : 0.0034728050231933594 time for calcul the mask position with numpy : 0.053194284439086914 nb_pixel_total : 620 time to create 1 rle with old method : 0.0008864402770996094 time for calcul the mask position with numpy : 0.05344414710998535 nb_pixel_total : 11738 time to create 1 rle with old method : 0.01313471794128418 time for calcul the mask position with numpy : 0.055422067642211914 nb_pixel_total : 274840 time to create 1 rle with new method : 1.3247690200805664 time for calcul the mask position with numpy : 0.053885698318481445 nb_pixel_total : 76767 time to create 1 rle with old method : 0.08442115783691406 time for calcul the mask position with numpy : 0.053186655044555664 nb_pixel_total : 581 time to create 1 rle with old method : 0.0010845661163330078 time for calcul the mask position with numpy : 0.05329775810241699 nb_pixel_total : 5069 time to create 1 rle with old method : 0.0056378841400146484 time for calcul the mask position with numpy : 0.055222272872924805 nb_pixel_total : 284364 time to create 1 rle with new method : 1.3451578617095947 time for calcul the mask position with numpy : 0.05448341369628906 nb_pixel_total : 674 time to create 1 rle with old method : 0.0008802413940429688 time for calcul the mask position with numpy : 0.05434870719909668 nb_pixel_total : 27983 time to create 1 rle with old method : 0.03203415870666504 time for calcul the mask position with numpy : 0.05479907989501953 nb_pixel_total : 175820 time to create 1 rle with new method : 1.0373361110687256 time for calcul the mask position with numpy : 0.05539751052856445 nb_pixel_total : 203907 time to create 1 rle with new method : 1.2373590469360352 time for calcul the mask position with numpy : 0.053510189056396484 nb_pixel_total : 29838 time to create 1 rle with old method : 0.033516645431518555 time for calcul the mask position with numpy : 0.053999900817871094 nb_pixel_total : 46029 time to create 1 rle with old method : 0.051666259765625 time for calcul the mask position with numpy : 0.05328178405761719 nb_pixel_total : 11282 time to create 1 rle with old method : 0.012857437133789062 time for calcul the mask position with numpy : 0.05487418174743652 nb_pixel_total : 15874 time to create 1 rle with old method : 0.017918825149536133 time for calcul the mask position with numpy : 0.0560147762298584 nb_pixel_total : 36835 time to create 1 rle with old method : 0.041402339935302734 time for calcul the mask position with numpy : 0.06126594543457031 nb_pixel_total : 34509 time to create 1 rle with old method : 0.045174360275268555 time for calcul the mask position with numpy : 0.05403614044189453 nb_pixel_total : 4939 time to create 1 rle with old method : 0.006031990051269531 time for calcul the mask position with numpy : 0.053586721420288086 nb_pixel_total : 35107 time to create 1 rle with old method : 0.03885626792907715 time for calcul the mask position with numpy : 0.05410313606262207 nb_pixel_total : 98044 time to create 1 rle with old method : 0.11029696464538574 time for calcul the mask position with numpy : 0.05684399604797363 nb_pixel_total : 465343 time to create 1 rle with new method : 1.0046656131744385 time for calcul the mask position with numpy : 0.06046581268310547 nb_pixel_total : 932868 time to create 1 rle with new method : 1.3243520259857178 time for calcul the mask position with numpy : 0.05427813529968262 nb_pixel_total : 41565 time to create 1 rle with old method : 0.046613216400146484 time for calcul the mask position with numpy : 0.054074764251708984 nb_pixel_total : 57781 time to create 1 rle with old method : 0.06396269798278809 time for calcul the mask position with numpy : 0.053624868392944336 nb_pixel_total : 30130 time to create 1 rle with old method : 0.033599138259887695 create new chi : 31.40547752380371 time to delete rle : 0.04129600524902344 batch 1 Loaded 72 chid ids of type : 4855 Number RLEs to save : 43293 TO DO : save crop sub photo not yet done ! save time : 2.461250066757202 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 47.53307628631592 time for calcul the mask position with numpy : 0.09506940841674805 nb_pixel_total : 928456 time to create 1 rle with new method : 1.657165288925171 time for calcul the mask position with numpy : 0.062485694885253906 nb_pixel_total : 54366 time to create 1 rle with old method : 0.06281685829162598 time for calcul the mask position with numpy : 0.0611422061920166 nb_pixel_total : 41384 time to create 1 rle with old method : 0.05016350746154785 time for calcul the mask position with numpy : 0.060309648513793945 nb_pixel_total : 1001 time to create 1 rle with old method : 0.0010106563568115234 time for calcul the mask position with numpy : 0.06097078323364258 nb_pixel_total : 4516 time to create 1 rle with old method : 0.005543947219848633 time for calcul the mask position with numpy : 0.06477522850036621 nb_pixel_total : 119741 time to create 1 rle with old method : 0.13828682899475098 time for calcul the mask position with numpy : 0.06170964241027832 nb_pixel_total : 19216 time to create 1 rle with old method : 0.01969766616821289 time for calcul the mask position with numpy : 0.06048297882080078 nb_pixel_total : 2982 time to create 1 rle with old method : 0.0032660961151123047 time for calcul the mask position with numpy : 0.06354284286499023 nb_pixel_total : 437363 time to create 1 rle with new method : 1.2561380863189697 time for calcul the mask position with numpy : 0.6800377368927002 nb_pixel_total : 4229706 time to create 1 rle with new method : 1.3497295379638672 time for calcul the mask position with numpy : 0.06270933151245117 nb_pixel_total : 18784 time to create 1 rle with old method : 0.021544456481933594 time for calcul the mask position with numpy : 0.07641005516052246 nb_pixel_total : 1253174 time to create 1 rle with new method : 1.9740846157073975 time for calcul the mask position with numpy : 0.07314062118530273 nb_pixel_total : 5678 time to create 1 rle with old method : 0.02852010726928711 time for calcul the mask position with numpy : 0.0798642635345459 nb_pixel_total : 19761 time to create 1 rle with old method : 0.028814315795898438 time for calcul the mask position with numpy : 0.48578572273254395 nb_pixel_total : 5843072 time to create 1 rle with new method : 3.402010202407837 create new chi : 12.31651520729065 time to delete rle : 0.004242658615112305 batch 1 Loaded 15 chid ids of type : 4855 Number RLEs to save : 27087 TO DO : save crop sub photo not yet done ! save time : 1.5473213195800781 nb_obj : 10 nb_hashtags : 5 time to prepare the origin masks : 28.106404781341553 time for calcul the mask position with numpy : 0.2283632755279541 nb_pixel_total : 2271200 time to create 1 rle with new method : 2.109678030014038 time for calcul the mask position with numpy : 0.06740021705627441 nb_pixel_total : 6308 time to create 1 rle with old method : 0.007275104522705078 time for calcul the mask position with numpy : 0.06421113014221191 nb_pixel_total : 30450 time to create 1 rle with old method : 0.03330683708190918 time for calcul the mask position with numpy : 0.08074760437011719 nb_pixel_total : 1159531 time to create 1 rle with new method : 2.1227481365203857 time for calcul the mask position with numpy : 0.06518173217773438 nb_pixel_total : 31127 time to create 1 rle with old method : 0.03460288047790527 time for calcul the mask position with numpy : 0.06926751136779785 nb_pixel_total : 92697 time to create 1 rle with old method : 0.10932207107543945 time for calcul the mask position with numpy : 0.09087181091308594 nb_pixel_total : 114450 time to create 1 rle with old method : 0.1280825138092041 time for calcul the mask position with numpy : 0.0699002742767334 nb_pixel_total : 254386 time to create 1 rle with new method : 1.696028709411621 time for calcul the mask position with numpy : 0.085601806640625 nb_pixel_total : 1565783 time to create 1 rle with new method : 1.0176398754119873 time for calcul the mask position with numpy : 0.09987115859985352 nb_pixel_total : 1139049 time to create 1 rle with new method : 2.1661300659179688 time for calcul the mask position with numpy : 0.6447501182556152 nb_pixel_total : 6314219 time to create 1 rle with new method : 2.4571692943573 create new chi : 13.808637142181396 time to delete rle : 0.006878852844238281 batch 1 Loaded 11 chid ids of type : 4855 Number RLEs to save : 20424 TO DO : save crop sub photo not yet done ! save time : 1.166886568069458 map_output_result : {1356331931: (0.49799030460068633, 'Should be the crop_list due to order', 0.6337069953671609), 1356331923: (0.49799030460068633, 'Should be the crop_list due to order', 0.501173702954605), 1356331918: (0.49799030460068633, 'Should be the crop_list due to order', 0.5881593313013342), 1356331909: (0.49799030460068633, 'Should be the crop_list due to order', 0.239939686908857), 1356331907: (0.49799030460068633, 'Should be the crop_list due to order', 0.5269718064714745)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 5 /1356331931.Didn't retrieve data . /1356331923.Didn't retrieve data . /1356331918.Didn't retrieve data . /1356331909.Didn't retrieve data . /1356331907.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.013427734375 save_final save missing photos in datou_result : time spend for datou_step_exec : 348.1556372642517 time spend to save output : 0.1886610984802246 total time spend for step 5 : 348.34429836273193 step6:crop_condition Tue May 6 12:11:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 5 ! batch 1 Loaded 209 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! About to upload 23 photos upload in portfolio : 3015255 init cache_photo without model_param we have 23 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526322_652770 we have uploaded 23 photos in the portfolio 3015255 time of upload the photos Elapsed time : 6.147515058517456 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 3015255 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526340_652770 we have uploaded 16 photos in the portfolio 3015255 time of upload the photos Elapsed time : 3.808741331100464 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 3015255 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526350_652770 we have uploaded 14 photos in the portfolio 3015255 time of upload the photos Elapsed time : 4.3127381801605225 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526358_652770 we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.5979092121124268 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 30 About to insert : list_path_to_insert length 30 new photo from crops ! About to upload 30 photos upload in portfolio : 3015255 init cache_photo without model_param we have 30 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526387_652770 we have uploaded 30 photos in the portfolio 3015255 time of upload the photos Elapsed time : 8.702577352523804 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 3015255 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526399_652770 we have uploaded 2 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.8302559852600098 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 23 About to insert : list_path_to_insert length 23 new photo from crops ! About to upload 23 photos upload in portfolio : 3015255 init cache_photo without model_param we have 23 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526432_652770 we have uploaded 23 photos in the portfolio 3015255 time of upload the photos Elapsed time : 5.921215772628784 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3015255 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526445_652770 we have uploaded 3 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.9582512378692627 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 30 About to insert : list_path_to_insert length 30 new photo from crops ! About to upload 30 photos upload in portfolio : 3015255 init cache_photo without model_param we have 30 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1746526564_652770 we have uploaded 30 photos in the portfolio 3015255 time of upload the photos Elapsed time : 8.893462657928467 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 142 /1356338346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338776Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338781Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338784Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338787Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338796Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338799Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338830Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338850Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338941Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338944Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338953Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338954Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1356338955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 431 time used for this insertion : 0.05108499526977539 save_final save missing photos in datou_result : time spend for datou_step_exec : 302.0784864425659 time spend to save output : 0.13025546073913574 total time spend for step 6 : 302.20874190330505 step7:ventilate_hashtags_in_portfolio Tue May 6 12:16:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 22713613 get user id for portfolio 22713613 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713613 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713613 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713613 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713613 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://www.fotonower.com/velours/22714185,22714186,22714187,22714188,22714189,22714194,22714196,22714197,22714198,22714199,22714200,22714201,22714203,22714204,22714205,22714206,22714207,22714208,22714209,22714210?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713613 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 1 /22713613. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.013897895812988281 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.247067928314209 time spend to save output : 0.01418924331665039 total time spend for step 7 : 4.261257171630859 step8:final Tue May 6 12:16:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1356331931: ('0.4894512328470049',), 1356331923: ('0.4894512328470049',), 1356331918: ('0.4894512328470049',), 1356331909: ('0.4894512328470049',), 1356331907: ('0.4894512328470049',)} new output for save of step final : {1356331931: ('0.4894512328470049',), 1356331923: ('0.4894512328470049',), 1356331918: ('0.4894512328470049',), 1356331909: ('0.4894512328470049',), 1356331907: ('0.4894512328470049',)} [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 5 /1356331931.Didn't retrieve data . /1356331923.Didn't retrieve data . /1356331918.Didn't retrieve data . /1356331909.Didn't retrieve data . /1356331907.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 15 time used for this insertion : 0.012564420700073242 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11342167854309082 time spend to save output : 0.013073444366455078 total time spend for step 8 : 0.1264951229095459 step9:velours_tree Tue May 6 12:16:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 1.803067922592163 time spend to save output : 6.246566772460938e-05 total time spend for step 9 : 1.8031303882598877 step10:send_mail_cod Tue May 6 12:16:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P22713613_06-05-2025_12_16_22.pdf 22714159 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227141591746526582 22714160 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227141601746526584 22714161 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227141611746526585 22714162 change filename to text .imagette227141621746526586 22714163 imagette227141631746526586 22714164 imagette227141641746526586 22714165 imagette227141651746526586 22714166 imagette227141661746526586 22714167 imagette227141671746526586 22714168 imagette227141681746526586 22714169 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227141691746526586 22714170 change filename to text .change filename to text .imagette227141701746526589 22714171 imagette227141711746526589 22714172 imagette227141721746526589 22714173 imagette227141731746526589 22714174 imagette227141741746526589 22714175 imagette227141751746526589 22714176 imagette227141761746526589 22714177 change filename to text .change filename to text .change filename to text .imagette227141771746526589 22714179 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette227141791746526589 22714180 imagette227141801746526591 22714181 imagette227141811746526591 22714182 imagette227141821746526591 22714183 imagette227141831746526591 22714184 imagette227141841746526591 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=22713613 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/22714185,22714186,22714187,22714188,22714189,22714194,22714196,22714197,22714198,22714199,22714200,22714201,22714203,22714204,22714205,22714206,22714207,22714208,22714209,22714210?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=22713613 your option no_mail is active, we will not send the real mail to your client args[1356331931] : ((1356331931, 2243.3896209577115, 492609224), (1356331931, -0.8329206454875222, 501862349), '0.4894512328470049') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331923] : ((1356331923, 533.6080237762293, 492688767), (1356331923, -0.2789053102806118, 496442774), '0.4894512328470049') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331918] : ((1356331918, 2704.1605869591826, 492609224), (1356331918, -0.506913594209797, 501862349), '0.4894512328470049') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331909] : ((1356331909, 1298.6623445949415, 2107751945), (1356331909, -0.7265442680348, 501862349), '0.4894512328470049') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1356331907] : ((1356331907, 500.26206196950665, 492688767), (1356331907, -1.06931458826972, 501862349), '0.4894512328470049') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.4894512328470049 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713613_06-05-2025_12_16_22.pdf results_Auto_P22713613_06-05-2025_12_16_22.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713613_06-05-2025_12_16_22.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','22713613','results_Auto_P22713613_06-05-2025_12_16_22.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713613_06-05-2025_12_16_22.pdf','pdf','','2.58','0.4894512328470049') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 5 time used for this insertion : 0.013888359069824219 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.978061199188232 time spend to save output : 0.014212369918823242 total time spend for step 10 : 12.992273569107056 step11:split_time_score Tue May 6 12:16:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('55', 1), ('56', 3), ('57', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 01052025 22713613 Nombre de photos uploadées : 5 / 23040 (0%) 01052025 22713613 Nombre de photos taguées (types de déchets): 0 / 5 (0%) 01052025 22713613 Nombre de photos taguées (volume) : 0 / 5 (0%) elapsed_time : load_data_split_time_score 3.5762786865234375e-06 elapsed_time : order_list_meta_photo_and_scores 8.821487426757812e-06 ????? elapsed_time : fill_and_build_computed_from_old_data 0.00035119056701660156 elapsed_time : insert_dashboard_record_day_entry 0.023457765579223633 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.39026506267119515 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713349_06-05-2025_12_09_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713349 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713349 AND mptpi.`type`=4855 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713611 order by id desc limit 1 Qualite : 0.3116130851449679 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713612_06-05-2025_12_01_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713612 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713612 AND mptpi.`type`=4855 To do Qualite : 0.4894512328470049 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P22713613_06-05-2025_12_16_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 22713613 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22713613 AND mptpi.`type`=4855 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'01052025': {'nb_upload': 5, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1356331931, 1356331923, 1356331918, 1356331909, 1356331907] Looping around the photos to save general results len do output : 1 /22713613Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331931', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331923', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331918', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331909', None, None, None, None, None, '2797355') ('4741', None, None, None, None, None, None, None, '2797355') ('4741', '22713613', '1356331907', None, None, None, None, None, '2797355') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015729188919067383 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.2433714866638184 time spend to save output : 0.015947341918945312 total time spend for step 11 : 1.2593188285827637 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 5 set_done_treatment 474.09user 361.55system 15:11.01elapsed 91%CPU (0avgtext+0avgdata 5355160maxresident)k 5220352inputs+542272outputs (50164major+39969065minor)pagefaults 0swaps