python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4741 -l 30 ' -s datou_current_4741 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2024502 load datou : 4741 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) load thcls load THCL from format json or kwargs add thcl : 3453 in CacheModelConfig load THCL from format json or kwargs add thcl : 3783 in CacheModelConfig load pdts add pdt : 5732 in CacheModelConfig add pdt : 5972 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4741, datou_cur_ids : ['3665904'] with mtr_portfolio_ids : ['26545452'] and first list_photo_ids : [] new path : /proc/2024502/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, mask_detect, blur_detection, brightness, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 3 ; length of list_pids : 3 ; length of list_args : 3 time to download the photos : 0.6339945793151855 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Sep 4 22:01:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10375 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-04 22:01:31.964389: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-04 22:01:31.992613: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-04 22:01:31.994237: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5b20000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-04 22:01:31.994299: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-04 22:01:31.997370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-04 22:01:32.143825: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x8034ab0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-04 22:01:32.143886: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-04 22:01:32.145102: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:01:32.145893: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:01:32.149780: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:01:32.152996: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:01:32.154276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:01:32.157980: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:01:32.159488: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:01:32.167126: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:01:32.168654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:01:32.168746: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:01:32.169535: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-04 22:01:32.169551: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-04 22:01:32.169561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-04 22:01:32.170871: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9610 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-04 22:01:32.641434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:01:32.641535: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:01:32.641556: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:01:32.641575: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:01:32.641592: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:01:32.641608: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:01:32.641625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:01:32.641641: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:01:32.642964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:01:32.645039: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:01:32.645147: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:01:32.645176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:01:32.645201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:01:32.645235: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:01:32.645256: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:01:32.645274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:01:32.645293: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:01:32.646857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:01:32.646900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-04 22:01:32.646912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-04 22:01:32.646922: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-04 22:01:32.648742: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9610 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3783 thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5972, 'learn_entrant_syctomXV_111023', 16384, 25088, 'learn_entrant_syctomXV_111023', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 10, 11, 15, 57, 35), datetime.datetime(2023, 10, 11, 15, 57, 35)) {'thcl': {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'], 'list_hashtags_csv': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'svm_hashtag_type_desc': 5972, 'photo_desc_type': 5972, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_entrant_syctomXV_111023 NUM_CLASSES 11 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_entrant_syctomXV_111023 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-04 22:01:47.001124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:01:47.228389: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_entrant_syctomXV_111023 /data/models_weight/learn_entrant_syctomXV_111023/mask_model.h5 size_local : 256052544 size in s3 : 256052544 create time local : 2023-10-26 16:36:43 create time in s3 : 2023-10-11 13:57:17 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3120.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3120.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 19) min: 0.00000 max: 3120.00000 nb d'objets trouves : 12 Detection mask done ! Trying to reset tf kernel 2024646 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1088 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 6377 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3783 Catched exception ! Connect or reconnect ! thcls : [{'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 3783, 'mtr_user_id': 31, 'name': 'learn_entrant_syctomXV_111023', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,sac,textiles,verre,organique,dasri,masque,encombrant,autre_emballage,autre_non_emballage,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4853, 'photo_desc_type': 5972, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5972 ['background', 'sac', 'textiles', 'verre', 'organique', 'dasri', 'masque', 'encombrant', 'autre_emballage', 'autre_non_emballage', 'environnement'] time for calcul the mask position with numpy : 0.0041120052337646484 nb_pixel_total : 133838 time to create 1 rle with old method : 0.14899945259094238 length of segment : 598 time for calcul the mask position with numpy : 0.0019156932830810547 nb_pixel_total : 113207 time to create 1 rle with old method : 0.12376666069030762 length of segment : 476 time for calcul the mask position with numpy : 0.0006546974182128906 nb_pixel_total : 17101 time to create 1 rle with old method : 0.019565820693969727 length of segment : 186 time for calcul the mask position with numpy : 0.0007376670837402344 nb_pixel_total : 34915 time to create 1 rle with old method : 0.038933753967285156 length of segment : 292 time for calcul the mask position with numpy : 0.0007660388946533203 nb_pixel_total : 30563 time to create 1 rle with old method : 0.033740997314453125 length of segment : 254 time for calcul the mask position with numpy : 0.0007281303405761719 nb_pixel_total : 30553 time to create 1 rle with old method : 0.0336759090423584 length of segment : 158 time for calcul the mask position with numpy : 0.0016760826110839844 nb_pixel_total : 67350 time to create 1 rle with old method : 0.07413482666015625 length of segment : 195 time for calcul the mask position with numpy : 0.0002853870391845703 nb_pixel_total : 12212 time to create 1 rle with old method : 0.013974905014038086 length of segment : 194 time for calcul the mask position with numpy : 0.0014247894287109375 nb_pixel_total : 65398 time to create 1 rle with old method : 0.07143783569335938 length of segment : 142 time for calcul the mask position with numpy : 0.0024924278259277344 nb_pixel_total : 134467 time to create 1 rle with old method : 0.14906954765319824 length of segment : 374 time for calcul the mask position with numpy : 0.03313183784484863 nb_pixel_total : 909161 time to create 1 rle with new method : 0.054433345794677734 length of segment : 1346 time for calcul the mask position with numpy : 0.0006315708160400391 nb_pixel_total : 27728 time to create 1 rle with old method : 0.03350424766540527 length of segment : 330 time for calcul the mask position with numpy : 0.014485836029052734 nb_pixel_total : 568450 time to create 1 rle with new method : 0.07010841369628906 length of segment : 1348 time for calcul the mask position with numpy : 0.0007939338684082031 nb_pixel_total : 23346 time to create 1 rle with old method : 0.026622295379638672 length of segment : 182 time for calcul the mask position with numpy : 0.0019609928131103516 nb_pixel_total : 89844 time to create 1 rle with old method : 0.10068583488464355 length of segment : 697 time for calcul the mask position with numpy : 0.0008893013000488281 nb_pixel_total : 33429 time to create 1 rle with old method : 0.04191446304321289 length of segment : 337 time for calcul the mask position with numpy : 0.04831051826477051 nb_pixel_total : 981515 time to create 1 rle with new method : 0.11260294914245605 length of segment : 1647 time for calcul the mask position with numpy : 0.00045108795166015625 nb_pixel_total : 13877 time to create 1 rle with old method : 0.02390289306640625 length of segment : 181 time for calcul the mask position with numpy : 0.001827239990234375 nb_pixel_total : 98839 time to create 1 rle with old method : 0.1210482120513916 length of segment : 646 time for calcul the mask position with numpy : 0.0029342174530029297 nb_pixel_total : 156658 time to create 1 rle with new method : 0.008040189743041992 length of segment : 407 time for calcul the mask position with numpy : 0.002008676528930664 nb_pixel_total : 40038 time to create 1 rle with old method : 0.04417085647583008 length of segment : 393 time for calcul the mask position with numpy : 0.0028989315032958984 nb_pixel_total : 59230 time to create 1 rle with old method : 0.06675362586975098 length of segment : 344 time for calcul the mask position with numpy : 0.008951425552368164 nb_pixel_total : 317298 time to create 1 rle with new method : 0.014394521713256836 length of segment : 889 time for calcul the mask position with numpy : 0.00188446044921875 nb_pixel_total : 22345 time to create 1 rle with old method : 0.025574684143066406 length of segment : 230 time for calcul the mask position with numpy : 0.0018925666809082031 nb_pixel_total : 47104 time to create 1 rle with old method : 0.05341935157775879 length of segment : 321 time for calcul the mask position with numpy : 0.0007619857788085938 nb_pixel_total : 20924 time to create 1 rle with old method : 0.0232391357421875 length of segment : 152 time for calcul the mask position with numpy : 0.004060268402099609 nb_pixel_total : 110906 time to create 1 rle with old method : 0.12439608573913574 length of segment : 248 time for calcul the mask position with numpy : 0.0015497207641601562 nb_pixel_total : 46508 time to create 1 rle with old method : 0.05559086799621582 length of segment : 231 time for calcul the mask position with numpy : 0.001216888427734375 nb_pixel_total : 34053 time to create 1 rle with old method : 0.04244112968444824 length of segment : 515 time for calcul the mask position with numpy : 0.0032968521118164062 nb_pixel_total : 66430 time to create 1 rle with old method : 0.0717318058013916 length of segment : 323 time for calcul the mask position with numpy : 0.0009644031524658203 nb_pixel_total : 19100 time to create 1 rle with old method : 0.020767688751220703 length of segment : 148 time for calcul the mask position with numpy : 0.0014183521270751953 nb_pixel_total : 42802 time to create 1 rle with old method : 0.048073768615722656 length of segment : 347 time for calcul the mask position with numpy : 0.0008478164672851562 nb_pixel_total : 22526 time to create 1 rle with old method : 0.025166749954223633 length of segment : 177 time for calcul the mask position with numpy : 0.0005507469177246094 nb_pixel_total : 8720 time to create 1 rle with old method : 0.010035037994384766 length of segment : 180 time for calcul the mask position with numpy : 0.0009248256683349609 nb_pixel_total : 15808 time to create 1 rle with old method : 0.018165111541748047 length of segment : 247 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 4059 time to create 1 rle with old method : 0.00501704216003418 length of segment : 83 time for calcul the mask position with numpy : 0.0014536380767822266 nb_pixel_total : 38111 time to create 1 rle with old method : 0.04521036148071289 length of segment : 238 time for calcul the mask position with numpy : 0.0006589889526367188 nb_pixel_total : 21250 time to create 1 rle with old method : 0.024495363235473633 length of segment : 120 time for calcul the mask position with numpy : 0.0005588531494140625 nb_pixel_total : 6072 time to create 1 rle with old method : 0.009879350662231445 length of segment : 101 time for calcul the mask position with numpy : 0.004558086395263672 nb_pixel_total : 106611 time to create 1 rle with old method : 0.12460613250732422 length of segment : 411 time for calcul the mask position with numpy : 0.0018146038055419922 nb_pixel_total : 27132 time to create 1 rle with old method : 0.04089951515197754 length of segment : 288 time for calcul the mask position with numpy : 0.0008046627044677734 nb_pixel_total : 16358 time to create 1 rle with old method : 0.019072532653808594 length of segment : 175 time for calcul the mask position with numpy : 0.0009310245513916016 nb_pixel_total : 27365 time to create 1 rle with old method : 0.031351327896118164 length of segment : 297 time for calcul the mask position with numpy : 0.0038864612579345703 nb_pixel_total : 122251 time to create 1 rle with old method : 0.14034295082092285 length of segment : 432 time for calcul the mask position with numpy : 0.002596139907836914 nb_pixel_total : 20583 time to create 1 rle with old method : 0.023468971252441406 length of segment : 462 time for calcul the mask position with numpy : 0.014507055282592773 nb_pixel_total : 586558 time to create 1 rle with new method : 0.0802145004272461 length of segment : 549 time for calcul the mask position with numpy : 0.3510866165161133 nb_pixel_total : 3224498 time to create 1 rle with new method : 0.5659892559051514 length of segment : 2284 time for calcul the mask position with numpy : 0.004166126251220703 nb_pixel_total : 185508 time to create 1 rle with new method : 0.023253202438354492 length of segment : 617 time for calcul the mask position with numpy : 0.0009164810180664062 nb_pixel_total : 63189 time to create 1 rle with old method : 0.07176685333251953 length of segment : 319 time for calcul the mask position with numpy : 0.00038909912109375 nb_pixel_total : 13680 time to create 1 rle with old method : 0.015573501586914062 length of segment : 160 time for calcul the mask position with numpy : 0.10300970077514648 nb_pixel_total : 2025647 time to create 1 rle with new method : 0.2249000072479248 length of segment : 2233 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 27308 time to create 1 rle with old method : 0.030849933624267578 length of segment : 224 time for calcul the mask position with numpy : 0.20197463035583496 nb_pixel_total : 2941419 time to create 1 rle with new method : 0.2392880916595459 length of segment : 1151 time for calcul the mask position with numpy : 0.0006060600280761719 nb_pixel_total : 22058 time to create 1 rle with old method : 0.02479839324951172 length of segment : 302 time for calcul the mask position with numpy : 0.3102588653564453 nb_pixel_total : 2996197 time to create 1 rle with new method : 0.27320241928100586 length of segment : 1331 length of segment : 0 time spent for convertir_results : 10.574478149414062 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 56 chid ids of type : 4854 Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 44.58282542228699 time spend to save output : 0.3112337589263916 total time spend for step 1 : 44.89405918121338 step2:mask_detect Thu Sep 4 22:02:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 6377 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-09-04 22:02:16.633127: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-04 22:02:16.660806: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-04 22:02:16.663758: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5b24000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-04 22:02:16.663840: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-04 22:02:16.667883: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-04 22:02:16.816051: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x96a98d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-04 22:02:16.816116: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-04 22:02:16.817424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:02:16.817831: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:02:16.820776: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:02:16.823445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:02:16.823934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:02:16.827502: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:02:16.828605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:02:16.833036: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:02:16.834274: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:02:16.834344: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:02:16.834987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-04 22:02:16.835002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-04 22:02:16.835010: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-04 22:02:16.836154: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5851 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-09-04 22:02:17.052646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:02:17.052740: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:02:17.052763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:02:17.052784: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:02:17.052804: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:02:17.052824: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:02:17.052844: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:02:17.052864: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:02:17.054145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:02:17.055328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-04 22:02:17.055367: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-04 22:02:17.055389: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:02:17.055408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-04 22:02:17.055426: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-04 22:02:17.055445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-04 22:02:17.055464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-04 22:02:17.055483: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-04 22:02:17.056795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-04 22:02:17.056828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-04 22:02:17.056838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-04 22:02:17.056848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-04 22:02:17.058152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5851 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl3453 thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5732, 'mask_refus_amiens_050123', 16384, 25088, 'mask_refus_amiens_050123', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 1, 5, 13, 23, 59), datetime.datetime(2023, 1, 5, 13, 23, 59)) {'thcl': {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'svm_hashtag_type_desc': 5732, 'photo_desc_type': 5732, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME mask_refus_amiens_050123 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : mask_refus_amiens_050123 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-09-04 22:02:28.161479: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-04 22:02:28.351364: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/mask_refus_amiens_050123 /data/models_weight/mask_refus_amiens_050123/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2023-01-05 13:44:07 create time in s3 : 2023-01-05 12:23:52 mask_model.h5 already exist and didn't need to update list_images length : 3 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3120.00000 nb d'objets trouves : 68 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3120.00000 nb d'objets trouves : 75 NEW PHOTO Processing 1 images image shape: (3120, 3120, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 3120.00000 nb d'objets trouves : 9 Detection mask done ! Trying to reset tf kernel 2027003 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5086 tf kernel not reseted sub process len(results) : 3 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 3 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10375 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl3453 Catched exception ! Connect or reconnect ! thcls : [{'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 3453, 'mtr_user_id': 31, 'name': 'mask_refus_amiens_050123', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 4485, 'photo_desc_type': 5732, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5732 ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] time for calcul the mask position with numpy : 0.0017888545989990234 nb_pixel_total : 52179 time to create 1 rle with old method : 0.06776785850524902 length of segment : 298 time for calcul the mask position with numpy : 0.01668715476989746 nb_pixel_total : 721215 time to create 1 rle with new method : 0.06162691116333008 length of segment : 891 time for calcul the mask position with numpy : 0.0005440711975097656 nb_pixel_total : 18332 time to create 1 rle with old method : 0.023417234420776367 length of segment : 160 time for calcul the mask position with numpy : 0.003566265106201172 nb_pixel_total : 114239 time to create 1 rle with old method : 0.1448967456817627 length of segment : 570 time for calcul the mask position with numpy : 0.02654266357421875 nb_pixel_total : 1363311 time to create 1 rle with new method : 1.193847417831421 length of segment : 1229 time for calcul the mask position with numpy : 0.0015735626220703125 nb_pixel_total : 80986 time to create 1 rle with old method : 0.10470390319824219 length of segment : 361 time for calcul the mask position with numpy : 0.001878976821899414 nb_pixel_total : 105976 time to create 1 rle with old method : 0.11696457862854004 length of segment : 463 time for calcul the mask position with numpy : 0.003703594207763672 nb_pixel_total : 143889 time to create 1 rle with old method : 0.157578706741333 length of segment : 959 time for calcul the mask position with numpy : 0.004492282867431641 nb_pixel_total : 257264 time to create 1 rle with new method : 0.010558366775512695 length of segment : 729 time for calcul the mask position with numpy : 0.00042748451232910156 nb_pixel_total : 14086 time to create 1 rle with old method : 0.016374826431274414 length of segment : 127 time for calcul the mask position with numpy : 0.0034151077270507812 nb_pixel_total : 164575 time to create 1 rle with new method : 0.008713722229003906 length of segment : 339 time for calcul the mask position with numpy : 0.0028350353240966797 nb_pixel_total : 134295 time to create 1 rle with old method : 0.14880919456481934 length of segment : 495 time for calcul the mask position with numpy : 0.0006947517395019531 nb_pixel_total : 26784 time to create 1 rle with old method : 0.02855992317199707 length of segment : 183 time for calcul the mask position with numpy : 0.0015418529510498047 nb_pixel_total : 60021 time to create 1 rle with old method : 0.06590032577514648 length of segment : 268 time for calcul the mask position with numpy : 0.0005733966827392578 nb_pixel_total : 20663 time to create 1 rle with old method : 0.022764205932617188 length of segment : 285 time for calcul the mask position with numpy : 0.002947568893432617 nb_pixel_total : 155387 time to create 1 rle with new method : 0.008437156677246094 length of segment : 591 time for calcul the mask position with numpy : 0.0002739429473876953 nb_pixel_total : 9291 time to create 1 rle with old method : 0.01093149185180664 length of segment : 110 time for calcul the mask position with numpy : 0.002343416213989258 nb_pixel_total : 94068 time to create 1 rle with old method : 0.12370467185974121 length of segment : 463 time for calcul the mask position with numpy : 0.0005509853363037109 nb_pixel_total : 10611 time to create 1 rle with old method : 0.012217521667480469 length of segment : 370 time for calcul the mask position with numpy : 0.007829427719116211 nb_pixel_total : 327752 time to create 1 rle with new method : 0.02002120018005371 length of segment : 575 time for calcul the mask position with numpy : 0.003712892532348633 nb_pixel_total : 139465 time to create 1 rle with old method : 0.1505568027496338 length of segment : 381 time for calcul the mask position with numpy : 0.007742404937744141 nb_pixel_total : 465330 time to create 1 rle with new method : 0.024932861328125 length of segment : 647 time for calcul the mask position with numpy : 0.0003628730773925781 nb_pixel_total : 15119 time to create 1 rle with old method : 0.017302989959716797 length of segment : 209 time for calcul the mask position with numpy : 0.0037109851837158203 nb_pixel_total : 188237 time to create 1 rle with new method : 0.009391069412231445 length of segment : 547 time for calcul the mask position with numpy : 0.0006325244903564453 nb_pixel_total : 21759 time to create 1 rle with old method : 0.024042844772338867 length of segment : 184 time for calcul the mask position with numpy : 0.0006525516510009766 nb_pixel_total : 30307 time to create 1 rle with old method : 0.034078121185302734 length of segment : 231 time for calcul the mask position with numpy : 0.0006420612335205078 nb_pixel_total : 38495 time to create 1 rle with old method : 0.04285764694213867 length of segment : 157 time for calcul the mask position with numpy : 0.0021271705627441406 nb_pixel_total : 105407 time to create 1 rle with old method : 0.11783266067504883 length of segment : 238 time for calcul the mask position with numpy : 0.0009813308715820312 nb_pixel_total : 29548 time to create 1 rle with old method : 0.04494595527648926 length of segment : 274 time for calcul the mask position with numpy : 0.0014450550079345703 nb_pixel_total : 31531 time to create 1 rle with old method : 0.046118736267089844 length of segment : 258 time for calcul the mask position with numpy : 0.002462625503540039 nb_pixel_total : 108086 time to create 1 rle with old method : 0.1492142677307129 length of segment : 396 time for calcul the mask position with numpy : 0.0023317337036132812 nb_pixel_total : 64170 time to create 1 rle with old method : 0.09050607681274414 length of segment : 333 time for calcul the mask position with numpy : 0.0017147064208984375 nb_pixel_total : 43256 time to create 1 rle with old method : 0.05894184112548828 length of segment : 180 time for calcul the mask position with numpy : 0.0030260086059570312 nb_pixel_total : 118391 time to create 1 rle with old method : 0.1506824493408203 length of segment : 329 time for calcul the mask position with numpy : 0.011321544647216797 nb_pixel_total : 418932 time to create 1 rle with new method : 0.03630948066711426 length of segment : 588 time for calcul the mask position with numpy : 0.0007441043853759766 nb_pixel_total : 19687 time to create 1 rle with old method : 0.029212474822998047 length of segment : 174 time for calcul the mask position with numpy : 0.0001881122589111328 nb_pixel_total : 3737 time to create 1 rle with old method : 0.007765769958496094 length of segment : 55 time for calcul the mask position with numpy : 0.004839420318603516 nb_pixel_total : 159346 time to create 1 rle with new method : 0.01344919204711914 length of segment : 628 time for calcul the mask position with numpy : 0.0034368038177490234 nb_pixel_total : 142425 time to create 1 rle with old method : 0.15776300430297852 length of segment : 615 time for calcul the mask position with numpy : 0.0026273727416992188 nb_pixel_total : 123396 time to create 1 rle with old method : 0.14019083976745605 length of segment : 387 time for calcul the mask position with numpy : 0.00186920166015625 nb_pixel_total : 80013 time to create 1 rle with old method : 0.08830761909484863 length of segment : 373 time for calcul the mask position with numpy : 0.018265962600708008 nb_pixel_total : 661792 time to create 1 rle with new method : 0.12174367904663086 length of segment : 1599 time for calcul the mask position with numpy : 0.0047457218170166016 nb_pixel_total : 205259 time to create 1 rle with new method : 0.01248478889465332 length of segment : 629 time for calcul the mask position with numpy : 0.0010919570922851562 nb_pixel_total : 55712 time to create 1 rle with old method : 0.0658121109008789 length of segment : 328 time for calcul the mask position with numpy : 0.0005195140838623047 nb_pixel_total : 17857 time to create 1 rle with old method : 0.02013874053955078 length of segment : 202 time for calcul the mask position with numpy : 0.0010559558868408203 nb_pixel_total : 55095 time to create 1 rle with old method : 0.06355524063110352 length of segment : 153 time for calcul the mask position with numpy : 0.013153076171875 nb_pixel_total : 461086 time to create 1 rle with new method : 0.06457734107971191 length of segment : 1284 time for calcul the mask position with numpy : 0.010765314102172852 nb_pixel_total : 533383 time to create 1 rle with new method : 0.03302955627441406 length of segment : 1238 time for calcul the mask position with numpy : 0.0012791156768798828 nb_pixel_total : 41914 time to create 1 rle with old method : 0.0466008186340332 length of segment : 300 time for calcul the mask position with numpy : 0.002334117889404297 nb_pixel_total : 98277 time to create 1 rle with old method : 0.11065435409545898 length of segment : 475 time for calcul the mask position with numpy : 0.0005557537078857422 nb_pixel_total : 18660 time to create 1 rle with old method : 0.021559715270996094 length of segment : 142 time for calcul the mask position with numpy : 0.0013203620910644531 nb_pixel_total : 28429 time to create 1 rle with old method : 0.03463244438171387 length of segment : 290 time for calcul the mask position with numpy : 0.0019922256469726562 nb_pixel_total : 84579 time to create 1 rle with old method : 0.1022486686706543 length of segment : 335 time for calcul the mask position with numpy : 0.001218557357788086 nb_pixel_total : 52351 time to create 1 rle with old method : 0.05968594551086426 length of segment : 373 time for calcul the mask position with numpy : 0.0009813308715820312 nb_pixel_total : 47710 time to create 1 rle with old method : 0.05474448204040527 length of segment : 209 time for calcul the mask position with numpy : 0.0010516643524169922 nb_pixel_total : 50898 time to create 1 rle with old method : 0.05813169479370117 length of segment : 230 time for calcul the mask position with numpy : 0.00045037269592285156 nb_pixel_total : 22926 time to create 1 rle with old method : 0.02595806121826172 length of segment : 161 time for calcul the mask position with numpy : 0.0003941059112548828 nb_pixel_total : 23737 time to create 1 rle with old method : 0.02721381187438965 length of segment : 169 time for calcul the mask position with numpy : 0.0016028881072998047 nb_pixel_total : 69110 time to create 1 rle with old method : 0.07531857490539551 length of segment : 369 time for calcul the mask position with numpy : 0.0070989131927490234 nb_pixel_total : 344293 time to create 1 rle with new method : 0.02051067352294922 length of segment : 495 time for calcul the mask position with numpy : 0.0026426315307617188 nb_pixel_total : 130590 time to create 1 rle with old method : 0.14255714416503906 length of segment : 394 time for calcul the mask position with numpy : 0.0007181167602539062 nb_pixel_total : 33814 time to create 1 rle with old method : 0.03708529472351074 length of segment : 212 time for calcul the mask position with numpy : 0.0009412765502929688 nb_pixel_total : 44463 time to create 1 rle with old method : 0.049283504486083984 length of segment : 328 time for calcul the mask position with numpy : 0.0013899803161621094 nb_pixel_total : 70201 time to create 1 rle with old method : 0.07816576957702637 length of segment : 202 time for calcul the mask position with numpy : 0.0009121894836425781 nb_pixel_total : 41084 time to create 1 rle with old method : 0.04475069046020508 length of segment : 471 time for calcul the mask position with numpy : 0.0010714530944824219 nb_pixel_total : 36457 time to create 1 rle with old method : 0.040457963943481445 length of segment : 226 time for calcul the mask position with numpy : 0.0011131763458251953 nb_pixel_total : 54482 time to create 1 rle with old method : 0.05888962745666504 length of segment : 210 time for calcul the mask position with numpy : 0.00550532341003418 nb_pixel_total : 298197 time to create 1 rle with new method : 0.01438593864440918 length of segment : 760 time for calcul the mask position with numpy : 0.0003275871276855469 nb_pixel_total : 11166 time to create 1 rle with old method : 0.012880802154541016 length of segment : 173 time for calcul the mask position with numpy : 0.0024793148040771484 nb_pixel_total : 84395 time to create 1 rle with old method : 0.12130403518676758 length of segment : 324 time for calcul the mask position with numpy : 0.0007448196411132812 nb_pixel_total : 18956 time to create 1 rle with old method : 0.02156805992126465 length of segment : 298 time for calcul the mask position with numpy : 0.0013701915740966797 nb_pixel_total : 44479 time to create 1 rle with old method : 0.049482107162475586 length of segment : 406 time for calcul the mask position with numpy : 0.0012459754943847656 nb_pixel_total : 54686 time to create 1 rle with old method : 0.0632627010345459 length of segment : 334 time for calcul the mask position with numpy : 0.0005433559417724609 nb_pixel_total : 17404 time to create 1 rle with old method : 0.019798994064331055 length of segment : 156 time for calcul the mask position with numpy : 0.01122736930847168 nb_pixel_total : 480052 time to create 1 rle with new method : 0.02364492416381836 length of segment : 860 time for calcul the mask position with numpy : 0.001773834228515625 nb_pixel_total : 60616 time to create 1 rle with old method : 0.0668938159942627 length of segment : 515 time for calcul the mask position with numpy : 0.0012226104736328125 nb_pixel_total : 30969 time to create 1 rle with old method : 0.04527425765991211 length of segment : 248 time for calcul the mask position with numpy : 0.0019440650939941406 nb_pixel_total : 78422 time to create 1 rle with old method : 0.08552932739257812 length of segment : 445 time for calcul the mask position with numpy : 0.0005893707275390625 nb_pixel_total : 20963 time to create 1 rle with old method : 0.023034095764160156 length of segment : 223 time for calcul the mask position with numpy : 0.0006384849548339844 nb_pixel_total : 19681 time to create 1 rle with old method : 0.022127866744995117 length of segment : 469 time for calcul the mask position with numpy : 0.00040650367736816406 nb_pixel_total : 12941 time to create 1 rle with old method : 0.015417098999023438 length of segment : 169 time for calcul the mask position with numpy : 0.00047898292541503906 nb_pixel_total : 17573 time to create 1 rle with old method : 0.020268678665161133 length of segment : 198 time for calcul the mask position with numpy : 0.0008497238159179688 nb_pixel_total : 27941 time to create 1 rle with old method : 0.03153061866760254 length of segment : 295 time for calcul the mask position with numpy : 0.004180431365966797 nb_pixel_total : 154744 time to create 1 rle with new method : 0.011662721633911133 length of segment : 630 time for calcul the mask position with numpy : 0.004689693450927734 nb_pixel_total : 268362 time to create 1 rle with new method : 0.013410568237304688 length of segment : 467 time for calcul the mask position with numpy : 0.0009374618530273438 nb_pixel_total : 38656 time to create 1 rle with old method : 0.04351949691772461 length of segment : 253 time for calcul the mask position with numpy : 0.00957942008972168 nb_pixel_total : 493047 time to create 1 rle with new method : 0.0210723876953125 length of segment : 711 time for calcul the mask position with numpy : 0.0015704631805419922 nb_pixel_total : 68376 time to create 1 rle with old method : 0.07757043838500977 length of segment : 453 time for calcul the mask position with numpy : 0.0010197162628173828 nb_pixel_total : 50520 time to create 1 rle with old method : 0.05546832084655762 length of segment : 207 time for calcul the mask position with numpy : 0.005685091018676758 nb_pixel_total : 240680 time to create 1 rle with new method : 0.018213510513305664 length of segment : 588 time for calcul the mask position with numpy : 0.002053976058959961 nb_pixel_total : 77263 time to create 1 rle with old method : 0.08494234085083008 length of segment : 274 time for calcul the mask position with numpy : 0.0023260116577148438 nb_pixel_total : 84283 time to create 1 rle with old method : 0.09185385704040527 length of segment : 401 time for calcul the mask position with numpy : 0.0014047622680664062 nb_pixel_total : 61092 time to create 1 rle with old method : 0.06625795364379883 length of segment : 352 time for calcul the mask position with numpy : 0.004463672637939453 nb_pixel_total : 181362 time to create 1 rle with new method : 0.012741327285766602 length of segment : 552 time for calcul the mask position with numpy : 0.0010356903076171875 nb_pixel_total : 45721 time to create 1 rle with old method : 0.050866127014160156 length of segment : 165 time for calcul the mask position with numpy : 0.002176523208618164 nb_pixel_total : 66338 time to create 1 rle with old method : 0.07171010971069336 length of segment : 454 time for calcul the mask position with numpy : 0.0036869049072265625 nb_pixel_total : 138463 time to create 1 rle with old method : 0.14759230613708496 length of segment : 803 time for calcul the mask position with numpy : 0.003191232681274414 nb_pixel_total : 106908 time to create 1 rle with old method : 0.1156458854675293 length of segment : 465 time for calcul the mask position with numpy : 0.0009331703186035156 nb_pixel_total : 41443 time to create 1 rle with old method : 0.04549384117126465 length of segment : 286 time for calcul the mask position with numpy : 0.0005631446838378906 nb_pixel_total : 19664 time to create 1 rle with old method : 0.02207350730895996 length of segment : 161 time for calcul the mask position with numpy : 0.003576040267944336 nb_pixel_total : 135202 time to create 1 rle with old method : 0.1451268196105957 length of segment : 486 time for calcul the mask position with numpy : 0.003052949905395508 nb_pixel_total : 125830 time to create 1 rle with old method : 0.13865208625793457 length of segment : 318 time for calcul the mask position with numpy : 0.003336668014526367 nb_pixel_total : 138027 time to create 1 rle with old method : 0.1479811668395996 length of segment : 569 time for calcul the mask position with numpy : 0.0011432170867919922 nb_pixel_total : 25368 time to create 1 rle with old method : 0.02930140495300293 length of segment : 162 time for calcul the mask position with numpy : 0.0007832050323486328 nb_pixel_total : 34245 time to create 1 rle with old method : 0.03673744201660156 length of segment : 228 time for calcul the mask position with numpy : 0.0006129741668701172 nb_pixel_total : 21082 time to create 1 rle with old method : 0.023777008056640625 length of segment : 221 time for calcul the mask position with numpy : 0.008442878723144531 nb_pixel_total : 327607 time to create 1 rle with new method : 0.024693012237548828 length of segment : 1075 time for calcul the mask position with numpy : 0.0026450157165527344 nb_pixel_total : 126932 time to create 1 rle with old method : 0.14227628707885742 length of segment : 380 time for calcul the mask position with numpy : 0.003656625747680664 nb_pixel_total : 182616 time to create 1 rle with new method : 0.010066986083984375 length of segment : 554 time for calcul the mask position with numpy : 0.013107538223266602 nb_pixel_total : 411175 time to create 1 rle with new method : 0.04278969764709473 length of segment : 1751 time for calcul the mask position with numpy : 0.0012166500091552734 nb_pixel_total : 55646 time to create 1 rle with old method : 0.059749603271484375 length of segment : 402 time for calcul the mask position with numpy : 0.0019197463989257812 nb_pixel_total : 81982 time to create 1 rle with old method : 0.08992362022399902 length of segment : 356 time for calcul the mask position with numpy : 0.00037741661071777344 nb_pixel_total : 12874 time to create 1 rle with old method : 0.01446986198425293 length of segment : 183 time for calcul the mask position with numpy : 0.0021076202392578125 nb_pixel_total : 74038 time to create 1 rle with old method : 0.0836935043334961 length of segment : 241 time for calcul the mask position with numpy : 0.008049964904785156 nb_pixel_total : 245391 time to create 1 rle with new method : 0.026353120803833008 length of segment : 1237 time for calcul the mask position with numpy : 0.009417533874511719 nb_pixel_total : 367367 time to create 1 rle with new method : 0.02809762954711914 length of segment : 1451 time for calcul the mask position with numpy : 0.004053354263305664 nb_pixel_total : 160478 time to create 1 rle with new method : 0.014409780502319336 length of segment : 719 time for calcul the mask position with numpy : 0.0039048194885253906 nb_pixel_total : 120984 time to create 1 rle with old method : 0.13035035133361816 length of segment : 507 time for calcul the mask position with numpy : 0.0004444122314453125 nb_pixel_total : 21953 time to create 1 rle with old method : 0.024472951889038086 length of segment : 231 time for calcul the mask position with numpy : 0.002778768539428711 nb_pixel_total : 134757 time to create 1 rle with old method : 0.14602017402648926 length of segment : 819 time for calcul the mask position with numpy : 0.0022919178009033203 nb_pixel_total : 133253 time to create 1 rle with old method : 0.1470963954925537 length of segment : 382 time for calcul the mask position with numpy : 0.0009076595306396484 nb_pixel_total : 31905 time to create 1 rle with old method : 0.03801441192626953 length of segment : 156 time for calcul the mask position with numpy : 0.0003769397735595703 nb_pixel_total : 11623 time to create 1 rle with old method : 0.0134735107421875 length of segment : 174 time for calcul the mask position with numpy : 0.01117086410522461 nb_pixel_total : 465313 time to create 1 rle with new method : 0.03092479705810547 length of segment : 1409 time for calcul the mask position with numpy : 0.0002987384796142578 nb_pixel_total : 8206 time to create 1 rle with old method : 0.009470701217651367 length of segment : 121 time for calcul the mask position with numpy : 0.0012314319610595703 nb_pixel_total : 55513 time to create 1 rle with old method : 0.06264090538024902 length of segment : 247 time for calcul the mask position with numpy : 0.0013077259063720703 nb_pixel_total : 53741 time to create 1 rle with old method : 0.05955648422241211 length of segment : 254 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 9795 time to create 1 rle with old method : 0.011219978332519531 length of segment : 82 time for calcul the mask position with numpy : 0.003225564956665039 nb_pixel_total : 134668 time to create 1 rle with old method : 0.14878630638122559 length of segment : 475 time for calcul the mask position with numpy : 0.0026404857635498047 nb_pixel_total : 129894 time to create 1 rle with old method : 0.13653874397277832 length of segment : 396 time for calcul the mask position with numpy : 0.0016512870788574219 nb_pixel_total : 66911 time to create 1 rle with old method : 0.07292556762695312 length of segment : 193 time for calcul the mask position with numpy : 0.01350092887878418 nb_pixel_total : 344632 time to create 1 rle with new method : 0.04015946388244629 length of segment : 1291 time for calcul the mask position with numpy : 0.0007646083831787109 nb_pixel_total : 37115 time to create 1 rle with old method : 0.041222572326660156 length of segment : 232 time for calcul the mask position with numpy : 0.009424209594726562 nb_pixel_total : 313169 time to create 1 rle with new method : 0.028177499771118164 length of segment : 621 time for calcul the mask position with numpy : 0.0007908344268798828 nb_pixel_total : 38993 time to create 1 rle with old method : 0.043060302734375 length of segment : 235 time for calcul the mask position with numpy : 0.0019512176513671875 nb_pixel_total : 111909 time to create 1 rle with old method : 0.1271381378173828 length of segment : 330 time for calcul the mask position with numpy : 0.0018870830535888672 nb_pixel_total : 47988 time to create 1 rle with old method : 0.06235146522521973 length of segment : 300 time for calcul the mask position with numpy : 0.0006372928619384766 nb_pixel_total : 31409 time to create 1 rle with old method : 0.03564763069152832 length of segment : 182 time for calcul the mask position with numpy : 0.0021724700927734375 nb_pixel_total : 105980 time to create 1 rle with old method : 0.12310290336608887 length of segment : 615 time for calcul the mask position with numpy : 0.0020608901977539062 nb_pixel_total : 86214 time to create 1 rle with old method : 0.0967717170715332 length of segment : 297 time for calcul the mask position with numpy : 0.0007233619689941406 nb_pixel_total : 31559 time to create 1 rle with old method : 0.03570055961608887 length of segment : 183 time for calcul the mask position with numpy : 0.008866548538208008 nb_pixel_total : 440778 time to create 1 rle with new method : 0.02230215072631836 length of segment : 640 time for calcul the mask position with numpy : 0.0016856193542480469 nb_pixel_total : 87307 time to create 1 rle with old method : 0.09523296356201172 length of segment : 302 time for calcul the mask position with numpy : 0.22110795974731445 nb_pixel_total : 4247010 time to create 1 rle with new method : 0.5256662368774414 length of segment : 1676 time for calcul the mask position with numpy : 0.00135040283203125 nb_pixel_total : 116238 time to create 1 rle with old method : 0.13570117950439453 length of segment : 346 time for calcul the mask position with numpy : 0.0006353855133056641 nb_pixel_total : 46840 time to create 1 rle with old method : 0.05648946762084961 length of segment : 204 time for calcul the mask position with numpy : 0.0009667873382568359 nb_pixel_total : 23983 time to create 1 rle with old method : 0.028254270553588867 length of segment : 470 time for calcul the mask position with numpy : 0.5957887172698975 nb_pixel_total : 4859056 time to create 1 rle with new method : 0.5372726917266846 length of segment : 2846 time for calcul the mask position with numpy : 0.0009865760803222656 nb_pixel_total : 52644 time to create 1 rle with old method : 0.05673789978027344 length of segment : 487 time for calcul the mask position with numpy : 0.0002148151397705078 nb_pixel_total : 12258 time to create 1 rle with old method : 0.013379096984863281 length of segment : 93 time for calcul the mask position with numpy : 0.0015192031860351562 nb_pixel_total : 95905 time to create 1 rle with old method : 0.10087895393371582 length of segment : 512 time for calcul the mask position with numpy : 0.002627134323120117 nb_pixel_total : 100322 time to create 1 rle with old method : 0.10960650444030762 length of segment : 504 time spent for convertir_results : 25.555020093917847 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 208 chid ids of type : 4854 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 0 save missing photos in datou_result : time spend for datou_step_exec : 74.50141596794128 time spend to save output : 0.4210798740386963 total time spend for step 2 : 74.92249584197998 step3:blur_detection Thu Sep 4 22:03:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: variance laplacian (3120, 3120) 1395.75771072317 (3120, 3120) 697.7327471185595 (3120, 3120) 481.29597273351766 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.03272891044616699 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.03792595863342285 save missing photos in datou_result : time spend for datou_step_exec : 0.4656503200531006 time spend to save output : 0.09113931655883789 total time spend for step 3 : 0.5567896366119385 step4:brightness Thu Sep 4 22:03:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1757016088_2024502_1381530276_f0b8076b33dc8b0a5b93ddbc1bdd9273.jpg treat image : temp/1757016088_2024502_1381530250_b8ec7529ffd865bbbff5d142d95ece1b.jpg treat image : temp/1757016088_2024502_1381530225_5142dd925c1f5cc0ccd768ca4d74e35b.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 3 time used for this insertion : 0.03931689262390137 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 3 time used for this insertion : 0.039069175720214844 save missing photos in datou_result : time spend for datou_step_exec : 3.1257002353668213 time spend to save output : 0.09518933296203613 total time spend for step 4 : 3.2208895683288574 step5:rle_unique_nms_with_priority Thu Sep 4 22:03:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 208 chid ids of type : 4854 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 85 nb_hashtags : 10 time to prepare the origin masks : 43.55379939079285 time for calcul the mask position with numpy : 0.08941078186035156 nb_pixel_total : 2022188 time to create 1 rle with new method : 1.2389721870422363 time for calcul the mask position with numpy : 0.03928017616271973 nb_pixel_total : 38415 time to create 1 rle with old method : 0.04129838943481445 time for calcul the mask position with numpy : 0.04061317443847656 nb_pixel_total : 472 time to create 1 rle with old method : 0.0007367134094238281 time for calcul the mask position with numpy : 0.03965139389038086 nb_pixel_total : 113095 time to create 1 rle with old method : 0.129119873046875 time for calcul the mask position with numpy : 0.03831648826599121 nb_pixel_total : 3724 time to create 1 rle with old method : 0.0043904781341552734 time for calcul the mask position with numpy : 0.04341292381286621 nb_pixel_total : 269170 time to create 1 rle with new method : 0.6720597743988037 time for calcul the mask position with numpy : 0.04347348213195801 nb_pixel_total : 104738 time to create 1 rle with old method : 0.1178898811340332 time for calcul the mask position with numpy : 0.04184603691101074 nb_pixel_total : 122725 time to create 1 rle with old method : 0.13590478897094727 time for calcul the mask position with numpy : 0.03875231742858887 nb_pixel_total : 18201 time to create 1 rle with old method : 0.019995689392089844 time for calcul the mask position with numpy : 0.03998517990112305 nb_pixel_total : 15036 time to create 1 rle with old method : 0.016573190689086914 time for calcul the mask position with numpy : 0.03909730911254883 nb_pixel_total : 5171 time to create 1 rle with old method : 0.006283283233642578 time for calcul the mask position with numpy : 0.03912019729614258 nb_pixel_total : 51686 time to create 1 rle with old method : 0.0541841983795166 time for calcul the mask position with numpy : 0.03854060173034668 nb_pixel_total : 895 time to create 1 rle with old method : 0.001341104507446289 time for calcul the mask position with numpy : 0.039543867111206055 nb_pixel_total : 54897 time to create 1 rle with old method : 0.05991935729980469 time for calcul the mask position with numpy : 0.038190364837646484 nb_pixel_total : 67049 time to create 1 rle with old method : 0.07251572608947754 time for calcul the mask position with numpy : 0.04057121276855469 nb_pixel_total : 274813 time to create 1 rle with new method : 1.0360562801361084 time for calcul the mask position with numpy : 0.038969993591308594 nb_pixel_total : 21647 time to create 1 rle with old method : 0.02359771728515625 time for calcul the mask position with numpy : 0.03934526443481445 nb_pixel_total : 20551 time to create 1 rle with old method : 0.022303104400634766 time for calcul the mask position with numpy : 0.03936457633972168 nb_pixel_total : 1468 time to create 1 rle with old method : 0.0017387866973876953 time for calcul the mask position with numpy : 0.04080796241760254 nb_pixel_total : 68957 time to create 1 rle with old method : 0.07955551147460938 time for calcul the mask position with numpy : 0.040137529373168945 nb_pixel_total : 43067 time to create 1 rle with old method : 0.047920942306518555 time for calcul the mask position with numpy : 0.04000043869018555 nb_pixel_total : 2538 time to create 1 rle with old method : 0.002890348434448242 time for calcul the mask position with numpy : 0.039624929428100586 nb_pixel_total : 78888 time to create 1 rle with old method : 0.09741592407226562 time for calcul the mask position with numpy : 0.04599452018737793 nb_pixel_total : 19610 time to create 1 rle with old method : 0.024519681930541992 time for calcul the mask position with numpy : 0.0460207462310791 nb_pixel_total : 615221 time to create 1 rle with new method : 0.9997644424438477 time for calcul the mask position with numpy : 0.03959465026855469 nb_pixel_total : 41901 time to create 1 rle with old method : 0.04569649696350098 time for calcul the mask position with numpy : 0.04088163375854492 nb_pixel_total : 256963 time to create 1 rle with new method : 0.7810463905334473 time for calcul the mask position with numpy : 0.03916168212890625 nb_pixel_total : 2122 time to create 1 rle with old method : 0.0026695728302001953 time for calcul the mask position with numpy : 0.039295196533203125 nb_pixel_total : 69139 time to create 1 rle with old method : 0.07674741744995117 time for calcul the mask position with numpy : 0.03836560249328613 nb_pixel_total : 33279 time to create 1 rle with old method : 0.03429889678955078 time for calcul the mask position with numpy : 0.038457632064819336 nb_pixel_total : 205170 time to create 1 rle with new method : 0.5299961566925049 time for calcul the mask position with numpy : 0.040006160736083984 nb_pixel_total : 20630 time to create 1 rle with old method : 0.02298593521118164 time for calcul the mask position with numpy : 0.04137372970581055 nb_pixel_total : 26660 time to create 1 rle with old method : 0.028820514678955078 time for calcul the mask position with numpy : 0.03817248344421387 nb_pixel_total : 34836 time to create 1 rle with old method : 0.03550887107849121 time for calcul the mask position with numpy : 0.037950754165649414 nb_pixel_total : 30362 time to create 1 rle with old method : 0.03322148323059082 time for calcul the mask position with numpy : 0.03847360610961914 nb_pixel_total : 136749 time to create 1 rle with old method : 0.16913938522338867 time for calcul the mask position with numpy : 0.04133152961730957 nb_pixel_total : 187416 time to create 1 rle with new method : 1.0135459899902344 time for calcul the mask position with numpy : 0.037842512130737305 nb_pixel_total : 49750 time to create 1 rle with old method : 0.05576753616333008 time for calcul the mask position with numpy : 0.03792929649353027 nb_pixel_total : 5211 time to create 1 rle with old method : 0.0056209564208984375 time for calcul the mask position with numpy : 0.03687858581542969 nb_pixel_total : 10377 time to create 1 rle with old method : 0.010806083679199219 time for calcul the mask position with numpy : 0.038289785385131836 nb_pixel_total : 30502 time to create 1 rle with old method : 0.03310084342956543 time for calcul the mask position with numpy : 0.039444684982299805 nb_pixel_total : 94022 time to create 1 rle with old method : 0.09915280342102051 time for calcul the mask position with numpy : 0.03850436210632324 nb_pixel_total : 143762 time to create 1 rle with old method : 0.17046666145324707 time for calcul the mask position with numpy : 0.03922605514526367 nb_pixel_total : 4509 time to create 1 rle with old method : 0.0054476261138916016 time for calcul the mask position with numpy : 0.03945636749267578 nb_pixel_total : 10603 time to create 1 rle with old method : 0.012032508850097656 time for calcul the mask position with numpy : 0.041754722595214844 nb_pixel_total : 134145 time to create 1 rle with old method : 0.29363346099853516 time for calcul the mask position with numpy : 0.05179023742675781 nb_pixel_total : 31252 time to create 1 rle with old method : 0.05640888214111328 time for calcul the mask position with numpy : 0.048895835876464844 nb_pixel_total : 26687 time to create 1 rle with old method : 0.030093669891357422 time for calcul the mask position with numpy : 0.04085278511047363 nb_pixel_total : 2712 time to create 1 rle with old method : 0.0032367706298828125 time for calcul the mask position with numpy : 0.04075121879577637 nb_pixel_total : 26718 time to create 1 rle with old method : 0.03161954879760742 time for calcul the mask position with numpy : 0.04099583625793457 nb_pixel_total : 63689 time to create 1 rle with old method : 0.07166647911071777 time for calcul the mask position with numpy : 0.04057621955871582 nb_pixel_total : 59503 time to create 1 rle with old method : 0.06711888313293457 time for calcul the mask position with numpy : 0.04042458534240723 nb_pixel_total : 171 time to create 1 rle with old method : 0.0002243518829345703 time for calcul the mask position with numpy : 0.040830135345458984 nb_pixel_total : 107691 time to create 1 rle with old method : 0.11703276634216309 time for calcul the mask position with numpy : 0.04120016098022461 nb_pixel_total : 324739 time to create 1 rle with new method : 1.1791644096374512 time for calcul the mask position with numpy : 0.04346323013305664 nb_pixel_total : 715 time to create 1 rle with old method : 0.0009169578552246094 time for calcul the mask position with numpy : 0.039037466049194336 nb_pixel_total : 169 time to create 1 rle with old method : 0.00019359588623046875 time for calcul the mask position with numpy : 0.039372920989990234 nb_pixel_total : 28010 time to create 1 rle with old method : 0.031060457229614258 time for calcul the mask position with numpy : 0.047049522399902344 nb_pixel_total : 10690 time to create 1 rle with old method : 0.011730194091796875 time for calcul the mask position with numpy : 0.03920459747314453 nb_pixel_total : 133464 time to create 1 rle with old method : 0.14436864852905273 time for calcul the mask position with numpy : 0.03957247734069824 nb_pixel_total : 6861 time to create 1 rle with old method : 0.007524013519287109 time for calcul the mask position with numpy : 0.042177438735961914 nb_pixel_total : 15594 time to create 1 rle with old method : 0.01694011688232422 time for calcul the mask position with numpy : 0.0393071174621582 nb_pixel_total : 27701 time to create 1 rle with old method : 0.03099203109741211 time for calcul the mask position with numpy : 0.03989362716674805 nb_pixel_total : 18157 time to create 1 rle with old method : 0.020534992218017578 time for calcul the mask position with numpy : 0.04717087745666504 nb_pixel_total : 1361584 time to create 1 rle with new method : 1.0143308639526367 time for calcul the mask position with numpy : 0.039357900619506836 nb_pixel_total : 30221 time to create 1 rle with old method : 0.03275346755981445 time for calcul the mask position with numpy : 0.043173789978027344 nb_pixel_total : 679851 time to create 1 rle with new method : 0.7829139232635498 time for calcul the mask position with numpy : 0.039646148681640625 nb_pixel_total : 114205 time to create 1 rle with old method : 0.12197279930114746 time for calcul the mask position with numpy : 0.038691043853759766 nb_pixel_total : 2710 time to create 1 rle with old method : 0.003171682357788086 time for calcul the mask position with numpy : 0.03879570960998535 nb_pixel_total : 1398 time to create 1 rle with old method : 0.0017180442810058594 time for calcul the mask position with numpy : 0.03866147994995117 nb_pixel_total : 16953 time to create 1 rle with old method : 0.0184171199798584 time for calcul the mask position with numpy : 0.039229631423950195 nb_pixel_total : 79796 time to create 1 rle with old method : 0.08554768562316895 time for calcul the mask position with numpy : 0.03962111473083496 nb_pixel_total : 729 time to create 1 rle with old method : 0.0010781288146972656 time for calcul the mask position with numpy : 0.04159808158874512 nb_pixel_total : 52251 time to create 1 rle with old method : 0.057166337966918945 time for calcul the mask position with numpy : 0.041123151779174805 nb_pixel_total : 5831 time to create 1 rle with old method : 0.0068323612213134766 time for calcul the mask position with numpy : 0.03845381736755371 nb_pixel_total : 56562 time to create 1 rle with old method : 0.06312918663024902 time for calcul the mask position with numpy : 0.03982114791870117 nb_pixel_total : 13309 time to create 1 rle with old method : 0.01491689682006836 time for calcul the mask position with numpy : 0.03887462615966797 nb_pixel_total : 16981 time to create 1 rle with old method : 0.01964592933654785 time for calcul the mask position with numpy : 0.044052839279174805 nb_pixel_total : 411715 time to create 1 rle with new method : 0.6841015815734863 time for calcul the mask position with numpy : 0.04114413261413574 nb_pixel_total : 164301 time to create 1 rle with new method : 0.8607993125915527 time for calcul the mask position with numpy : 0.04053139686584473 nb_pixel_total : 96399 time to create 1 rle with old method : 0.10594344139099121 time for calcul the mask position with numpy : 0.03990936279296875 nb_pixel_total : 52119 time to create 1 rle with old method : 0.056104421615600586 time for calcul the mask position with numpy : 0.03914165496826172 nb_pixel_total : 4768 time to create 1 rle with old method : 0.005372524261474609 time for calcul the mask position with numpy : 0.039705753326416016 nb_pixel_total : 46066 time to create 1 rle with old method : 0.05033683776855469 time for calcul the mask position with numpy : 0.03902029991149902 nb_pixel_total : 14684 time to create 1 rle with old method : 0.016457319259643555 time for calcul the mask position with numpy : 0.03945660591125488 nb_pixel_total : 65114 time to create 1 rle with old method : 0.07082819938659668 create new chi : 18.16399574279785 time to delete rle : 0.07062697410583496 batch 1 Loaded 86 chid ids of type : 4855 Number RLEs to save : 52631 TO DO : save crop sub photo not yet done ! save time : 5.4629411697387695 nb_obj : 97 nb_hashtags : 11 time to prepare the origin masks : 38.96291208267212 time for calcul the mask position with numpy : 0.06136488914489746 nb_pixel_total : 3087585 time to create 1 rle with new method : 1.647606372833252 time for calcul the mask position with numpy : 0.03950238227844238 nb_pixel_total : 110513 time to create 1 rle with old method : 0.1256880760192871 time for calcul the mask position with numpy : 0.0461118221282959 nb_pixel_total : 76890 time to create 1 rle with old method : 0.09675312042236328 time for calcul the mask position with numpy : 0.03896069526672363 nb_pixel_total : 78208 time to create 1 rle with old method : 0.08564281463623047 time for calcul the mask position with numpy : 0.04051351547241211 nb_pixel_total : 27867 time to create 1 rle with old method : 0.03280782699584961 time for calcul the mask position with numpy : 0.03907656669616699 nb_pixel_total : 126761 time to create 1 rle with old method : 0.13582801818847656 time for calcul the mask position with numpy : 0.03827667236328125 nb_pixel_total : 6609 time to create 1 rle with old method : 0.007551431655883789 time for calcul the mask position with numpy : 0.038307905197143555 nb_pixel_total : 3890 time to create 1 rle with old method : 0.004332542419433594 time for calcul the mask position with numpy : 0.03903770446777344 nb_pixel_total : 167716 time to create 1 rle with new method : 0.9658782482147217 time for calcul the mask position with numpy : 0.03918886184692383 nb_pixel_total : 66653 time to create 1 rle with old method : 0.07273602485656738 time for calcul the mask position with numpy : 0.03934812545776367 nb_pixel_total : 15315 time to create 1 rle with old method : 0.016841411590576172 time for calcul the mask position with numpy : 0.04204678535461426 nb_pixel_total : 852 time to create 1 rle with old method : 0.0010614395141601562 time for calcul the mask position with numpy : 0.040125370025634766 nb_pixel_total : 137664 time to create 1 rle with old method : 0.1585400104522705 time for calcul the mask position with numpy : 0.04558825492858887 nb_pixel_total : 139204 time to create 1 rle with old method : 0.15532994270324707 time for calcul the mask position with numpy : 0.038991451263427734 nb_pixel_total : 9682 time to create 1 rle with old method : 0.010697364807128906 time for calcul the mask position with numpy : 0.03874945640563965 nb_pixel_total : 31656 time to create 1 rle with old method : 0.03474164009094238 time for calcul the mask position with numpy : 0.0390627384185791 nb_pixel_total : 134992 time to create 1 rle with old method : 0.1439194679260254 time for calcul the mask position with numpy : 0.03952789306640625 nb_pixel_total : 306993 time to create 1 rle with new method : 0.8448488712310791 time for calcul the mask position with numpy : 0.03940463066101074 nb_pixel_total : 58865 time to create 1 rle with old method : 0.0668487548828125 time for calcul the mask position with numpy : 0.03870391845703125 nb_pixel_total : 160632 time to create 1 rle with new method : 1.240506649017334 time for calcul the mask position with numpy : 0.04228687286376953 nb_pixel_total : 320834 time to create 1 rle with new method : 1.324709177017212 time for calcul the mask position with numpy : 0.04024004936218262 nb_pixel_total : 2728 time to create 1 rle with old method : 0.0030524730682373047 time for calcul the mask position with numpy : 0.040010929107666016 nb_pixel_total : 34518 time to create 1 rle with old method : 0.03812813758850098 time for calcul the mask position with numpy : 0.03969717025756836 nb_pixel_total : 32 time to create 1 rle with old method : 8.034706115722656e-05 time for calcul the mask position with numpy : 0.03986048698425293 nb_pixel_total : 18712 time to create 1 rle with old method : 0.02106499671936035 time for calcul the mask position with numpy : 0.03977704048156738 nb_pixel_total : 66066 time to create 1 rle with old method : 0.07309317588806152 time for calcul the mask position with numpy : 0.040467023849487305 nb_pixel_total : 125359 time to create 1 rle with old method : 0.13738083839416504 time for calcul the mask position with numpy : 0.03955960273742676 nb_pixel_total : 12807 time to create 1 rle with old method : 0.014136314392089844 time for calcul the mask position with numpy : 0.039411067962646484 nb_pixel_total : 40838 time to create 1 rle with old method : 0.044293880462646484 time for calcul the mask position with numpy : 0.03877878189086914 nb_pixel_total : 44409 time to create 1 rle with old method : 0.052506446838378906 time for calcul the mask position with numpy : 0.040558576583862305 nb_pixel_total : 24213 time to create 1 rle with old method : 0.027324199676513672 time for calcul the mask position with numpy : 0.03962421417236328 nb_pixel_total : 231701 time to create 1 rle with new method : 1.279987096786499 time for calcul the mask position with numpy : 0.039426565170288086 nb_pixel_total : 20866 time to create 1 rle with old method : 0.022667884826660156 time for calcul the mask position with numpy : 0.0393373966217041 nb_pixel_total : 36796 time to create 1 rle with old method : 0.039404869079589844 time for calcul the mask position with numpy : 0.039064884185791016 nb_pixel_total : 45540 time to create 1 rle with old method : 0.06916594505310059 time for calcul the mask position with numpy : 0.04486513137817383 nb_pixel_total : 102669 time to create 1 rle with old method : 0.11665153503417969 time for calcul the mask position with numpy : 0.03919386863708496 nb_pixel_total : 2017 time to create 1 rle with old method : 0.0023233890533447266 time for calcul the mask position with numpy : 0.03910970687866211 nb_pixel_total : 84201 time to create 1 rle with old method : 0.09157657623291016 time for calcul the mask position with numpy : 0.0400996208190918 nb_pixel_total : 16267 time to create 1 rle with old method : 0.018234968185424805 time for calcul the mask position with numpy : 0.03988909721374512 nb_pixel_total : 30919 time to create 1 rle with old method : 0.033675432205200195 time for calcul the mask position with numpy : 0.03781723976135254 nb_pixel_total : 25202 time to create 1 rle with old method : 0.026578426361083984 time for calcul the mask position with numpy : 0.03929424285888672 nb_pixel_total : 138309 time to create 1 rle with old method : 0.1645214557647705 time for calcul the mask position with numpy : 0.04474472999572754 nb_pixel_total : 73089 time to create 1 rle with old method : 0.07879257202148438 time for calcul the mask position with numpy : 0.039232730865478516 nb_pixel_total : 11139 time to create 1 rle with old method : 0.012239456176757812 time for calcul the mask position with numpy : 0.03883075714111328 nb_pixel_total : 66328 time to create 1 rle with old method : 0.07431435585021973 time for calcul the mask position with numpy : 0.04045438766479492 nb_pixel_total : 19035 time to create 1 rle with old method : 0.021355867385864258 time for calcul the mask position with numpy : 0.03949904441833496 nb_pixel_total : 2936 time to create 1 rle with old method : 0.003321409225463867 time for calcul the mask position with numpy : 0.03851771354675293 nb_pixel_total : 29630 time to create 1 rle with old method : 0.0319828987121582 time for calcul the mask position with numpy : 0.03957653045654297 nb_pixel_total : 21001 time to create 1 rle with old method : 0.023201942443847656 time for calcul the mask position with numpy : 0.039937496185302734 nb_pixel_total : 159582 time to create 1 rle with new method : 0.6765093803405762 time for calcul the mask position with numpy : 0.04107093811035156 nb_pixel_total : 314434 time to create 1 rle with new method : 0.9373960494995117 time for calcul the mask position with numpy : 0.039824485778808594 nb_pixel_total : 64704 time to create 1 rle with old method : 0.07038712501525879 time for calcul the mask position with numpy : 0.038872480392456055 nb_pixel_total : 84126 time to create 1 rle with old method : 0.09323310852050781 time for calcul the mask position with numpy : 0.03970170021057129 nb_pixel_total : 55568 time to create 1 rle with old method : 0.06057095527648926 time for calcul the mask position with numpy : 0.04004669189453125 nb_pixel_total : 145915 time to create 1 rle with old method : 0.16032671928405762 time for calcul the mask position with numpy : 0.039598703384399414 nb_pixel_total : 9979 time to create 1 rle with old method : 0.011375188827514648 time for calcul the mask position with numpy : 0.03831028938293457 nb_pixel_total : 22301 time to create 1 rle with old method : 0.02361774444580078 time for calcul the mask position with numpy : 0.037513017654418945 nb_pixel_total : 18896 time to create 1 rle with old method : 0.019707441329956055 time for calcul the mask position with numpy : 0.03760409355163574 nb_pixel_total : 14474 time to create 1 rle with old method : 0.015464305877685547 time for calcul the mask position with numpy : 0.03837728500366211 nb_pixel_total : 135429 time to create 1 rle with old method : 0.14180231094360352 time for calcul the mask position with numpy : 0.039197444915771484 nb_pixel_total : 1112 time to create 1 rle with old method : 0.0014128684997558594 time for calcul the mask position with numpy : 0.0388946533203125 nb_pixel_total : 22000 time to create 1 rle with old method : 0.0240478515625 time for calcul the mask position with numpy : 0.03993725776672363 nb_pixel_total : 17323 time to create 1 rle with old method : 0.019320011138916016 time for calcul the mask position with numpy : 0.03974270820617676 nb_pixel_total : 1156 time to create 1 rle with old method : 0.0014941692352294922 time for calcul the mask position with numpy : 0.03941059112548828 nb_pixel_total : 36674 time to create 1 rle with old method : 0.03995966911315918 time for calcul the mask position with numpy : 0.037932395935058594 nb_pixel_total : 60555 time to create 1 rle with old method : 0.06552600860595703 time for calcul the mask position with numpy : 0.03906559944152832 nb_pixel_total : 154680 time to create 1 rle with new method : 0.7881486415863037 time for calcul the mask position with numpy : 0.038626670837402344 nb_pixel_total : 34161 time to create 1 rle with old method : 0.03649187088012695 time for calcul the mask position with numpy : 0.03810882568359375 nb_pixel_total : 22166 time to create 1 rle with old method : 0.02418041229248047 time for calcul the mask position with numpy : 0.03840494155883789 nb_pixel_total : 11423 time to create 1 rle with old method : 0.012291669845581055 time for calcul the mask position with numpy : 0.038336992263793945 nb_pixel_total : 54435 time to create 1 rle with old method : 0.05822563171386719 time for calcul the mask position with numpy : 0.03894495964050293 nb_pixel_total : 12845 time to create 1 rle with old method : 0.013630151748657227 time for calcul the mask position with numpy : 0.03815484046936035 nb_pixel_total : 60931 time to create 1 rle with old method : 0.06645894050598145 time for calcul the mask position with numpy : 0.03830742835998535 nb_pixel_total : 3245 time to create 1 rle with old method : 0.0036118030548095703 time for calcul the mask position with numpy : 0.0375363826751709 nb_pixel_total : 38501 time to create 1 rle with old method : 0.04009604454040527 time for calcul the mask position with numpy : 0.037941932678222656 nb_pixel_total : 55337 time to create 1 rle with old method : 0.058034658432006836 time for calcul the mask position with numpy : 0.03822803497314453 nb_pixel_total : 67532 time to create 1 rle with old method : 0.0702664852142334 time for calcul the mask position with numpy : 0.039292335510253906 nb_pixel_total : 180184 time to create 1 rle with new method : 0.8257884979248047 time for calcul the mask position with numpy : 0.039377450942993164 nb_pixel_total : 41453 time to create 1 rle with old method : 0.04465818405151367 time for calcul the mask position with numpy : 0.03833198547363281 nb_pixel_total : 25339 time to create 1 rle with old method : 0.027714967727661133 time for calcul the mask position with numpy : 0.03920459747314453 nb_pixel_total : 182230 time to create 1 rle with new method : 0.696131706237793 time for calcul the mask position with numpy : 0.03804135322570801 nb_pixel_total : 47272 time to create 1 rle with old method : 0.04914975166320801 time for calcul the mask position with numpy : 0.037480831146240234 nb_pixel_total : 17487 time to create 1 rle with old method : 0.01851177215576172 time for calcul the mask position with numpy : 0.03800153732299805 nb_pixel_total : 5935 time to create 1 rle with old method : 0.007156848907470703 time for calcul the mask position with numpy : 0.043279170989990234 nb_pixel_total : 492369 time to create 1 rle with new method : 0.8995096683502197 time for calcul the mask position with numpy : 0.038079023361206055 nb_pixel_total : 21141 time to create 1 rle with old method : 0.023018836975097656 time for calcul the mask position with numpy : 0.038294076919555664 nb_pixel_total : 238381 time to create 1 rle with new method : 0.7457554340362549 time for calcul the mask position with numpy : 0.03908824920654297 nb_pixel_total : 6291 time to create 1 rle with old method : 0.007115840911865234 time for calcul the mask position with numpy : 0.03931427001953125 nb_pixel_total : 11086 time to create 1 rle with old method : 0.012284994125366211 time for calcul the mask position with numpy : 0.03905010223388672 nb_pixel_total : 129063 time to create 1 rle with old method : 0.1364731788635254 time for calcul the mask position with numpy : 0.03849196434020996 nb_pixel_total : 156189 time to create 1 rle with new method : 0.49746012687683105 time for calcul the mask position with numpy : 0.03876662254333496 nb_pixel_total : 425 time to create 1 rle with old method : 0.0005559921264648438 time for calcul the mask position with numpy : 0.03770256042480469 nb_pixel_total : 37562 time to create 1 rle with old method : 0.03892183303833008 time for calcul the mask position with numpy : 0.038030147552490234 nb_pixel_total : 12715 time to create 1 rle with old method : 0.013586044311523438 time for calcul the mask position with numpy : 0.03930258750915527 nb_pixel_total : 416 time to create 1 rle with old method : 0.0006165504455566406 time for calcul the mask position with numpy : 0.03880167007446289 nb_pixel_total : 1877 time to create 1 rle with old method : 0.002250194549560547 time for calcul the mask position with numpy : 0.03806877136230469 nb_pixel_total : 50460 time to create 1 rle with old method : 0.05342888832092285 time for calcul the mask position with numpy : 0.04015684127807617 nb_pixel_total : 333 time to create 1 rle with old method : 0.0004589557647705078 create new chi : 21.661386013031006 time to delete rle : 0.005919218063354492 batch 1 Loaded 98 chid ids of type : 4855 Number RLEs to save : 62505 TO DO : save crop sub photo not yet done ! save time : 4.810190200805664 nb_obj : 15 nb_hashtags : 7 time to prepare the origin masks : 27.217777013778687 time for calcul the mask position with numpy : 0.057370662689208984 nb_pixel_total : 763311 time to create 1 rle with new method : 0.7288215160369873 time for calcul the mask position with numpy : 0.04866838455200195 nb_pixel_total : 83411 time to create 1 rle with old method : 0.11280417442321777 time for calcul the mask position with numpy : 0.049875497817993164 nb_pixel_total : 825 time to create 1 rle with old method : 0.0010099411010742188 time for calcul the mask position with numpy : 0.04295921325683594 nb_pixel_total : 11272 time to create 1 rle with old method : 0.012262105941772461 time for calcul the mask position with numpy : 0.031662702560424805 nb_pixel_total : 25779 time to create 1 rle with old method : 0.028514623641967773 time for calcul the mask position with numpy : 0.0297393798828125 nb_pixel_total : 333 time to create 1 rle with old method : 0.00041985511779785156 time for calcul the mask position with numpy : 0.12212944030761719 nb_pixel_total : 1446387 time to create 1 rle with new method : 0.9489002227783203 time for calcul the mask position with numpy : 0.054308176040649414 nb_pixel_total : 179 time to create 1 rle with old method : 0.00031256675720214844 time for calcul the mask position with numpy : 0.05007147789001465 nb_pixel_total : 99238 time to create 1 rle with old method : 0.10860109329223633 time for calcul the mask position with numpy : 0.05533742904663086 nb_pixel_total : 23945 time to create 1 rle with old method : 0.02601766586303711 time for calcul the mask position with numpy : 0.05577969551086426 nb_pixel_total : 3919 time to create 1 rle with old method : 0.004543304443359375 time for calcul the mask position with numpy : 0.0543365478515625 nb_pixel_total : 92723 time to create 1 rle with old method : 0.10049843788146973 time for calcul the mask position with numpy : 0.11650562286376953 nb_pixel_total : 2812711 time to create 1 rle with new method : 1.0416672229766846 time for calcul the mask position with numpy : 0.031386613845825195 nb_pixel_total : 107049 time to create 1 rle with old method : 0.1166219711303711 time for calcul the mask position with numpy : 0.030458450317382812 nb_pixel_total : 20509 time to create 1 rle with old method : 0.02208232879638672 time for calcul the mask position with numpy : 0.1993999481201172 nb_pixel_total : 4242809 time to create 1 rle with new method : 0.8861351013183594 create new chi : 5.321710824966431 time to delete rle : 0.0020461082458496094 batch 1 Loaded 16 chid ids of type : 4855 Number RLEs to save : 18971 TO DO : save crop sub photo not yet done ! save time : 1.9035568237304688 map_output_result : {1381530276: (0.6806530483075267, 'Should be the crop_list due to order', 0.5633071826272015), 1381530250: (0.6806530483075267, 'Should be the crop_list due to order', 0.5694948699536152), 1381530225: (0.6806530483075267, 'Should be the crop_list due to order', 0.9091570923417636)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 3 /1381530276.Didn't retrieve data . /1381530250.Didn't retrieve data . /1381530225.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.03325080871582031 save_final save missing photos in datou_result : time spend for datou_step_exec : 167.93493103981018 time spend to save output : 0.03685903549194336 total time spend for step 5 : 167.97179007530212 step6:crop_condition Thu Sep 4 22:06:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4855 Loading chi in step crop for list_pids : 3 ! batch 1 Loaded 200 chid ids of type : 4855 begin to crop the class : papier param for this class : {'min_score': 0.1} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! About to upload 21 photos upload in portfolio : 3015255 init cache_photo without model_param we have 21 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016395_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 21 photos in the portfolio 3015255 time of upload the photos Elapsed time : 7.482515811920166 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.1} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 30 About to insert : list_path_to_insert length 30 new photo from crops ! About to upload 30 photos upload in portfolio : 3015255 init cache_photo without model_param we have 30 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016415_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 30 photos in the portfolio 3015255 time of upload the photos Elapsed time : 12.955424785614014 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.1} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3015255 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016429_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.5241029262542725 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.1} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3015255 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016432_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.7089998722076416 we have finished the crop for the class : pet_clair begin to crop the class : pehd param for this class : {'min_score': 0.1} filtre for class : pehd hashtag_id of this class : 628944319 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016435_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.1030738353729248 we have finished the crop for the class : pehd begin to crop the class : pet_fonce param for this class : {'min_score': 0.1} filtre for class : pet_fonce hashtag_id of this class : 2107755900 begin to crop the class : sac param for this class : {'min_score': 0.1} filtre for class : sac hashtag_id of this class : 492618350 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! About to upload 18 photos upload in portfolio : 3015255 init cache_photo without model_param we have 18 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016443_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 18 photos in the portfolio 3015255 time of upload the photos Elapsed time : 6.688940048217773 we have finished the crop for the class : sac begin to crop the class : textiles param for this class : {'min_score': 0.1} filtre for class : textiles hashtag_id of this class : 494789423 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 3015255 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016452_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 3 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.3430044651031494 we have finished the crop for the class : textiles begin to crop the class : verre param for this class : {'min_score': 0.1} filtre for class : verre hashtag_id of this class : 492631919 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 3015255 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016454_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 1 photos in the portfolio 3015255 time of upload the photos Elapsed time : 0.7357425689697266 we have finished the crop for the class : verre begin to crop the class : organique param for this class : {'min_score': 0.1} filtre for class : organique hashtag_id of this class : 530726390 begin to crop the class : dasri param for this class : {'min_score': 0.1} filtre for class : dasri hashtag_id of this class : 1608424396 begin to crop the class : masque param for this class : {'min_score': 0.1} filtre for class : masque hashtag_id of this class : 492626781 begin to crop the class : encombrant param for this class : {'min_score': 0.1} filtre for class : encombrant hashtag_id of this class : 605975549 begin to crop the class : autre param for this class : {'min_score': 0.1} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 29 About to insert : list_path_to_insert length 29 new photo from crops ! About to upload 29 photos upload in portfolio : 3015255 init cache_photo without model_param we have 29 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016480_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 29 photos in the portfolio 3015255 time of upload the photos Elapsed time : 12.229296922683716 we have finished the crop for the class : autre begin to crop the class : autre_emballage param for this class : {'min_score': 0.1} filtre for class : autre_emballage hashtag_id of this class : 2107760328 map_result returned by crop_photo_return_map_crop : length : 0 About to insert : list_path_to_insert length 0 new photo from crops ! About to upload 0 photos WARNING : list_path_to_insert is empty, cannot upload ! we have finished the crop for the class : autre_emballage begin to crop the class : autre_non_emballage param for this class : {'min_score': 0.1} filtre for class : autre_non_emballage hashtag_id of this class : 2107760329 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 3015255 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016495_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 4 photos in the portfolio 3015255 time of upload the photos Elapsed time : 1.722579002380371 we have finished the crop for the class : autre_non_emballage begin to crop the class : environnement param for this class : {'min_score': 0.1} filtre for class : environnement hashtag_id of this class : 493012381 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! About to upload 28 photos upload in portfolio : 3015255 init cache_photo without model_param we have 28 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1757016543_2024502 batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! batch_size : 0, verbose : False, strat_bulk_insert : ignore_different_from_first This is a hack ! we have uploaded 28 photos in the portfolio 3015255 time of upload the photos Elapsed time : 9.23636269569397 we have finished the crop for the class : environnement delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 142 /1381541627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541839Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541845Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541853Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541875Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541945Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381541962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381542999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1381543047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 429 time used for this insertion : 0.08167886734008789 save_final save missing photos in datou_result : time spend for datou_step_exec : 172.95681715011597 time spend to save output : 0.08495879173278809 total time spend for step 6 : 173.04177594184875 step7:ventilate_hashtags_in_portfolio Thu Sep 4 22:09:13 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 26545452 get user id for portfolio 26545452 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26545452 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26545452 AND mptpi.`type`=4855 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26545452 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26545452 AND mptpi.`type`=4856 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('papier','carton','metal','pet_clair','pehd','pet_fonce','pet_opaque','barquette_opaque','film_plastique','ela','sac','textiles','verre','organique','dasri','masque','encombrant','autre_emballage','autre_non_emballage','environnement','autre','flux_dev','mal_croppe','flou','film_dev_souple','sac_om_plein')) AND mptpi.`min_score`=0.1 To do lien utilise dans velours : https://marlene.fotonower.com/velours/26546831,26546832,26546833,26546834,26546835,26546840,26546842,26546843,26546844,26546845,26546846,26546847,26546849,26546850,26546851,26546853,26546854,26546856,26546857,26546858?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26545452 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 1 /26545452. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.037259817123413086 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.711066246032715 time spend to save output : 0.03746438026428223 total time spend for step 7 : 13.748530626296997 step8:final Thu Sep 4 22:09:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1381530276: ('0.5741268958732723',), 1381530250: ('0.5741268958732723',), 1381530225: ('0.5741268958732723',)} new output for save of step final : {1381530276: ('0.5741268958732723',), 1381530250: ('0.5741268958732723',), 1381530225: ('0.5741268958732723',)} [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 3 /1381530276.Didn't retrieve data . /1381530250.Didn't retrieve data . /1381530225.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.0390169620513916 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3223230838775635 time spend to save output : 0.07990407943725586 total time spend for step 8 : 0.40222716331481934 step9:velours_tree Thu Sep 4 22:09:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2914748191833496 time spend to save output : 6.508827209472656e-05 total time spend for step 9 : 0.29153990745544434 step10:send_mail_cod Thu Sep 4 22:09:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P26545452_04-09-2025_22_09_28.pdf 26546792 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265467921757016568 26546793 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265467931757016569 26546794 change filename to text .change filename to text .change filename to text .change filename to text .imagette265467941757016571 26546795 change filename to text .change filename to text .change filename to text .imagette265467951757016571 26546796 change filename to text .imagette265467961757016572 26546797 imagette265467971757016572 26546798 imagette265467981757016572 26546799 imagette265467991757016572 26546800 imagette265468001757016572 26546801 imagette265468011757016572 26546802 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265468021757016572 26546803 change filename to text .change filename to text .change filename to text .imagette265468031757016573 26546804 change filename to text .imagette265468041757016573 26546805 imagette265468051757016573 26546806 imagette265468061757016574 26546807 imagette265468071757016574 26546808 imagette265468081757016574 26546809 imagette265468091757016574 26546810 change filename to text .change filename to text .change filename to text .change filename to text .imagette265468101757016574 26546812 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette265468121757016574 26546813 imagette265468131757016576 26546814 imagette265468141757016576 26546815 imagette265468151757016576 26546816 imagette265468161757016576 26546817 imagette265468171757016576 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=26545452 and hashtag_type = 4855 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/26546831,26546832,26546833,26546834,26546835,26546840,26546842,26546843,26546844,26546845,26546846,26546847,26546849,26546850,26546851,26546853,26546854,26546856,26546857,26546858?tags=papier,carton,metal,pet_clair,pehd,ela,textiles,verre,organique,dasri,masque,encombrant,autre_non_emballage,environnement,autre,flux_dev,mal_croppe,flou,film_dev_souple,sac_om_plein&datou_id_consolidate=4742&port_consolidate=26545452 your option no_mail is active, we will not send the real mail to your client args[1381530276] : ((1381530276, 1395.75771072317, 2107751945), (1381530276, -0.12723543402301246, 496442774), '0.5741268958732723') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1381530250] : ((1381530250, 697.7327471185595, 492688767), (1381530250, -0.5593975796751679, 501862349), '0.5741268958732723') We are sending mail with results at report@fotonower.com,cod@fotonower.com args[1381530225] : ((1381530225, 481.29597273351766, 492688767), (1381530225, -1.8145850472691012, 501862349), '0.5741268958732723') We are sending mail with results at report@fotonower.com,cod@fotonower.com refus_total : 0.5741268958732723 2022-04-13 10:29:59 0 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26545452_04-09-2025_22_09_28.pdf results_Auto_P26545452_04-09-2025_22_09_28.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26545452_04-09-2025_22_09_28.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4741','26545452','results_Auto_P26545452_04-09-2025_22_09_28.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26545452_04-09-2025_22_09_28.pdf','pdf','','1.35','0.5741268958732723') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.03277945518493652 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.239831686019897 time spend to save output : 0.032933712005615234 total time spend for step 10 : 11.272765398025513 step11:split_time_score Thu Sep 4 22:09:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('50', 2), ('51', 1)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 04092025 26545452 Nombre de photos uploadées : 3 / 23040 (0%) 04092025 26545452 Nombre de photos taguées (types de déchets): 0 / 3 (0%) 04092025 26545452 Nombre de photos taguées (volume) : 0 / 3 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 7.867813110351562e-06 ??? elapsed_time : fill_and_build_computed_from_old_data 0.0002548694610595703 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.5824365615844727 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.5741268958732723 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P26545452_04-09-2025_22_09_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26545452 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13156 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13160 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13160 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13158 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13157 crop_condition is not consistent : 4 used against 2 in the step definition ! WARNING : number of outputs for step 13157 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13161 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13159 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13159 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13156 doesn't seem to be define in the database( WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13161 doesn't seem to be define in the database( WARNING : type of input 3 of step 13159 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13163 have datatype=6 whereas input 2 of step 13157 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 0 of step 13162 have datatype=6 whereas input 2 of step 13157 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13161 have datatype=10 whereas input 3 of step 13164 have datatype=6 WARNING : output 0 of step 13161 have datatype=10 whereas input 0 of step 13166 have datatype=18 WARNING : type of input 5 of step 13164 doesn't seem to be define in the database( WARNING : output 0 of step 13166 have datatype=11 whereas input 5 of step 13164 have datatype=None WARNING : type of input 2 of step 13157 doesn't seem to be define in the database( WARNING : output 1 of step 13158 have datatype=7 whereas input 2 of step 13157 have datatype=None WARNING : type of output 3 of step 13157 doesn't seem to be define in the database( WARNING : type of input 3 of step 13161 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13163 doesn't seem to be define in the database( WARNING : type of output 2 of step 13160 doesn't seem to be define in the database( WARNING : type of input 1 of step 13162 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=26545452 AND mptpi.`type`=4855 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 26545453 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'04092025': {'nb_upload': 3, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1381530276, 1381530250, 1381530225] Looping around the photos to save general results len do output : 1 /26545452Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530276', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530250', None, None, None, None, None, '3665904') ('4741', None, None, None, None, None, None, None, '3665904') ('4741', '26545452', '1381530225', None, None, None, None, None, '3665904') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 4 time used for this insertion : 0.03441452980041504 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.3768887519836426 time spend to save output : 0.034609079360961914 total time spend for step 11 : 2.4114978313446045 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 3 set_done_treatment 290.02user 130.69system 8:16.69elapsed 84%CPU (0avgtext+0avgdata 5487480maxresident)k 1236648inputs+289584outputs (19940major+20076067minor)pagefaults 0swaps